Internal:APBNRRMachineLearning

Machine Learning Paper Checklist

• Collect training data
• Categorize and parse training data for machine learning
  • Build program that parses APBNRR output data, categorizes it via human input, and constructs a CSV file for training, evaluation, or prediction
• Research neural network architectures
• Build neural network model with Keras and Tensorflow
• Train model with training data from up to 27 cases
• Evaluate the model with evaluation data from up to 3 cases
• Finalize paper

Feed-Forward Neural Network Architecture Construction Methods

There is generally no standard formula for constructing neural networks. Their architecture heavily varies from problem to problem. There are however ways to optimize a neural network.

Number of Hidden Layers

For most problems, one hidden layer is sufficient. Two hidden layers can model any function. More than two hidden layers are rare, but are used when a problem is very complex. Theoretically though, there is no reason to use more than two hidden layers.

Number of Neurons in Hidden Layers

1. Trial and Error

   The simplest method, where a different number of neurons in each hidden layer is tried successively to reach a lower error rate. There are two main ways of going about doing this:

   1. Forward Approach: Start with few neurons in each layer and progressively add more until a desirable error rate has been achieved.
   2. Backward Approach: Start with many neurons in each layer, and progressively remove neurons until a desirable error rate has been achieved.
2. Rules of Thumb

   There are three main rules of thumb for choosing the number of neurons in a hidden layer:

   1. The number of hidden neurons should be between the size of the input layer and the size of the output layer.
   2. The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.
   3. The number of hidden neurons should be less than twice the size of the input layer.
3. Pruning and Constructive Methods

   Pruning and constructive methods include several automatic algorithms for optimizing the size and speed of a neural network. These methods typically work by incrementally removing the least effective neurons. There are several pruning algorithms, and some of the more well-known are:

   1. Optimal Brain Damage: Works by incrementally removing the neuron that causes the least decrease of training error.
   2. Optimal Brain Surgeon: An improved version of Optimal Brain Damage. Works by incrementally removing the neuron that causes the least decrease of training error.
4. Grid Search

   Brute force different number of neurons. Not practical for many real world problems.

5. Neuroevolution and Genetic Algorithms

   Utilize evolutionary and genetic algorithms that can generate or optimize a neural network's topology and parameters. These methods are relatively newer than the previous ones.

Resources

http://dstath.users.uth.gr/papers/IJRS2009_Stathakis.pdf

https://www.ijcsmc.com/docs/papers/November2014/V3I11201499a19.pdf

https://arxiv.org/pdf/1711.07655.pdf

http://ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/