CNF:Machine Learning

This is the page for the CNF-Machine Learning project. It will be updated frequently with reports about the progress of the project.

The Problem

Traditional tracking algorithms are computationally intensive, especially for high luminosity experiments with multi-track final states where all combinations of segments in drift chambers have to be considered for firing best track candidates. At high luminosity the number of random segments (unrelated to the tracks) are increasing and as a result the number of possible combinations also increases, making the whole process longer.

The Goal

Using machine learning one can recognize the patterns that are valid in order to find the correct track faster. The model will be trained on real pre-labeled data and as an outcome, it will be able to label track combinations as valid or not.

The Data

A representation of the drift chambers. The red squares represent a sensor that has detected a hit

The drift chambers consist of 6 layers, of 6 wires, of 112 sensors each for a total of 4032 sensors (see picture on the right). The data provided let us know whether a sensor has detected a hit or not. Those detections might be part of the trajectory that we want to track or can be irrelevant (noise). The labeled data consist of all the possible combinations that form a track as rows (events) and the state of each sensor (detected something or not) as columns (features). The label provides information on whether a combination produces the valid track or not. An example of the input to be used by the model can be found in [1].

Timeframe

Week 1

Create the pipeline

• Load data in Spark Dataframes
• Split data in training/validation sets
• Experiment with different classification methods
  • Tree-based
  • SVM
• Train Model
• Validate
• Report accuracy and confusion matrix

Results for input with 4032 features

Method	Split	Accuracy	False Positives	False Negatives	Training Accuracy	Time to train (sec)	Time to predict (sec)
Majority Voting Ensemble	80/20	0.962	1.54	2.26	1.0
Multilayer Perceptron	80/20	0.953	2.17	2.5	1.0
Extremely Randomized Trees	80/20	0.952	3.96	0.77
k-Nearest Neighbors	80/20	0.933	0	6.4	0.97	2.5	0.02
Linear SVM	80/20	0.764	21	2.5	0.81	658	0.01
RBF SVM	80/20	0.724	27.6	0.0	1.0	3571	0.02
Gaussian Process	80/20	0.724	27.6	0.0	0.724	3484	0.01
Decision Tree	80/20	0.726	26.6	0	0.73	3.18	2.0e-06
Random Forest	80/20	0.724	27.6	0	0.724	0.2	3.2e-06
Neural Net	80/20	0.90	7.6	2.3	0.982	142	1.5-06
AdaBoost	80/20	0.725	24.6	2.9	0.734	48.5	0.0001
Gaussian Naive Bayes	80/20	0.353	0	64.3	0.355	1.1	1.8e-05
QDA	80/20	0.947	5.3	0	1.0	12	5.1e-05
Multilayer Perceptron	60/40	0.922	4	3.8
Extremely Randomized Trees	60/40	0.93	5.82	1.16
k-Nearest Neighbors	60/40	0.916	2.27	6.17
Linear SVM	60/40	0.756	22.2	2.1
RBF SVM	60/40	0.721	27.9	0
Gaussian Process	60/40	0.721	27.9	0
Decision Tree	60/40	0.723	26	1.64
Random Forest	60/40	0.721	27.9	0
Neural Net	60/40	0.9	7	2.95
AdaBoost	60/40	0.722	24.6	3.15
Gaussian Naive Bayes	60/40	0.355	0.59	63.9
QDA	60/40	0.758	22.6	1.58

Results for input with 36 features

Method	Split	Accuracy	False Positives %	False Negatives %	Training Accuracy
Multilayer Perceptron	N/A	0.905	3.4	6.1	0.963
k-Nearest Neighbors	80/20	0.853	6.8	7.9	0.919
Linear SVM	80/20	0.846	0	15.4	0.848
[[#RBF SVM |RBF SVM]	80/20	0.846	0	15.4	1.0
[[#Decision Tree | Decision Tree]	80/20	0.846	0	15.4	0.848
[[#Random Forest | Random Forest]	80/20	0.846	0	15.4	0.848
[[#Neural Net | Neural Net]	80/20	0.873	0.9	11.7	0.874
[[#AdaBoost | AdaBoost]	80/20	0.846	0	15.4	0.848
[[#Naive Bayes | Naive Bayes]	80/20	0.659	24.8	9.4	0.659
[[#QDR | QDR]	80/20	0.619	34.4	3.7	0.617