CNF:ML Data Denoise

This page will be updated about the machine learning data de-noising effort.

Problem

Traditional tracking algorithms are computationally intensive, especially for high luminosity experiments with multi-track final states where all combinations of segments in drift chambers have to be considered for firing best track candidates. At high luminosity the number of random segments (unrelated to the tracks) are increasing and as a result the number of possible combinations also increases, making the whole process longer.

Goal

Use Deep Learning Autoencoders that are trained on noisy tracks to produce the original track without the noise.

Results

Hardware specs:

Tesla V100 GPU

3D Input

Single Track

Example noisy input:	Example correct output:

4x6 Kernels

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case
20	0.000064	0.00007	7822	0.00087

Different Noise

Example noisy input:	Example correct output:

Noisy Correct Denoised

Noisy Correct Denoised

250 Tracks

Example noisy input:	Example correct output:

Data Augmentation

The given dataset for 250 tracks only consists of 1000 data points. This hurts the ability of the network to learn. To overcome this limitation we augment the dataset with synthetic data points by using random combinations of the 1 track dataset tracks.

Noisy Correct Denoised

Noisy Correct Denoised

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot
20	0.065	0.064	1300	0.000645

Z Plane Quantization

Quantization Process

1. Establish an array of Z ranges (must have an odd size)
2. The number of added planes between each base plane is equal to the truncated half size of the Z range array
3. Iterate hitlist activations
   1. Get the plane, ring, and pad numbers
   2. Offset the plane number to ignore added planes
   3. Offset the plane number by the difference of the center index of the Z ranges array from the index of the range the Z value belongs

No Quantization (10 planes)	Quantization (21 planes)	Quantization (32 planes)	Quantization (43 planes)
Z Ranges: {}	Z Ranges: {(-20, -1.5), (-1.5, 1.5), (1.5, 21)}	Z Ranges: {(-20, -10), (-10, -1.5), (-1.5, 1.5), (1.5, 10), (10, 21)}	Z Ranges: {(-20, -15), (-15, -10), (-10, -1.5), (-1.5, 1.5), (1.5, 10), (10, 15), (15, 21)}

Results

Noisy Correct Denoised

Noisy Correct Denoised

Epochs	Training loss	Validation loss	Training Time	Inference	Train/Val Loss Plot
10	0.021	0.022	1784	0.003

2D Input

Example noisy input:

Example correct input:

Both the MLP and CNN networks used relu activation for all hidden layers and sigmoid for the output. Binary cross-entropy was used as the loss function and Adam as the optimizer.

Multilayer Perceptrons

Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
0.0018	0.0219	298	0.000182

Convolutional Networks

Train and test on single track

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.0024	0.0024	238	0.000123
100	0.0023	0.0025	471.2	0.0001

Train and test on single track V2

4x6 Kernels

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
12	0.00048	0.00085	204	0.00024

Per segment histograms:

Train and test on multiple tracks V2

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
20	0.0029	0.0030	683	0.0002		File:Cnf ml cnn denoised mult track v2.png

Train on single and test on multiple tracks

Epochs	Validation loss	Inference Time	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.027	0.00012
100	0.024	0.000125

Train and test on multiple tracks

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.00537	0.00505		0.0001		File:Cnf ml cnn denoised mult track.png

Train and test on multiple tracks with more kernels

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.0041	0.0040	943	0.0001		File:Cnf ml cnn denoised mult track more.png

Train on multiple and test on 2-3 tracks

Correct Track:

Noisy Input Track:

Epochs	Validation loss	Inference Time	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.0078

Train on multiple and test on 2-3 tracks V2

Epochs	Validation loss	Inference Time	Example Case	Valid Hits Histogram	Noise Added Histogram
20	0.0010	0.0007

Train on new 2-3 tracks and test on multitrack V2

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
20	0.0043	0.003	517.2	0.0003	File:Cnn ae train val mult loss more V2 new multitrack.png	File:Cnf ml cnn denoised mult2 3 track V2 new multitrack.png

Train on new 2-3 tracks and test on old 2-3 tracks V2

Epochs	Training loss	Validation loss	Training Time	Inference Time	Train/Val Loss Plot	Example Case	Valid Hits Histogram	Noise Added Histogram
20	0.0043	0.0045	517.2	0.0003

Train on multiple and test on 2-3 tracks with more kernels

Epochs	Validation loss	Inference Time	Example Case	Valid Hits Histogram	Noise Added Histogram
50	0.0075	0.0008

Examples

Noisy	Correct	MLP	CNN	CNN V2	CNN V2 new
		File:Ae mlp denoised 1.png
		File:Ae mlp denoised 2.png
		File:Ae mlp denoised 3.png
		File:Ae mlp denoised 1m.png
		File:Ae mlp denoised 2m.png
		File:Ae mlp denoised 3m.png
		File:Ae mlp denoised 4m.png

Work Plan

• create a histogram comparing how many points in the reconstructed image match points in the ground truth
• create a histogram comparing how many points in reconstructed image are noise (are not in the original image)
• try network (trained with one track) with new data set that contains more than one track, and provide example plots for presentation
• create the efficiency and noise plots for the multi-track data sample reconstruction.
• train network with multi-track data and evaluate performance on multi-track testing sample.
• put on the web site architecture of the network and try with a little deeper architecture (4032->2016->1008->336 -> 224 -> 336 -> 1008 -> 2016 -> 4032)

Work Plan (November 17

• Provide workflow for testing, given input image write out the truth and AI inferred image
• Write software to create an image from AI inferred image and Truth image to reconstruct better truth, by filling in missing pieces in AI produced image from truth image when the points are close
• Evaluate efficiency of reconstructing the truth image and noisiness of resulting image. Also make graph now much noise was removed from original image.
• Start working with Hall-C data. much bigger input and evaluate efficiency of cleaning noise
• at the first stage noise has to be added to the Hall-C data for training, it can start with one track sample, then 250 track sample.

Resources

https://userweb.jlab.org/~gavalian/Documents/NN_Vary_Width_Depth%20(Nguyen).pdf

https://userweb.jlab.org/~gavalian/ML/2020/AutoEncoder/