https://crtc.cs.odu.edu/api.php?action=feedcontributions&feedformat=atom&user=Spyridon97crtc.cs.odu.edu - User contributions [en]2026-11-06T04:04:21ZUser contributionsMediaWiki 1.44.6https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8482Internal:GPCD2021-06-08T17:38:48Z<p>Spyridon97: /* (Spiros/Αngelos) Compute the Solvent Excluded Image */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius = 0<br />
### All pixel values >= probe-radius = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of atom type with the greatest VDW radius in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Paper and Presenation ==<br />
# [https://docs.google.com/presentation/d/1rYKfxhgCc9oUKI3xEZ0gLZQOapG64C2-6hNvoDtdpXY/edit?usp=sharing Presentation]<br />
# [https://www.overleaf.com/3442949237tvggkctnjdbj Paper]<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8481Internal:GPCD2021-06-03T17:41:59Z<p>Spyridon97: /* Paper and Presenation */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Paper and Presenation ==<br />
# [https://docs.google.com/presentation/d/1rYKfxhgCc9oUKI3xEZ0gLZQOapG64C2-6hNvoDtdpXY/edit?usp=sharing Presentation]<br />
# [https://www.overleaf.com/3442949237tvggkctnjdbj Paper]<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8480Internal:GPCD2021-06-03T17:41:51Z<p>Spyridon97: /* Paper and Presenation */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Paper and Presenation ==<br />
[https://docs.google.com/presentation/d/1rYKfxhgCc9oUKI3xEZ0gLZQOapG64C2-6hNvoDtdpXY/edit?usp=sharing Presentation]<br />
[https://www.overleaf.com/3442949237tvggkctnjdbj Paper]<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8479Internal:GPCD2021-06-03T17:41:02Z<p>Spyridon97: /* Paper and Presenation */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Paper and Presenation ==<br />
[https://docs.google.com/presentation/d/1rYKfxhgCc9oUKI3xEZ0gLZQOapG64C2-6hNvoDtdpXY/edit?usp=sharing Presentation]<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8478Internal:GPCD2021-06-03T17:40:55Z<p>Spyridon97: /* Results */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Paper and Presenation ==<br />
[[https://docs.google.com/presentation/d/1rYKfxhgCc9oUKI3xEZ0gLZQOapG64C2-6hNvoDtdpXY/edit?usp=sharing Presentation]]<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:PODM_3D&diff=8477Internal:PODM 3D2021-06-03T17:35:43Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Fall 2019==<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_56.pdf PODM3D] paper✔<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_58.pdf Guaranteed Quality Tetrahedral Delaunay Meshing for Medical Images] paper (Sect. 9) (TODO)<br />
# Write a proper README.md file✔<br />
# Add the [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] into the Appendix of the November Report✔<br />
# Create [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity CNF Adaptivity] Report<br />
<br />
==Spring 2020==<br />
# Create a [https://crtc.cs.odu.edu/CRTC_I2M User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Fall 2019==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
==Spring 2020==<br />
# Create meshes for the Medical Image Computing part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Medical_Imaging_Example_Meshes Medical Imaging Example Meshes] ✔<br />
<br />
==Summer 2020==<br />
# Create meshes for the Bioinformatics part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Bioinformatics_Example_Meshes Bioinformatics Example Meshes] ✔<br />
<br />
=Code=<br />
==Fall 2019==<br />
# Restructure repository of PODM3D based on Software engineering standards✔<br />
# Decouple Args from PODM✔<br />
# Create setArguments function✔<br />
# Create setInputImage(ImageType::Pointer)✔<br />
# Remove IOs from PODM✔<br />
# Implement a sizing function that will work as an extra rule and it will use the points of the tetrahedron that is tested✔<br />
## Create UML diagrams to understand the code and the Meshing Procedure in general✔<br />
## Create a list of functions that need to be modified to implement the extra Rule ✔<br />
## Modify the created list ([https://docs.google.com/spreadsheets/d/13M6qLRtS3wMH5B8vXwcqA-gP2b9p8QGwbhrn68VXoj0/edit#gid=0 Extra Rule sizing Function list]) of functions✔<br />
## Create the following test sizing function to understand if modifications have been done correctly✔ <syntaxhighlight lang="c++" line='line'><br />
bool checkSizingFunction(double coordinatesOfTetrahedron**)<br />
{<br />
double volume = volume(coordinatesOfTetrahedron);<br />
double* baryCenter = baryCenter(coordinatesOfTetrahedron)<br />
<br />
// if x axis <= 55 and volume of tetrahedron > 10<br />
if (baryCenter[0] <= 55 && volume > 10)<br />
return true;<br />
else<br />
return false<br />
}<br />
</syntaxhighlight><br />
## Create the infrastructure to support Sizing Function pointer✔<br />
## Incorporate Function Pointer to PODM by creating a setSizingFunction()✔<br />
## Replace the test sizing function with the Adaptivity function that was created by Christos (The algorithm is described [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity here])✔<br />
### Understand that the current implementation supports negative weights✔<br />
### Change the current implementation to use the minimum-maximum edge length of the element instead of its volume✔<br />
### Identify an issue of checking sample points outside the image and solve it according to Christos's suggestions✔<br />
# Separate the interface, the IOs and the filters from PODM✔<br />
<br />
==Spring 2020==<br />
# Create a general PODM driver✔<br />
# Set a warning if delta > min-spacing * min-dimension-size / 5✔<br />
# Set a default delta: ✔<br />
## if sizing-function is off then default delta = min-spacing * min-dimension-size / 100 ✔<br />
## if sizing-function is on then default delta = min-spacing * min-dimension-size / 20 ✔<br />
# Remove labels array if there is only 1 label of interest ✔<br />
# Print a warning if a label is not recovered✔<br />
# Add Numa flag in CmakeLists.txt ✔<br />
# Add binary file support ✔<br />
# Improve Text output ✔<br />
# Create label-filter for images ✔<br />
## Preserve a set of labels of an image ✔<br />
## Exclude a set of labels of an image ✔<br />
# Remove redundant volume threshold checks that don't contribute anything to the meshing procedure ✔<br />
<br />
==Summer 2020==<br />
# Libraries:<br />
## Update ITK to v5.1.0 ✔<br />
## Update VTK to v9.0.1 ✔<br />
## Remove Boost from PODM (preserve it only for CGAL) ✔<br />
# Save the label of a cell after its creation and Remove the expensive label recalculation in GetCleanMesh/GetCleanMeshParallel✔<br />
# Compute Mesh Results in Parallel (TODO)<br />
## Implement Parallel version of GetCleanMesh function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:mesh-results-speed-up.png|500px|File:mesh-results-speed-up.png]] ✔<br />
## Assigned as default ✔<br />
# Upgrade the define NO_IO_WRITING<br />
## Rename it to IO_WRITING ✔<br />
## Create functions for loading and writing of auxiliary files ✔<br />
## Create a compile flag for it, which will also enable the creation of the auxiliary files folder ✔<br />
## Fix truncation error when loading box ✔<br />
# Compute Box, Resized image and surface in Parallel ✔<br />
## Implement Parallel version of ComputeBoxAndSurface function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:box-surface-speed-up.png|500px|File:box-surface-speed-up.png]]✔<br />
## Assigned as default ✔<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Fall 2020==<br />
# Fix shift of origin if delta not multiple of the spacing ✔<br />
# Handle CNF images with spacing different than 1, and origin different than {0, 0, 0} ✔<br />
## Upgrade Image Segmentation ✔<br />
## Upgrade Sizing function ✔<br />
## Upgrade Linear Interpolation ✔<br />
# Change ImageSegmentation to create an image inside the requested grid domain ✔<br />
<br />
==Spring 2021==<br />
# [Christos] Addition of Extra points at the beginning of meshing ✔<br />
# [Polykarpos] Loading and saving the state of PODM mesher ✔<br />
# Load Dynamically Sizing Functions and their initialization as a shared Library using function pointers ✔<br />
# Detect special cases of Christos and Polykarpos additions ✔<br />
# Accept double instead of float type precision images ✔<br />
# Add a vector of weights for each vertex ✔<br />
# Add a vector of sizing functions for each thread ✔<br />
==Summer 2021==<br />
# Integrate PODM3D of APBNRR with the latest PODM3D (TODO)<br />
# Fix add points issue (TODO)<br />
# Fix PODM not working on Wahab (TODO)<br />
# Remeshing by using a mesh instead of an image as input and background mesh (TODO)<br />
# Replace CGAL Predicates with: (TODO)<br />
## Predicates Construction Kit:<br />
### [https://hal.inria.fr/hal-01225202 Paper]<br />
### [https://gforge.inria.fr/frs/?group_id=5833 Code]<br />
### Benchmark: [https://crtc.cs.odu.edu/Internal:4D_Predicates_Benchmark PCK vs CGAL Predicates Benchmark]<br />
### Note: see Geogram branch<br />
## Adaptive Precision Floating-Point Arithmetic and Fast Robust Predicates for Computational Geometry:<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-predicates.ps Paper]<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/code/predicates.c Code]<br />
# Smoothing (TODO)<br />
## [https://hal.inria.fr/inria-00413248/document Feature preserving Delaunay mesh generation from 3D multi-material images]<br />
## CGAL Sources: <br />
### https://doc.cgal.org/latest/Mesh_3/index.html<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3protectionimage3D<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3HybridDomainExample<br />
## PODM3D citations:<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999101968160 Volume conserving smoothing for piecewise linear curves, surfaces, and triple lines]<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999108004981 An anisotropic scale-invariant unstructured mesh generator suitable for volumetric imaging data]<br />
### [https://pubmed.ncbi.nlm.nih.gov/20161102/ Variational generation of prismatic boundary-layer meshes for biomedical computing, International Journal for Numerical Methods]<br />
## Prof, Andrey Suggestions:<br />
### [https://people.eecs.berkeley.edu/~jrs/papers/stuffing.pdf Isosurface Stuffing: Fast Tetrahedral Meshes with Good Dihedral Angles]<br />
### [https://imr.sandia.gov/_assets/documents/2019_IMR_Papers/3B.2-Chernikov.pdf CERTIFIED FUNCTIONS FOR MESH GENERATION]<br />
### [https://crtc.cs.odu.edu/pub/papers/journal_49.pdf GENERALIZED INSERTION REGION GUIDES FOR DELAUNAY MESH REFINEMENT]<br />
# Μultiple δ (TODO)<br />
## Implement multiple δ (TODO)<br />
## Use different δ values (possibly uniform based) according to the surface of each label of a segmented image (TODO)<br />
# Produce the Voronoi Mesh out of the Delaunay (TODO)<br />
## Check what TetGen does (TODO)<br />
## Research conversion methods (TODO)</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:PODM_3D&diff=8476Internal:PODM 3D2021-06-02T20:06:14Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Fall 2019==<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_56.pdf PODM3D] paper✔<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_58.pdf Guaranteed Quality Tetrahedral Delaunay Meshing for Medical Images] paper (Sect. 9) (TODO)<br />
# Write a proper README.md file✔<br />
# Add the [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] into the Appendix of the November Report✔<br />
# Create [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity CNF Adaptivity] Report<br />
<br />
==Spring 2020==<br />
# Create a [https://crtc.cs.odu.edu/CRTC_I2M User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Fall 2019==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
==Spring 2020==<br />
# Create meshes for the Medical Image Computing part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Medical_Imaging_Example_Meshes Medical Imaging Example Meshes] ✔<br />
<br />
==Summer 2020==<br />
# Create meshes for the Bioinformatics part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Bioinformatics_Example_Meshes Bioinformatics Example Meshes] ✔<br />
<br />
=Code=<br />
==Fall 2019==<br />
# Restructure repository of PODM3D based on Software engineering standards✔<br />
# Decouple Args from PODM✔<br />
# Create setArguments function✔<br />
# Create setInputImage(ImageType::Pointer)✔<br />
# Remove IOs from PODM✔<br />
# Implement a sizing function that will work as an extra rule and it will use the points of the tetrahedron that is tested✔<br />
## Create UML diagrams to understand the code and the Meshing Procedure in general✔<br />
## Create a list of functions that need to be modified to implement the extra Rule ✔<br />
## Modify the created list ([https://docs.google.com/spreadsheets/d/13M6qLRtS3wMH5B8vXwcqA-gP2b9p8QGwbhrn68VXoj0/edit#gid=0 Extra Rule sizing Function list]) of functions✔<br />
## Create the following test sizing function to understand if modifications have been done correctly✔ <syntaxhighlight lang="c++" line='line'><br />
bool checkSizingFunction(double coordinatesOfTetrahedron**)<br />
{<br />
double volume = volume(coordinatesOfTetrahedron);<br />
double* baryCenter = baryCenter(coordinatesOfTetrahedron)<br />
<br />
// if x axis <= 55 and volume of tetrahedron > 10<br />
if (baryCenter[0] <= 55 && volume > 10)<br />
return true;<br />
else<br />
return false<br />
}<br />
</syntaxhighlight><br />
## Create the infrastructure to support Sizing Function pointer✔<br />
## Incorporate Function Pointer to PODM by creating a setSizingFunction()✔<br />
## Replace the test sizing function with the Adaptivity function that was created by Christos (The algorithm is described [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity here])✔<br />
### Understand that the current implementation supports negative weights✔<br />
### Change the current implementation to use the minimum-maximum edge length of the element instead of its volume✔<br />
### Identify an issue of checking sample points outside the image and solve it according to Christos's suggestions✔<br />
# Separate the interface, the IOs and the filters from PODM✔<br />
<br />
==Spring 2020==<br />
# Create a general PODM driver✔<br />
# Set a warning if delta > min-spacing * min-dimension-size / 5✔<br />
# Set a default delta: ✔<br />
## if sizing-function is off then default delta = min-spacing * min-dimension-size / 100 ✔<br />
## if sizing-function is on then default delta = min-spacing * min-dimension-size / 20 ✔<br />
# Remove labels array if there is only 1 label of interest ✔<br />
# Print a warning if a label is not recovered✔<br />
# Add Numa flag in CmakeLists.txt ✔<br />
# Add binary file support ✔<br />
# Improve Text output ✔<br />
# Create label-filter for images ✔<br />
## Preserve a set of labels of an image ✔<br />
## Exclude a set of labels of an image ✔<br />
# Remove redundant volume threshold checks that don't contribute anything to the meshing procedure ✔<br />
<br />
==Summer 2020==<br />
# Libraries:<br />
## Update ITK to v5.1.0 ✔<br />
## Update VTK to v9.0.1 ✔<br />
## Remove Boost from PODM (preserve it only for CGAL) ✔<br />
# Save the label of a cell after its creation and Remove the expensive label recalculation in GetCleanMesh/GetCleanMeshParallel✔<br />
# Compute Mesh Results in Parallel (TODO)<br />
## Implement Parallel version of GetCleanMesh function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:mesh-results-speed-up.png|500px|File:mesh-results-speed-up.png]] ✔<br />
## Assigned as default ✔<br />
# Upgrade the define NO_IO_WRITING<br />
## Rename it to IO_WRITING ✔<br />
## Create functions for loading and writing of auxiliary files ✔<br />
## Create a compile flag for it, which will also enable the creation of the auxiliary files folder ✔<br />
## Fix truncation error when loading box ✔<br />
# Compute Box, Resized image and surface in Parallel ✔<br />
## Implement Parallel version of ComputeBoxAndSurface function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:box-surface-speed-up.png|500px|File:box-surface-speed-up.png]]✔<br />
## Assigned as default ✔<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Fall 2020==<br />
# Fix shift of origin if delta not multiple of the spacing ✔<br />
# Handle CNF images with spacing different than 1, and origin different than {0, 0, 0} ✔<br />
## Upgrade Image Segmentation ✔<br />
## Upgrade Sizing function ✔<br />
## Upgrade Linear Interpolation ✔<br />
# Change ImageSegmentation to create an image inside the requested grid domain ✔<br />
<br />
==Spring 2021==<br />
# [Christos] Addition of Extra points at the beginning of meshing ✔<br />
# [Polykarpos] Loading and saving the state of PODM mesher ✔<br />
# Load Dynamically Sizing Functions and their initialization as a shared Library using function pointers ✔<br />
# Detect special cases of Christos and Polykarpos additions ✔<br />
# Accept double instead of float type precision images ✔<br />
# Add a vector of weights for each vertex ✔<br />
# Add a vector of sizing functions for each thread ✔<br />
# Integrate PODM3D of APBNRR with the latest PODM3D (TODO)<br />
# Fix PODM not working on Wahab (TODO)<br />
# Remeshing by using a mesh instead of an image as input and background mesh (TODO)<br />
# Create upscale-image-filter (TODO)<br />
# Replace CGAL Predicates with: (TODO)<br />
## Predicates Construction Kit:<br />
### [https://hal.inria.fr/hal-01225202 Paper]<br />
### [https://gforge.inria.fr/frs/?group_id=5833 Code]<br />
### Benchmark: [https://crtc.cs.odu.edu/Internal:4D_Predicates_Benchmark PCK vs CGAL Predicates Benchmark]<br />
### Note: see Geogram branch<br />
## Adaptive Precision Floating-Point Arithmetic and Fast Robust Predicates for Computational Geometry:<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-predicates.ps Paper]<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/code/predicates.c Code]<br />
# Smoothing (TODO)<br />
## [https://hal.inria.fr/inria-00413248/document Feature preserving Delaunay mesh generation from 3D multi-material images]<br />
## CGAL Sources: <br />
### https://doc.cgal.org/latest/Mesh_3/index.html<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3protectionimage3D<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3HybridDomainExample<br />
## PODM3D citations:<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999101968160 Volume conserving smoothing for piecewise linear curves, surfaces, and triple lines]<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999108004981 An anisotropic scale-invariant unstructured mesh generator suitable for volumetric imaging data]<br />
### [https://pubmed.ncbi.nlm.nih.gov/20161102/ Variational generation of prismatic boundary-layer meshes for biomedical computing, International Journal for Numerical Methods]<br />
## Prof, Andrey Suggestions:<br />
### [https://people.eecs.berkeley.edu/~jrs/papers/stuffing.pdf Isosurface Stuffing: Fast Tetrahedral Meshes with Good Dihedral Angles]<br />
### [https://imr.sandia.gov/_assets/documents/2019_IMR_Papers/3B.2-Chernikov.pdf CERTIFIED FUNCTIONS FOR MESH GENERATION]<br />
### [https://crtc.cs.odu.edu/pub/papers/journal_49.pdf GENERALIZED INSERTION REGION GUIDES FOR DELAUNAY MESH REFINEMENT]<br />
# Μultiple δ (TODO)<br />
## Implement multiple δ (TODO)<br />
## Use different δ values (possibly uniform based) according to the surface of each label of a segmented image (TODO)<br />
# Produce the Voronoi Mesh out of the Delaunay (TODO)<br />
## Check what TetGen does (TODO)<br />
## Research conversion methods (TODO)</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:PartonsToNRRD&diff=8475CNF:PartonsToNRRD2021-06-02T20:05:27Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/partonstonrrd/ Gitlab]<br />
# [https://github.com/spyridon97/PartonsToNRRD/ Github]<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Create tool to generate NRRD images using partons data ✔<br />
<br />
== Fall 2020 ==<br />
# Enable the creation of images using specific spacing ✔<br />
<br />
== Spring 2021 ==<br />
# Create double instead of float type images ✔<br />
# Create 4D image</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:PODM_3D&diff=8474Internal:PODM 3D2021-06-02T15:57:58Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Fall 2019==<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_56.pdf PODM3D] paper✔<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_58.pdf Guaranteed Quality Tetrahedral Delaunay Meshing for Medical Images] paper (Sect. 9) (TODO)<br />
# Write a proper README.md file✔<br />
# Add the [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] into the Appendix of the November Report✔<br />
# Create [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity CNF Adaptivity] Report<br />
<br />
==Spring 2020==<br />
# Create a [https://crtc.cs.odu.edu/CRTC_I2M User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Fall 2019==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
==Spring 2020==<br />
# Create meshes for the Medical Image Computing part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Medical_Imaging_Example_Meshes Medical Imaging Example Meshes] ✔<br />
<br />
==Summer 2020==<br />
# Create meshes for the Bioinformatics part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Bioinformatics_Example_Meshes Bioinformatics Example Meshes] ✔<br />
<br />
=Code=<br />
==Fall 2019==<br />
# Restructure repository of PODM3D based on Software engineering standards✔<br />
# Decouple Args from PODM✔<br />
# Create setArguments function✔<br />
# Create setInputImage(ImageType::Pointer)✔<br />
# Remove IOs from PODM✔<br />
# Implement a sizing function that will work as an extra rule and it will use the points of the tetrahedron that is tested✔<br />
## Create UML diagrams to understand the code and the Meshing Procedure in general✔<br />
## Create a list of functions that need to be modified to implement the extra Rule ✔<br />
## Modify the created list ([https://docs.google.com/spreadsheets/d/13M6qLRtS3wMH5B8vXwcqA-gP2b9p8QGwbhrn68VXoj0/edit#gid=0 Extra Rule sizing Function list]) of functions✔<br />
## Create the following test sizing function to understand if modifications have been done correctly✔ <syntaxhighlight lang="c++" line='line'><br />
bool checkSizingFunction(double coordinatesOfTetrahedron**)<br />
{<br />
double volume = volume(coordinatesOfTetrahedron);<br />
double* baryCenter = baryCenter(coordinatesOfTetrahedron)<br />
<br />
// if x axis <= 55 and volume of tetrahedron > 10<br />
if (baryCenter[0] <= 55 && volume > 10)<br />
return true;<br />
else<br />
return false<br />
}<br />
</syntaxhighlight><br />
## Create the infrastructure to support Sizing Function pointer✔<br />
## Incorporate Function Pointer to PODM by creating a setSizingFunction()✔<br />
## Replace the test sizing function with the Adaptivity function that was created by Christos (The algorithm is described [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity here])✔<br />
### Understand that the current implementation supports negative weights✔<br />
### Change the current implementation to use the minimum-maximum edge length of the element instead of its volume✔<br />
### Identify an issue of checking sample points outside the image and solve it according to Christos's suggestions✔<br />
# Separate the interface, the IOs and the filters from PODM✔<br />
<br />
==Spring 2020==<br />
# Create a general PODM driver✔<br />
# Set a warning if delta > min-spacing * min-dimension-size / 5✔<br />
# Set a default delta: ✔<br />
## if sizing-function is off then default delta = min-spacing * min-dimension-size / 100 ✔<br />
## if sizing-function is on then default delta = min-spacing * min-dimension-size / 20 ✔<br />
# Remove labels array if there is only 1 label of interest ✔<br />
# Print a warning if a label is not recovered✔<br />
# Add Numa flag in CmakeLists.txt ✔<br />
# Add binary file support ✔<br />
# Improve Text output ✔<br />
# Create label-filter for images ✔<br />
## Preserve a set of labels of an image ✔<br />
## Exclude a set of labels of an image ✔<br />
# Remove redundant volume threshold checks that don't contribute anything to the meshing procedure ✔<br />
<br />
==Summer 2020==<br />
# Libraries:<br />
## Update ITK to v5.1.0 ✔<br />
## Update VTK to v9.0.1 ✔<br />
## Remove Boost from PODM (preserve it only for CGAL) ✔<br />
# Save the label of a cell after its creation and Remove the expensive label recalculation in GetCleanMesh/GetCleanMeshParallel✔<br />
# Compute Mesh Results in Parallel (TODO)<br />
## Implement Parallel version of GetCleanMesh function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:mesh-results-speed-up.png|500px|File:mesh-results-speed-up.png]] ✔<br />
## Assigned as default ✔<br />
# Upgrade the define NO_IO_WRITING<br />
## Rename it to IO_WRITING ✔<br />
## Create functions for loading and writing of auxiliary files ✔<br />
## Create a compile flag for it, which will also enable the creation of the auxiliary files folder ✔<br />
## Fix truncation error when loading box ✔<br />
# Compute Box, Resized image and surface in Parallel ✔<br />
## Implement Parallel version of ComputeBoxAndSurface function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:box-surface-speed-up.png|500px|File:box-surface-speed-up.png]]✔<br />
## Assigned as default ✔<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Fall 2020==<br />
# Fix shift of origin if delta not multiple of the spacing ✔<br />
# Handle CNF images with spacing different than 1, and origin different than {0, 0, 0} ✔<br />
## Upgrade Image Segmentation ✔<br />
## Upgrade Sizing function ✔<br />
## Upgrade Linear Interpolation ✔<br />
# Change ImageSegmentation to create an image inside the requested grid domain ✔<br />
<br />
==Spring 2021==<br />
# [Christos] Addition of Extra points at the beginning of meshing ✔<br />
# [Polykarpos] Loading and saving the state of PODM mesher ✔<br />
# Load Dynamically Sizing Functions and their initialization as a shared Library using function pointers ✔<br />
# Detect special cases of Christos and Polykarpos additions ✔<br />
# Accept double instead of float type precision images ✔<br />
# Add a vector of weights for each vertex ✔<br />
# Integrate PODM3D of APBNRR with the latest PODM3D (TODO)<br />
# Fix PODM not working on Wahab (TODO)<br />
# Remeshing by using a mesh instead of an image as input and background mesh (TODO)<br />
# Create upscale-image-filter (TODO)<br />
# Replace CGAL Predicates with: (TODO)<br />
## Predicates Construction Kit:<br />
### [https://hal.inria.fr/hal-01225202 Paper]<br />
### [https://gforge.inria.fr/frs/?group_id=5833 Code]<br />
### Benchmark: [https://crtc.cs.odu.edu/Internal:4D_Predicates_Benchmark PCK vs CGAL Predicates Benchmark]<br />
### Note: see Geogram branch<br />
## Adaptive Precision Floating-Point Arithmetic and Fast Robust Predicates for Computational Geometry:<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-predicates.ps Paper]<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/code/predicates.c Code]<br />
# Smoothing (TODO)<br />
## [https://hal.inria.fr/inria-00413248/document Feature preserving Delaunay mesh generation from 3D multi-material images]<br />
## CGAL Sources: <br />
### https://doc.cgal.org/latest/Mesh_3/index.html<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3protectionimage3D<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3HybridDomainExample<br />
## PODM3D citations:<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999101968160 Volume conserving smoothing for piecewise linear curves, surfaces, and triple lines]<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999108004981 An anisotropic scale-invariant unstructured mesh generator suitable for volumetric imaging data]<br />
### [https://pubmed.ncbi.nlm.nih.gov/20161102/ Variational generation of prismatic boundary-layer meshes for biomedical computing, International Journal for Numerical Methods]<br />
## Prof, Andrey Suggestions:<br />
### [https://people.eecs.berkeley.edu/~jrs/papers/stuffing.pdf Isosurface Stuffing: Fast Tetrahedral Meshes with Good Dihedral Angles]<br />
### [https://imr.sandia.gov/_assets/documents/2019_IMR_Papers/3B.2-Chernikov.pdf CERTIFIED FUNCTIONS FOR MESH GENERATION]<br />
### [https://crtc.cs.odu.edu/pub/papers/journal_49.pdf GENERALIZED INSERTION REGION GUIDES FOR DELAUNAY MESH REFINEMENT]<br />
# Μultiple δ (TODO)<br />
## Implement multiple δ (TODO)<br />
## Use different δ values (possibly uniform based) according to the surface of each label of a segmented image (TODO)<br />
# Produce the Voronoi Mesh out of the Delaunay (TODO)<br />
## Check what TetGen does (TODO)<br />
## Research conversion methods (TODO)</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8473CNF:CNF2021-05-27T19:44:19Z<p>Spyridon97: /* References */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Minimizes the amount of partons calculations needed to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was mentioned<br />
### Cons:<br />
#### Maybe caching issues, but they are possibly insignificant since PODM is already really fast.<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
## [https://www.hpl.hp.com/techreports/2002/HPL-2002-320.pdf n-Simplex Interpolation]<br />
## [https://www.techscience.com/CMES/v94n4/26992/pdf Quadratic Interpolation with Tetrahedra for 3D Scalar Fields]<br />
## [https://www.researchgate.net/publication/326950468_A_look-up_table_method_based_on_unstructured_grids_and_its_application_to_non-ideal_compressible_fluid_dynamic_simulations A look-up table method based on unstructured grids and its application to non-ideal compressible fluid dynamic simulations]<br />
## [https://arxiv.org/pdf/physics/0203033.pdf Foam: A General-Purpose Cellular Monte Carlo Event Generator]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8472CNF:CNF2021-05-21T22:59:21Z<p>Spyridon97: /* References */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Minimizes the amount of partons calculations needed to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was mentioned<br />
### Cons:<br />
#### Maybe caching issues, but they are possibly insignificant since PODM is already really fast.<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
## [https://www.hpl.hp.com/techreports/2002/HPL-2002-320.pdf n-Simplex Interpolation]<br />
## [https://www.techscience.com/CMES/v94n4/26992/pdf Quadratic Interpolation with Tetrahedra for 3D Scalar Fields]<br />
## [https://www.researchgate.net/publication/326950468_A_look-up_table_method_based_on_unstructured_grids_and_its_application_to_non-ideal_compressible_fluid_dynamic_simulations A look-up table method based on unstructured grids and its application to non-ideal compressible fluid dynamic simulations]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8471CNF:CNF2021-05-21T15:58:11Z<p>Spyridon97: /* References */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Minimizes the amount of partons calculations needed to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was mentioned<br />
### Cons:<br />
#### Maybe caching issues, but they are possibly insignificant since PODM is already really fast.<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
## [https://www.hpl.hp.com/techreports/2002/HPL-2002-320.pdf n-Simplex Interpolation]<br />
## [https://www.techscience.com/CMES/v94n4/26992/pdf Quadratic Interpolation with Tetrahedra for 3D Scalar Fields]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8470CNF:CNF2021-05-21T02:19:42Z<p>Spyridon97: /* References */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Minimizes the amount of partons calculations needed to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was mentioned<br />
### Cons:<br />
#### Maybe caching issues, but they are possibly insignificant since PODM is already really fast.<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
## [https://www.hpl.hp.com/techreports/2002/HPL-2002-320.pdf n-Simplex Interpolation]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8466CNF:CNF2021-04-28T21:42:36Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Minimizes the amount of partons calculations needed to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was mentioned<br />
### Cons:<br />
#### Maybe caching issues, but they are possibly insignificant since PODM is already really fast.<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8465CNF:CNF2021-04-28T21:32:13Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### By splitting a cell that one CFF requires while another CFF does not, we create a mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was originally mentioned<br />
### Cons:<br />
#### Maybe caching issues<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8464CNF:CNF2021-04-28T21:27:43Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### By splitting a cell if a CFF requires that while another CFF does not, we create a Mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation accuracy throughout every cell of a cell, with the minimum partons calculations) are:<br />
## (Optional) Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
#### Only for one CFF<br />
## Create a dense enough uniform mesh in order to satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### You don't know how big your mesh should be<br />
#### Only for one CFF<br />
## Create a mesh in an iterative fashion by adding random points to the mesh till you satisfy the desired interpolation accuracy<br />
### Pros:<br />
#### Achieves the required interpolation accuracy<br />
### Cons:<br />
#### Big mesh size<br />
#### Many Partons calculations<br />
#### Only for one CFF<br />
## Create a mesh using the linear interpolation error as the sizing function<br />
### Pros:<br />
#### Everything that was originally mentioned<br />
### Cons:<br />
#### Maybe caching issues<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8463CNF:CNF2021-04-28T21:21:47Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### By splitting a cell if a CFF requires that while another CFF does not, we create a Mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation error throughout every cell of a cell, with the minimum partons calculations) are:<br />
## Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
## Create a dense enough<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8462CNF:CNF2021-04-28T21:21:20Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### By splitting a cell if a CFF requires that while another CFF does not, we create a Mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
# The steps that we will describe in the paper that we tried to achieve the desired result (sufficient interpolation error throughout every cell of a cell) are:<br />
## Gagik's approach: he proposed to decimate a huge image (1,000,000 partons calculation) and then generate a tessellation that would capture the important features of the CFF described<br />
### Pros:<br />
#### Predefined number of partons calculations<br />
#### Small mesh size<br />
### Cons:<br />
#### Many Partons calculations<br />
#### No interpolation error guarantee<br />
## Create a dense enough<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8461CNF:CNF2021-04-28T21:15:13Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of CNF models<br />
### By splitting a cell if a CFF requires that while another CFF does not, we create a Mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8460CNF:CNF2021-04-28T21:14:55Z<p>Spyridon97: /* Week April 26 - May 2 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Simpler model storage for the future database of every CFF<br />
### By splitting a cell if a CFF requires that while another CFF does not, we create a Mesh with a better error distribution than the one that would have if we did not split that cell. <br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF&diff=8459CNF:CNF2021-04-28T21:09:48Z<p>Spyridon97: /* Week March 8 - 14 */</p>
<hr />
<div>= Description =<br />
We propose an interdisciplinary project to leverage advanced computational methods from medical imaging for processing phase-space distributions in nuclear femtography experiments, with the aim to enable next-generation process simulations, data analyses, and physics model comparisons.<br />
<br />
A detailed list of results for CNF cases received from JLAB are presented here:<br />
[https://crtc.cs.odu.edu/CNF:Example_Meshes Example Meshes]<br />
<br />
= SURA CNF19-04 Project =<br />
== Objectives ==<br />
* (O1) Implement an n-dimensional mesh-based representation of phase space data using tessellation methods and test it in low-dimensional scenarios (n=3) <br />
* (O2) Explore potential physics model comparisons enabled by the new technologies<br />
<br />
== Short-term Deliverables (6 months) ==<br />
[[File: distributionquarks.png|thumb|right|350px| '''Distribution of the probability of quarks' existence based on the change in momentum''']]<br />
# Post-processing of the output of the PARTONS GPD physics model for generation of NRDD/DICOM images and their segmentation (by July 1); -- [<span style="color:green">DONE</span>]<br />
# Generation of n-dimensional sequence of tessellations for numerical integration (by July 1); -- [<span style="color:green">DONE</span>] <br />
## software delivered to JLAB, May 2019: [https://crtc.cs.odu.edu/CRTC_I2M CRTC Image to Mesh Conversion Software Suite].<br />
## software developed at JLAB to transform physics observable grids generated by the PARTONS framework to the data format used by ODU provided software.<br />
# Use of Tesselation software gives the ability to view and analyze physics observables in a new way with modern tools. It also allows for fast, efficient Monte-Carlo generation of physics observables for event generation from tessellated objects. Computation of proton structures and physics observables are otherwise very computationally extensive. -- [<span style="color:blue">DONE</span>] <br />
## explore the option of generating graded meshes (by September).<br />
# Experiment with 3D Slicer and Paraview open-source imaging and visualization systems to display visual representations of the entire space and/or m-slices, with m < n (by Sept. 30); -- [<span style="color:green">DONE</span>]<br />
## [http://www.cs.odu.edu/crtc/videos/paraview.html Imaging Demo using Paraview]<br />
## [https://kitware.github.io/paraview-glance/doc/ Paraview Glance]<br />
# Topical workshop including experts in imaging, exascale computing, uncertainty quantification, and nuclear physics to guide further developments (by Sept. 30). -- [<span style="color:green">DONE</span>] <br />
## Invited visitors and agenda is completed. <br />
## The [https://cepm.cs.odu.edu/CNF_Imaging_Workshop CNF Imaging Workshop] took place on August 19th and 20th at JLab.<br />
# [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF mesh examples]<br />
<br />
= SURA CNF20-001 Project =<br />
== Objectives==<br />
(O1) Demonstrate that 3-dimensional tessellations can improve the speed of basic nuclear femtography operations like Monte Carlo computations using a Nuclear Physics example<br />
<br />
(O2) Extend the isotropic tessellations 4 dimensions Images (eg. 4D NRRD images)<br />
<br />
(O3) Explore gradation in 4D tessellations using a given metric or sizing function<br />
<br />
== Short-term Deliverables (3 months)==<br />
<br />
'''August 15'''<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Convert the output of PARTONS and UVA’s GPD generators into Nearly Raw Raster Data (NRRD) format (Liuiti, Heddle, Chrisochoides). This (ODU) group will be responsible for PARTONS and will assist UVA with GPD generators.<br />
** Co-organize with Dr. Hyde as a chair a virtual workshop around mid-June on "The Integration of Imaging and Visualization in Femtography" (Hyde, Liuiti, Heddle, Chrisochoides)<br />
* fine-tune existing 3D I2M conversion software tool and deliver a new Docker to JLab that enhances I/O and meets new requirements identified in the last six months (Chrisochoides/CS Gard. Student);<br />
* pre-process 4D CNF data and create a 4D synthetic data set for developing the 4D triangulation method (Gavalian/Chrisochoides/Computer Science (CS) Grad. Student);<br />
'''September 30, 2020:'''<br />
<br />
* Integration with FEMT-006-03 (PI Dr. Heddle) and FEMT-006-05 (Dr. Liuiti):<br />
** Tesselation of Compton Form Factors using PARTONS and UVA GPD generators (Hyde, Liuiti, Chrisochoides)<br />
* create 4D triangulation algorithm and code as a prelude to future physics implementations (Chrisochodes/CS Gard. Student);<br />
* 3D-visualization plugins in ParaView of the flavor-dependent Compton form factors (Hyde/Chrisochoides/Physics and CS Gard. Students);<br />
* “smart 3-dimensional lookup table” based on an '''adaptive or graded''' mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student);<br />
<br />
== Smart look-up tables ==<br />
<br />
Objective: “smart 3-dimensional lookup table” based on adaptive mesh-based tessellated representation of the CFFs for efficient evaluation of Monte Carlo simulations (Gavalian/Hyde/Physics Grad. Student)<br />
<br />
=== Timetable for paper on "smart lookup table" for fast MC simulations ===<br />
# Define Region of Interest (RoI) <br />
## Use NRRD image (done with CNF19-004): might be inefficient, but the definition of RoI is outside the scope of the paper<br />
## Use a cube or closed 3D surface (to be tested, it can be done within a week): very efficient way to define the RoI, as well as it is known.<br />
## use sparse and course set of points in a broader area and slowly converge to the RoI with as many points and their tessellation are required by a given metric.<br />
### need to define a metric (Charles/Mitch): preliminary ideas by mid-August and finalize metric by end of August<br />
# Implement smart lookup table using tessellation generated from (1)<br />
## reproduce Gagik's results<br />
### Mitch: understand the code and write a one-page description (Gagik to supervise the effort): mid of July, report progress in mid-report<br />
### Mitch/Charles define (if there is a need) a new way that might involve UVA's data fitting method: a proposal by end of July.<br />
#define an example problem pertinent to CNF/IEC where "smart lookup table for fast MC simulations''' are the dominant term in the execution time (Charles/Christian/Gagik)<br />
## Putting Things Alltogether (end of August the latest first week of Sept.)<br />
# Finish paper by end of September (All, Charles/Christian will define the skeleton and assign sections to CS and Physic subgroups)<br />
<br />
==Weekly Reports==<br />
<br />
=== Week May 17 - 22 ===<br />
# (Michell/Spiros) Mitch needs to get the I2M Docker and become familiar with its usage [<span style="color:green">DONE</span>]<br />
# (All) Get to know the team: Michell (Physics) and the CS students that will be assisting in the API for the tessellation (Spiros) and ParaView for Visualization (Kevin) [<span style="color:green">DONE</span>]<br />
# (Charles/Michell/Nikos) Setup a proposal for a time table for this aspect of the project it will be fine-tuned and finalized in the first official meeting [<span style="color:green">DONE</span>]<br />
# (All) Talk about the wiki page we need to set up for our weekly reports [<span style="color:green">DONE</span>]<br />
<br />
=== Week May 25 - 30 ===<br />
# (Spiros) Smooth transition Michell to use CRTC I2M conversion [<span style="color:green">DONE</span>]<br />
# (Mitch) Comfortable using CRTC's I2M conversion Docker, User Guided, read VTK tessellation file, etc.. [<span style="color:green">DONE</span>]<br />
# (Kevin) Identify 3D ParaView Plugins that we might need to display results [<span style="color:green">DONE</span>]<br />
# (Charles) Identify problem(s) to be addressed with the UVA and PARTON models [<span style="color:orange">TODO</span>]<br />
# (Charles/Chistian/Gagik) Examples of target Journals we want to consider for paper submission [<span style="color:green">DONE</span>]<br />
# (Christos) Slides from 05-22 [https://docs.google.com/presentation/d/1yz8dxVXO82rFCtD9KWjG1PBvvP5VikAxTz07qoM3IqU here] [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 1 - 6 ===<br />
# (Spiros) Add binary IO for output meshes. [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Finish converting PARTONS output data into NRRD format for PODM. [<span style="color:green">DONE</span>]<br />
# (Kevin) Practice utilizing and applying custom functions on data within Paraview [<span style="color:green">DONE</span>] <br />
## Examples: [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data Visualization page]<br />
# (Charles) Why use NRRD format if it is optimal to calculate on non-uniform grid?<br />
## (Spiros) because PODM currently supports images that are uniform grids. [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 8 - 12 ===<br />
# (Mitch/Charles) Compute PARTONS points at vertices of simplices, <br />
## Question: Does the tesselation software store the interpolated values at the vertices or the simplex barycentres?<br />
### Vertices [<span style="color:green">DONE</span>]<br />
## Construct difference of interpolation values at vertices with new computed PARTONS values [<span style="color:green">DONE</span>]<br />
# (Spiros/Nikos) Explore ways of producing the ith step of adaptive tessellation using Mitch's new values on the (i-1)th tessellation<br />
## (Spiros) By creating Images out of the meshes using interpolation: [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software]<br />
# (Mitch/Kevin/Charles) Help display Mitch's 3D data using predefined labels, etc..<br />
## Experiment with 2D image of CFF(xi,t) at fixed Q2 vs. 3D image of CFF(xi,t,Q2)<br />
### Linear vs. log axis. [<span style="color:green">DONE</span>]<br />
### Axis labels: convert (x,y,z), to (xi,t,CFF) or (xi,t,Q2) with color CFF scale [<span style="color:green">DONE</span>]<br />
### [https://crtc.cs.odu.edu/CNF:Visualization_for_CNF_data#Other_Options_in_Paraview Instructions Available Here]<br />
## Change PARTONS mesh to equal steps in log(xi) and log(Q2) [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik) Reproduce either Reimann or Lebesgue Integral calculations using [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for 2D [<span style="color:green">DONE</span>]<br />
<br />
=== Week June 15 - 19 ===<br />
# (Spiros) Develop a generic interpolation filter for Mitch and write a high-level description (including VTK/ITK kernels used) to assist for future modifications<br />
## [https://odu.app.box.com/folder/115714696781 Code folder] of [https://crtc.cs.odu.edu/CNF:Weighted_Mesh_To_Image Weighted Mesh To Image Software] [<span style="color:green">DONE</span>]<br />
## Fomulation of [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] [<span style="color:green">DONE</span>]<br />
## Compare [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation] with [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] of Charles [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) finish up the API for passing data image and vtk data to PARTON and close the loop<br />
## (Spiros) Develop a function that reads and converts an image to a 3D lattice [<span style="color:green">DONE</span>]<br />
## (Spiros) Develop a function that reads the points and their weights of a tessellation [<span style="color:green">DONE</span>]<br />
## (Mitch) Close the loop [<span style="color:orange">TODO</span>]<br />
# (Spiros) Convert partons data to nrrd properly, so that they can be visualized with Paraview [<span style="color:green">DONE</span>]<br />
# (Mitch/Gagik/Spiros) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] for “smart lookup table for MC kernel”, produce 3D results and refactor the code:<br />
## (Mitch) Figure out the computational details of [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] [<span style="color:green">DONE</span>] <br />
## (Spiros) Refactor Gagik's code, use VTK library and add comments [<span style="color:green">DONE</span>] <br />
## '''(Mitch/Gagik) Produce results using 3D tesselations [<span style="color:orange">TODO</span>]'''<br />
# (Charles) in parallel work on a method for "smart lookup table"<br />
## [http://www.lions.odu.edu/~chyde/Research/TechnicalNotes/TessellationInterpolation.pdf Tessellation & Interpolation document] [<span style="color:green">DONE</span>], corrections, update, expansion in progress<br />
## Compare with [https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel] and either consolidate the two or end up with two different methods. [<span style="color:orange">TODO</span>]<br />
<br />
=== Week June 22 - 26 ===<br />
# (Christian) Summary of CNF projects for 2019/20, [https://docs.google.com/presentation/d/1YuvSjHGvkB0I36gIxla58GxR-H3NtPd01w7ZtARDvuU/edit?usp=sharing CNF Summary Slides] [<span style="color:green">DONE</span>]: <br />
# (Charles/Gagik) Couple of Slides for use of adaptive tessellation in Monte Carlo and CFF approximation<br />
# (Spiros) Creationg of a 4D Synthetic Image Generator <br />
## HyperSphere (4D Sphere) [<span style="color:green">DONE</span>]<br />
## HyperCube (4D Cube) [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) Parallelize Partons' code using OpenMP and submit jobs on Turing/Wahab. <br />
## (Spiros) Help Mitch login and view files on Wahab [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) Explore potential to parallelize Partons' code [<span style="color:orange">TODO: Year 2021</span>]<br />
### It looks like requires more effort than our project could afford. It has to be moved as a task for next year. <br />
## (Spiros) Explore potential to execute multiple jobs on Wahab [<span style="color:green">DONE</span>]<br />
### If this is hard as the item above might need to move for next year. <br />
# (Mitch) Create write-up explanation of Gagiks Monte Carlo code [<span style="color:green">DONE</span>]<br />
##(Mitch/Spiros) Generalize MC Gagik's 2D code to 3D and consider potential higher dimensions [<span style="color:orange">TODO: Year 2021</span>]<br />
# (Mitch) Re-run PARTONS at mesh points [<span style="color:green">DONE</span>]<br />
## Plot difference of original mesh to re-evaluated mesh in Paraview: Squared Difference [<span style="color:green">DONE</span>]<br />
## Plot new values on original tessellation points<br />
<br />
=== Week June 29 - July 3 ===<br />
# '''(O1)''' (Mitch/Gagik/Charles) Smart Lookup Table for Monte Carlo <br />
## Compare Tessellation with either |f_a – f_b| metric or |f_a – f_b| / |f_a + f_b|<br />
## First step is to redo PODM tessellation with same algorithm, but pass ln[Im[CFF]] instead of Im[CFF] to PODM<br />
### In anticipation of 4D representations, calculate PARTONS values for full DVCS cross-section:<br />
#### For discussion: Which variables to keep fixed, which variables to display<br />
# '''(O2)''' (Spiros/Nikos) 3D/4D Tesselation for Nuclear Femtography<br />
## Explore a new way to leverage 3D/4D I2M conversion and the new requirements for adaptivity relying on min I/O for PARTONS [<span style="color:green">Done</span>]<br />
### [https://crtc.cs.odu.edu/Internal:TetGen_CNF CNF Constained Mesher Prototype]<br />
## (Christos/Spiros) Consider PDR worker initialization for initializing PODM and enabling remeshing or an existing mesh. [<span style="color:orange">TODO: Year 2021</span>]<br />
### In the PDR.PODM framework after the initial mesh construction each worker receives a part of the mesh and refines it. During initialization one essentially starts from a serialized version of the mesh (similar to reading a file) and populates internal data structures. After all, is set, PODM works as usual. <br />
### The same initialization method can be used here. Using this approach one can refine an existing mesh while keeping all previous vertex-based information. <br />
### Keeping vertex-based information means than we can get produce adaptive results using just information of the new weights skipping thus the creation of intermediate images.<br />
### As a further (future) step this approach can enable an "always in memory mesh" where the data are always in memory and being accessed by a common API from each component of the pipeline.<br />
# '''(O3)''' (Spiros/Mitch/Charles) Visualization<br />
## (Spiros) Explore a visualization option for 4D tesselations by extracting a 3D "slice" and then use ParaView to display results [<span style="color:orange">TODO</span>]<br />
### Define what a 3D "slice" means in 4D space [<span style="color:orange">TODO</span>]<br />
## (Mitch/Charles) Generate PARTON 4D and a 3D "slice" for these data, for validation of viz. [<span style="color:orange">TODO</span>] <br />
### 4D (unpolarized cross-sections) differential in log(Q2), log(xB), t, phi [<span style="color:orange">TODO</span>]<br />
## (Mitch/Kevin) Explore color palette options for displaying positive and negative values simultaneously [<span style="color:green">DONE</span>]<br />
# '''(O4)''' Integration with UVA and CNU<br />
## Think about an on-line meeting on Monte Carlo Simulations in Nuclear Femtography?<br />
## What is the status with UVA, are there any changes due to lack of additional funding?<br />
<br />
=== Week July 6 - July 10 ===<br />
# (Spiros / Mitch) Adaptivity Metrics<br />
## Using local denominator<br />
### max-min / mean (Not working)<br />
### rms / mean (Not working)<br />
### Note: Local denominators are not working well because of the fact that they perform local normalization which results in uniform gradation, and therefore a uniform mesh<br />
## Using no denominator<br />
### max-min: [https://crtc.cs.odu.edu/CNF:Mesh_Sizing_for_CNF_data#Method Original Method]<br />
### Note: No denominators can work, but for each Image, the weight-limit needs to be tuned based on its weights.<br />
## Using global denominator<br />
### rms / globalMean (an idea which needs tweaking)<br />
#### Note: not bounded<br />
### rms / globalRms (Possibly working)<br />
#### Note: not upper bounded [0, +oo)<br />
### max-min / elementWeightRange<br />
#### Note: bounded [0, 1]<br />
### Note: Global denominators work, and the weight-limit is normalized based on the given image.<br />
## [[CNF:Metric Figures | Metric Figures]]<br />
## Useful Filter: [https://itk.org/Doxygen/html/classitk_1_1StatisticsImageFilter.html StatisticsImageFilter]<br />
<br />
=== Week July 13 - July 17 ===<br />
# (Spiros/Mitch/Wirawan) Software Engineering Aspects for [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] <br />
## (Spiros/Mitch) Maintenance and Clarity (re-organized Mitch's code) [<span style="color:green">DONE</span>]<br />
## (Spiros/Wirawan) Improve End-User Productivity with Distributed Computing of Partons CNF Data Generation [<span style="color:orange">TODO</span>]<br />
# (Spiros/Mitch/Charles) Design of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities]<br />
## (Mitch/Charles) Identify Requirements [TODO]<br />
## (Spiros) Design a library that satisfies the requirements [<span style="color:orange">TODO with a supplement</span>]<br />
# (Spiros) Design of a templated software, [https://crtc.cs.odu.edu/CNF:Monte_Carlo_Event_Generator_using_a_Tessellation Monte Carlo Event Generator using a Tessellation] [<span style="color:orange">TODO with a supplement</span>]<br />
## Re-organize Gagik's 2D code, CMI, templated for 2-3-4 dimensions, improve the complexity of search for given cell weight (from linear to binary) and parallelization using OpenMP <br />
# (Spiros) Sizing function Modifications:<br />
## Change way of calculating value for sampling points<br />
### Old version: assign the value of sampling point using to the closest pixel value<br />
### New version: assign the value of sampling point using interpolation over the image [<span style="color:green">DONE</span>]<br />
### Reason of change: The problem with the old way of assigning values to the sampling points is that there is a high possibility of 2 or more sampling points having the same assigned value. I think that we can solve this issue by assigning values using interpolation to be sure that every sampling point would have a unique assigned value. This way we would need fewer elements to be able to capture the gradation.<br />
## Change Default max-Edge from Diagonal * 1/10 to Diagonal 2/10 [<span style="color:green">DONE</span>]<br />
### Reason of change: this way we create larger elements in regions that are not of interest<br />
# (Spiros/Kevin/Mitch) Results:<br />
## (Spiros/Kevin) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img GPDGK16_uH_img] (with new sizing function) [<span style="color:green">DONE</span>]<br />
### Mitch generated the data the previous week<br />
## (Mitch) Generate a new tesselation with GPDGK16_uH_nxi_210 (with old sizing function) [https://crtc.cs.odu.edu/CNF_Example_Meshes#GPDGK16_uH_img_nxi.3D211 GPDGK16_uH_img_nxi=211] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles/Spiros) Generate Monte Carlo events of unpolarized DVCS Cross-Section <br />
## Generalize Gagiks monte carlo kernel into 3D or N-Dimensional [<span style="color:green">DONE</span>]<br />
## Implement "acceptance rejection" algorithm from [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
## [https://odu.box.com/s/m4xoofpqziz57jnvwlis9iqnm9toram5 VTK Interpolation]<br />
## Updated Document on Interpolation with Simplices: [https://odu.box.com/s/1jrq704pytgxafz9tvoe12jmbsjuc9rg Tessellation & Interpolation document] [<span style="color:green">DONE</span>]<br />
### Plot in ROOT CFF monte carlo events from tessellation [<span style="color:green">DONE</span>]<br />
### Generate unpolarized cross-section tessellation (needs to be parallelized ~10min per point) [<span style="color:orange">TODO</span>]<br />
## (Spiros/Mitch) Add needed methods from Spiros's library [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:orange">TODO</span>]<br />
# (Mitch/Charles) Analyze CFF and Cross Section results<br />
## Plot RMS deviation of cell points from CFF mesh<br />
## Plot and fit t dependence of CFF at fixed value of Q<sup>2</sup>, calculate RMSD of fit, extract fit parameters <br />
# (All) [https://docs.google.com/presentation/d/1z_7XDMlsOUpJg-0yk0kMJmjLJdG4y7wx8bBrc_MMrC0/edit?usp=sharing CNF Adaptivity Pipeline] [<span style="color:green">DONE</span>] and good progress toward "Smart Lookup Table"[[File:Photo_Mid_Report_July16_2020.png|500px|Photo_Mid_Report_July16_2020.png]]<br />
<br />
=== Week July 20 - July 24 ===<br />
# (Spiros) Implementation of [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart LookUp Table Utilities] [<span style="color:green">DONE</span>]<br />
# (Spiros) Interpolate Kevin's CDT3D meshes [https://odu.box.com/s/vpps3wl1k9szx213y95q6sj4mim36sts Box Folder] [<span style="color:green">DONE</span>]<br />
# (Mitch) Make flow-chart explanation of "smart look-up table" and Monte Carlo code [https://docs.google.com/presentation/d/1KeR3h6ReyzVARX0qnBKaht97GCy0pa-6ruFDkmhoHMU/edit?usp=sharing] [<span style="color:green">DONE</span>]<br />
## Write driver for point look-up [<span style="color:orange">Done but needs testing</span>]<br />
# (Mitch/Kevin) Generate mesh from PARTONS values generated on the boundaries of phase space volume<br />
# (Mitch/Charles) Compare the different meshes and evaluate the accuracy of the interpolations<br />
## Using barycenter [[Media:CentroidError.png]] [<span style="color:green">DONE</span>]<br />
# (Mitch/Charles) Change of variables for CFF generation from: ξ-> X<sub>Bj</sub> and t->t' . Reproduce mesh for new variables [<span style="color:green">DONE</span>]<br />
<br />
=== Week July 27 - July 31 ===<br />
# (Spiros) New version of CRTC_I2M docker file and documentation. [<span style="color:green">DONE</span>]<br />
## The Changes for PODM3D are:<br />
### --min-edge is now a parameter that expresses the percentage of the diagonal that will be used as min-edge. Default 1/ sqrt(3).<br />
### (New parameter) --max-edge is a parameter that expresses the percentage of the diagonal that will be used as max-edge. Default 10.<br />
### (New parameter) --parallel-mesh-results is a flag that computes the mesh results in parallel if more than 1 thread is used.<br />
### (New parameter) --file-format is a parameter that defines the output format BINARY or ASCII. Default binary.<br />
# (Mitch) Generate tessellations in X<sub>Bj</sub> and t' for imaginary and ln(real) parts of CFF using new PODM3D docker<br />
## Plot interpolation error at the centroid<br />
# (Mitch) Outline a code to compute DVCS cross-section<br />
## Understand/modify the Bethe Heitler CS code <br />
## Work through the Dirac algebra<br />
<br />
=== Week Aug 3 - Aug 7 ===<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities] [<span style="color:green">DONE</span>]<br />
## Get distance-based weight mean value of a point using k nearest points <br />
# (Charles, Mitch) New sizing function S[f,g]<br />
## f = Im[CFF], g = Re[CFF]<br />
## for each simplex with points '''X'''(''i'') ''i=0,1,2,3,'' and functional values ''f(i), g(i)'' [[File:SizingReIm.png]]<br />
## New variables: Log10(xBj), Sqrt[t']/(0.5 GeV), Log2(Q2)<br />
# (Charles, Mitch) Competition to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
## S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
## Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
<br />
=== Week Aug 10 - Aug 14 ===<br />
# (Spiros) Implement Sizing Functions with real and imaginary images [<span style="color:green">DONE</span>]<br />
## [[File:SizingReIm.png]]<br />
## Element weight range based:<br />
### elementWeightRangeF = abs(max(f) - min(f))<br />
### elementWeightRangeG = abs(max(g) - min(g))<br />
### Condition1: elementWeightRangeF / imageWeightRangeF + elementWeightRangeG / imageWeightRangeG > 2 * elementWeightLimit<br />
### Condition2: elementWeightRangeF / imageWeightRangeF > elementWeightLimit && elementWeightRangeG / imageWeightRangeG > elementWeightLimit<br />
## Results: will be produced on the spot<br />
# (Spiros) [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing Efficient Monte Carlo Simulation using a Compton Form Factor Tessellation as a smart lookup table ] [<span style="color:green">DONE</span>]<br />
# (Spiros) Slice of 4D mesh<br />
## Current implementation does not produce results with cells.<br />
# (Mitch/Spiros) Update slides Look-up Table pipeline [https://docs.google.com/presentation/d/1HhL_PECZmL4CYFbrk65z-XIiCe1N0gWYS8gotFNfuyE/edit?usp=sharing]<br />
# (Mitch) Review the Foam event generator [https://drive.google.com/file/d/1VOgkSdV6VUA_ZmufPRsQSLyP6AJBdg_Y/view?usp=sharing Foam]<br />
# (Mitch) Produce meshes for ln(|real|) and imaginary for new variables using uniform tessellation and new sizing functions in (1)<br />
## [[Media:uH_lnReal.png|Uniform mesh of the ln(|real|) part]] and [[Media:lnRealNoCrinkle.gif|Gif of evolution of the ln(|real|) part]]<br />
## [[Media:CentroidError_lnReal.png|Barycenter error]]<br />
=== Week Aug 17 - Aug 21 ===<br />
# (All) Define the structure of the paper<br />
<br />
=== Week Aug 24 - Aug 28 ===<br />
# (Spiros/Wirawan/John/Mitch) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation] on Wahab<br />
## (Wirawan/John/Spiros/) Sequentially<br />
### (John) Set up a MySQL server on Wahab for partons [<span style="color:green">DONE</span>]<br />
### (Wirawan) Create a container with all the required dependencies [<span style="color:green">DONE</span>]<br />
### (Spiros) Set it up and test it [<span style="color:green">DONE</span>]<br />
## (Spiros/Mitch) In a distributed way <br />
### (Spiros) Create the required tools to run everything in a distributed way [<span style="color:green">DONE</span>]<br />
### (Spiros) Test the ability to do it [<span style="color:green">DONE</span>]<br />
### Notes:<br />
#### (Spiros/Mitch) The execution time of the partons calculations on each node differs based on the coordinates of the kinematics that it calculates.<br />
#### (Spiros) First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics.<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### (Spiros) Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. [<span style="color:green">DONE</span>]<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## (Spiros) In a shared memory way:<br />
### Executing multiple instances of the same program on the same node using CPU affinity to specify the CPU core that the process is going to utilize. [<span style="color:green">DONE</span>]<br />
## (Spiros) In a distributed + shared memory way:<br />
### Create a script that runs partons in parallel using shared memory. [<span style="color:green">Done</span>]<br />
### Create a script that runs partons in parallel using distributed + shared memory. [<span style="color:green">Done</span>]<br />
## (Spiros) Speedup:<br />
### [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
# (Mitch) Write script to calculate PARTONS images and generate meshes for all quark flavors and GPD Types.<br />
## 4 quark flavors x 4 twist-2 GPDs x (real+imag) = 32 meshes<br />
## Recalculate PARTONS values of vertices for each mesh<br />
## Script to generate interpolation error histograms for each mesh<br />
## 2D histogram of the standard deviations of each interpolation error histogram [<span style="color:orange">TODO</span>]<br />
<br />
=== Week Aug 31 - Sept 3 ===<br />
# (Spiros) [https://odu.app.box.com/folder/121612352972?s=ebnisy49hevzj5hap674ujnbtn4ujqrg Synthetic 4D data] [<span style="color:green">DONE</span>]<br />
## Create synthetic HyperCube and HyperSphere images<br />
## Generate 4D uniform meshes out of the synthetic images<br />
## Create a reader for the 4D meshes<br />
<br />
=== Week Sept 7 - Sept 11 ===<br />
# (Mitch/Spiros) [https://crtc.cs.odu.edu/CNF:CNF_Data_Generation CNF Data Generation]<br />
## Fix Axis ordering. [<span style="color:green">DONE</span>]<br />
# (Spiros/Mitch) PartonsToNRRD<br />
## Generate Images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
# (Spiros) PODM3D<br />
## Handle cnf images with spacing other than 1. [<span style="color:green">DONE</span>]<br />
## Create meshes inside well-defined geometric domain . [<span style="color:green">DONE</span>]<br />
# (Spiros) [https://crtc.cs.odu.edu/CNF:Smart_LookUp_Table_Utilities Smart lookup table utilities]<br />
## Get neighbors of a cell [<span style="color:green">DONE</span>]<br />
# (Spiros) Interesting paper<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
=== Week Sept 28 - Oct 2 ===<br />
# (Spiros) Extend Monte Carlo Event Generation to work with negative values [<span style="color:green">DONE</span>]<br />
# (Spiros) Generate randomly uniform points inside a cell using: [<span style="color:green">DONE</span>]<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== <span style="color:orange">TODO</span> with a potential supplement ==<br />
# (Spiros) Compile a list of Utilities for pre- and post-processing functions for I2M conversion<br />
## Examples should include: (i) fast search methods, (ii) smart lookup table functionality, etc.<br />
# (All) Identify applications that could benefit from tessellations like nD numerical integration, data preservation, etc..<br />
# (Spiros/Nikos) Explore Christian's request to compute 2D/3D function approximation for square/cube and a given analytic function defined on the square/cube<br />
<br />
= SURA CNF21 Project (C2020-FEMT-006-01) =<br />
== Objectives ==<br />
* Close the adaptivity loop<br />
* Deliver Parallel (Distributed) PARTONS<br />
* Generate 4D tessellations on real data from PARTONS<br />
* Publish a paper for the 3D CFF approximation<br />
<br />
== Short-term Deliverables (6 months) ==<br />
# (Mitch) Use the Shared memory PARTONS to generate 4D data (by March 15) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Generate 4D Tessellations using the 4D data (by March 30) [<span style="color:orange">TODO</span>]<br />
# (Spiros) Deliver the Distributed version of PARTONS to Mitch (by April 15) [<span style="color:orange">TODO</span>]<br />
# (Mitch/Spiros) Generate Larger 4D tessellations and deliver these to N. Polys (by April 30) [<span style="color:orange">TODO</span>]<br />
# (Christos/Spiros) Deliver an adaptive I2M Tessellator, that can add additional points. (by May 15) [<span style="color:orange">TODO</span>]<br />
# (Polykarpos/Christos/Spiros) Deliver preliminary data on Data Preservation and other issues related to CNF data and CFF approximations could be used as a case study (Sept 15)<br />
# (All) work on the next proposals (Sept 30)<br />
<br />
== Weekly Reports ==<br />
=== Week April 26 - May 2 ===<br />
# Using the interpolation error as the sizing function is the way to go to close the "loop" for the following reasons:<br />
## Performs exactly the amount of partons calculations (NO MORE) to guarantee that the interpolation error within every cell is below <= error-threshold<br />
## Minimizes the number of points/cells of the mesh. The benefits of this are:<br />
### Fewer points/cells -> faster Monte Carlo Event Generation<br />
### Fewer points/cells -> faster IO <br />
## Enables the saving of all the CFFs into one mesh rather than creating a mesh for each CFF. The benefits of this are:<br />
### Let's say that we create a mesh for each CFF and every mesh has 100k cells. So the total number of cells needed are: number of CFFs * 100k. <br> Creating the common denominator mesh of all these meshes will reduce the total PARTONS calculations by almost (!) as much as the number of CFFs.<br />
### Additionally, for the Monte Carlo event generation step. If we were to create a random point within a domain, we would have to locate the cell in which this point lies into for every CFFs mesh (complexity = O (log (number of cells) * number of CFFs)). <br> While we could just randomly create a point based on the volume of every cell, as we do it currently just for one CFF (complexity O(1)).<br />
<br />
= References =<br />
# Related Efforts<br />
## Similar Effort identifed by Charles/Mitch to our Tessellation and Smart Look-up Tables from FOAM structure in root.cern.ch software package<br />
### S. Jadach, Computer Physics Communications v.152, number 1 (2003) p.55 [http://www.sciencedirect.com/science/article/pii/S0010465502007555?via%3Dihub]<br />
### Preprint ArXiv.org: [https://arxiv.org/abs/physics/0203033]<br />
# Examples for the Journal paper on "Smart Loolup Tables for MC" <br />
##'''Computer Physics Communications:''' [https://reader.elsevier.com/reader/sd/pii/S0010465520302423?token=62A1DEE0773D7DBA5F1869DD2E080F70865DCBEC7F89378DD98807130C5E565C2F775A9B028B96211CE3F6B49715A1DB ARTEMIS: Ab initio restructuring tool enabling the modeling of interface structures]<br />
## '''Journal of Computational Physics:''' [https://www.sciencedirect.com/science/article/pii/S0021999117300670 Physics motivated example paper on tessellations from Journal of Computational Physics]<br />
## '''Nature:''' [https://www.nature.com/articles/s41586-019-0925-9 Nature paper from JLab Nuclear Physics], [https://arxiv.org/pdf/2004.12065.pdf PDF]<br />
## Other<br />
### [https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.21973 Paper relevant to tessellation and Monte Carlo calculations from Journal of Computational Chemistry]<br />
# Other papers<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
## [https://link.springer.com/chapter/10.1007%2F978-3-319-76201-2_3 Interpolation on Unstructured Tetrahedral Grids]<br />
<br />
= Restricted Access (available to CNF Imaging Project i.e., CRTC/JLAB only) =<br />
* [https://odu.box.com/s/ce6wc1ht1f96jx7lpafys1ue1piiyekc Shared box directory] <br />
* [https://git-community.cs.odu.edu/crtc/Mesh-Generation/Parallel-Mesh-Generation/podm-superbuild PODM Docker image source files] <br />
* [https://git-community.cs.odu.edu/crtc/websites/podm-documentation CRTC_I2M_Suite website build files]<br />
* Adaptivity<br />
** [[CNF:Mesh Sizing for CNF data]]<br />
** [[CNF:CNF_Adaptivity]]<br />
* [[CNF:Visualization for CNF data]]<br />
* [[CNF:Machine Learning]]<br />
<br />
* https://www.jlab.org/indico/event/335/<br />
* [http://www.cs.odu.edu/crtc/videos/PODM3D.html Video of meshing procedure]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Smart_LookUp_Table_Utilities&diff=8458CNF:Smart LookUp Table Utilities2021-04-21T17:46:01Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/smartlookuptableutilities Gitlab]<br />
# [https://github.com/spyridon97/SmartLookUpTableUtilities Github]<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Find Cell that includes a point (2D/3D)✔<br />
# Get points of a cell (2D/3D/4D) ✔<br />
# Get points weights of a cell (2D/3D/4D) ✔<br />
# Get the volume of a cell (2D/3D/4D) ✔<br />
# Get centroid coordinates of a cell (2D/3D/4D) ✔<br />
# Get the interpolated weight of a point inside a cell (2D/3D/4D) ✔<br />
<br />
== Fall 2020 ==<br />
# Get neighbors of a cell (3D) ✔<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔<br />
# Find Cell with exact predicates [TODO]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8457Internal:GPCD2021-04-20T18:39:49Z<p>Spyridon97: /* (Angelos/Spiros) Cavity Detection */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 8. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Spiros) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 20.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8456Internal:GPCD2021-04-20T18:39:01Z<p>Spyridon97: /* (Angelos/Spiros) Cavity Detection */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = 2 * minEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### (Angelos) Esp = 5. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Spiros/Angelos) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 10.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8455Internal:GPCD2021-04-19T15:11:27Z<p>Spyridon97: /* (Spiros/Αngelos) Compute the Solvent Excluded Image */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's implementation [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = maxEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### (Angelos) Esp = 5. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Spiros/Angelos) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 10.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8454Internal:GPCD2021-04-19T15:11:17Z<p>Spyridon97: /* (Spiros/Αngelos) Compute the Solvent Excluded Image */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the optimal bounding box of the protein chain using CGAL's algorithm [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = maxEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### (Angelos) Esp = 5. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Spiros/Angelos) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 10.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:GPCD&diff=8453Internal:GPCD2021-04-19T14:51:40Z<p>Spyridon97: /* (Spiros/Αngelos) Compute the Solvent Excluded Image */</p>
<hr />
<div>== Introduction ==<br />
<br />
=== Protein Cavities ===<br />
Cavities are structural features of proteins and they are often important for drug discovery and design. There are different kinds of cavities, including pockets, voids, tunnels, and channels. The types of cavities are listed in more detail in figure 1.<br />
<br />
<br />
[[File:proteinCavityTypes.jpg|center|frame| Figure 1: The basic types of protein cavities [1]]]<br />
<br />
<br />
# '''Void''': A cavity that looks like internal "hole" in a protein with no connection to the surface.<br />
# '''Cleft''': A pocket that looks like a "dent" in the surface of a protein. Can also be found as part of a tunnel or channel.<br />
# '''Invagination''': A pocket that looks like a "cove". It has a small opening on the surface of a protein and leads to a larger interior. Can also be found as part of a tunnel or after a cleft.<br />
# '''Tunnel''': A pocket that has an opening on the surface of a protein and leads to somewhere in the interior. Has no opening on the other side.<br />
# '''Channel''': A cavity that has an opening on one side of the surface of a protein and leads to another opening in another location on the surface of the protein.<br />
<br />
<br />
Of course, the types of cavities listed in figure 1 are only the basic ones. Combinations of these are also possible as shown in figure 2.<br />
<br />
<br />
[[File:proteinCavityAdvancedTypes.jpg|center|frame| Figure 2: Combination types of protein cavities [1]]]<br />
<br />
=== Protein Cavity Detection Methods ===<br />
There are several methods for detecting protein cavities. There are sphere based methods, grid based methods, grid-and-sphere based methods, surface based methods, grid-and-surface based methods, time-varying methods, GPU based methods, consensus based methods, and finally tessellation based methods. These algorithms are shown in more detail in figure 3. Tessellation based methods mainly rely on Voronoi tessellation. Some methods rely on weighted Voronoi tessellation because of its ability to more accurately represent the volume of atoms. The Voronoi tessellation in these methods is computed from a set of points representing the atomic centers of the underlying protein.<br />
<br />
<br />
[[File:proteinCavityDetectionAlgorithms.jpg|center|frame| Figure 3: A hierarchy of protein cavity detection algorithms [1]]]<br />
<br />
<br />
== Literature Review ==<br />
<br />
=== MOLE ===<br />
<br />
==== Method ====<br />
The MOLE algorithm is specifically designed to identify tunnels and channels. Finding superficial cavities is a secondary concern and does so as part of the surface approximation step. MOLE utilizes the surface obtained by the refinement of the convex hull to find superficial cavities such as clefts and invaginations. In essence, the approximation of a more accurate surface and the detection of these superficial cavities are closely related. As tetrahedra are removed from the convex hull to get a more accurate surface approximation, a parameter called ''interior threshold'' controls how many superficial cavities are found. For example, if the number of tetrahedra removed from a specific part of the molecule exceeds this threshold, the algorithm marks these tetrahedra as part of a superficial cavity.<br />
<br />
<br />
[[File:moleVoronoiTunnelPath.jpg|center|frame| Figure 4: Shows in 2D the Voronoi diagram (thin lines) and the convex hull (dotted line) of molecule represented by atoms (circles). Each Voronoi edge is evaluated by the cost function (numbers). The thick line represents the optimal path found by the Dijkstra's algorithm from the given starting point (black circle), i.e., centerline of found tunnel. [1]]]<br />
<br />
<br />
For the more complicated types of cavities, tunnels and channels, MOLE uses the Voronoi diagram to identify them. Start and end points are defined as the centers of the deepest tetrahedra in each cavity. The depth of the tetrahedron is defined as the number of Voronoi edges from the closest boundary tetrahedron (on the surface). Tunnels/channels are computed as the shortest paths between all pairs of start and end points in the same cavity diagram.<br />
<br />
[[File:MoleDijkstraWeightFunction.png|center|frame| Figure 5: Shows the weight function used to estimate the weight of each Voronoi edge for Dijkstra's algorihtm. l(e) is the length of the edge, d(e) is the distance of the edge to the closest atom Van der Waals sphere and ϵ is a small number to avoid division by zero [1]]]<br />
<br />
==== Complexity ====<br />
* Worst-case: O(N^2 log N), where N is the number of atoms in the molecule. <br />
* Average: O(M log M), where M is the number of vertices in the Voronoi diagram (in the worst case, M = N^2, but in most cases M = O(N)). As a result of the use of the incremental algorithm and Hilbert curve ordering, the complexity of calculating the Delaunay triangulation of most molecular structures is O(N log N). Finally, the complexity of all the remaining steps of the algorithm is at most O(M log M).<br />
<br />
<br />
=== MolAxis ===<br />
<br />
==== Method ====<br />
The MolAxis algorithm relies on Voronoi and Dijkstra's algorithm to find channels, similarly to MOLE. The algorithm consists of the following high-level steps:<br />
<br />
# Approximate the molecule using a collection of '''fixed-size spheres'''. The number of balls needed depends on the required quality of the approximation. In the extreme case, where the user is interested in a high resolution, each atom is approximated using a single sphere. <br />
# Compute the '''Voronoi diagram''' of the centers of the fixed-sized spheres. The diagram is intersected with a user-defined bounding sphere. This defines the area of interest.<br />
# Discard the parts of the Voronoi diagram that intersect the molecule.<br />
# Define a '''graph''', using the Voronoi vertices and edges. A weight is given to each edge, which depends on its length and on its distance from the surface of the molecule. <br />
# Define a '''source point (root vertex)'''. The user can either supply a source point or allow MolAxis to automatically choose the source point to be the '''center of the largest cavity in the molecule''' (basically detect channels inside-out).<br />
# Apply a shortest path optimization algorithm ('''Dijktra's algorithm''') on the graph, computing a tree called the "corridor tree".<br />
# The desired channels are paths in the corridor tree. Duplicate channels are avoided by using the split distance parameter. The split distance bounds the distance to the least common ancestor of the channel and all previously reported channels.<br />
<br />
<br />
[[File:MolAxisVoronoi.png|center|frame| Figure 6: Shows an example of fixed-size circles (light-blue) approximating varying-size circles and the pathway diagram of their centers. The discarded portion of the Voronoi diagram of their centers is depicted using dotted lines [2]. This figure also helps illustrate the usefulness of the Voronoi tessellation in computing pathways.]]<br />
<br />
== Protein Energetics ==<br />
=== Metrics ===<br />
# '''Hydrophobicity''': Physical property of a molecule that is seemingly repelled from a mass of water. Hydrophilic molecules are attracted to water. Hydrophobicity plays a major role in protein folding. Hydrophobic molecules also tend to be nonpolar and prefer other neutral molecules and nonpolar solvents. Hydrophobicity is the most widely used metric.<br />
# '''Hydrogen Bonds''': Weak bonds between two molecules resulting from an electrostatic attraction between a proton in one molecule and an electronegative atom in the other. We often see these bonds between oxygen and hydrogen.<br />
# '''Van der Waals Interactions''': Also known as Van der Waals forces, they are forces driven by induced electrical interactions between two or more atoms or molecules that are very close to each other. Van der Waals interaction is the weakest of all intermolecular attractions between molecules. These forces are distance-dependent.<br />
# '''Salt Bridges''': Non-covalent (no sharing of eletrons; electromagnetic) interaction between two ionized sites. It has two components: a hydrogen bond and an electrostatic interaction.<br />
# '''Disulfide Bridges''': Strong bonds that can form between two cysteines (an amino acid). The strength of disulfide bridges helps stabilize a protein. Disulfide bridges are especially common in proteins that are secreted from cells.<br />
# '''Charge''': Plays a role in strength of bonds. A higher charge means that an atom will have stronger forces in an electromagnetic field.<br />
# '''Polarity''': Separation of electric charge leading to a molecule or its chemical groups having an electric dipole moment, with a negatively charged end and a positively charged end. Indicates the existence of polar bonds, which are covalent (sharing of electrons) and are quite strong. About [https://chem.libretexts.org/Bookshelves/Inorganic_Chemistry/Book%3A_Introduction_to_Inorganic_Chemistry/01%3A_Review_of_Chemical_Bonding/1.04%3A_Bond_Polarity_and_Bond_Strength bond polarity].<br />
<br />
== Candidate Method: Geometric Protein Cavities Detection ('''GPCD''') ==<br />
The Pipeline contains the following high-level algorithmic steps:<br />
<br />
=== (Angelos/Spiros) Read protein ===<br />
# Input: PDB file<br />
# Output: A Polydata of the protein<br />
# Algorithmic Step:<br />
## (Angelos) Using vtkPDBReader, read the PDB file that included the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
# Challenges:<br />
## (Angelos) Upgrade vtkPDBReader to be able to manage multimodel proteins [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Fix vtkPDBReader error of setting wrong atom numbers [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Set up vtkPDBReader to utilize [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] for the Covalent radius and Van-der-Waals radius [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Ideas that might be used:<br />
## Improve [https://sourceforge.net/projects/bodr/ The Blue Obelisk Dataset] by using: [<span style="color:orange">TODO</span>]<br />
### updated Van-der-Waals radius values provided by Los Alamos National Laboratory<br />
### updated Covalent radius values provided by Cambridge Structural Database<br />
<br />
=== (Spiros) Extract unique protein Chains ===<br />
# Input: A Polydata of the protein<br />
# Output: Unique protein chains without het-atoms and Symmetric Information<br />
# Algorithmic step:<br />
## Implement a filter that removes the het-atoms and creates the polydata of the protein chains. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Detect the unique protein chains using Symmetry detection techniques ([https://en.wikipedia.org/wiki/Kabsch_algorithm Kabsch algorithm]) [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Store the Symmetry information (rotation and translation). [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: This way we can mesh each unique chain individually<br />
<br />
=== (Spiros) Sort Atoms By Radius ===<br />
# Input: A Polydata of a unique protein chain without Het atoms<br />
# Output: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order<br />
# Algorithmic step:<br />
## Using vtkSortFieldData we sort the non-Het atoms by radius in descending order. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed using vtkSMPTools [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros/Αngelos) Compute the Solvent Excluded Image ===<br />
# Input: A Polydata of a unique protein chain without Het atoms sorted by radius in descending order, probe radius, spacing<br />
# Output: Solvent Excluded Image of a chain<br />
# Algorithmic steps:<br />
## (Spiros) Calculate the minimum bounding box of the protein chain using CGAL's algorithm [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## (Spiros/Angelos) Create the Solvent Accessible image by creating spheres for each atom that their radius is: Van der Waals radius + probe radius. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos) Perform Euclidean Distance Transform on the Solvent Accessible image using the '''itk::SignedMaurerDistanceMapImageFilter'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
## (Spiros) Extract Solvent Excluded image from the EDT result by setting: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### All pixel values < probe-radius - truncationError[atomType] - paddingDistance = 0<br />
### All pixel values >= probe-radius - truncationError[atomType] - paddingDistance = 1<br />
### '''Note 1''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 2''': GPU parallelism can be employed [TODO]<br />
# Challenges:<br />
## (Spiros) Handle the calculation of truncationError[atomType] in a thread-safe way by sorting the atoms by radius in descending order [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Truncation error is caused because of the spacing of the image.<br />
## (Spiros/Angelos) Select a paddingDistance value to help PODM3D produce an accurate surface [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Spiros) Close holes that should not exist using the distance of the pixel point vs the actual atom [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## (Spiros) Create smaller images using the concept that is described in the EDTSurf paper [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros) Compute the Solvent Excluded Surface ('''SES''')===<br />
# Input: Solvent Excluded Image of a chain<br />
# Output: Solvent Excluded Surface of a chain<br />
# Algorithmic steps:<br />
## Using '''PODM3D''', produce a volumetric mesh of the image. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''PODM3D''', extract the volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## Using '''vtkDataSetSurfaceFilter''', extract the surface of volumetric mesh. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note 1''' this filter is used because when PODM is extracting the surface, its triangles are oriented the wrong way.<br />
### '''Note 2''': Shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
# Advantages:<br />
## Can control the number of elements on the surface<br />
## the edge lengths bounds are relatively small which helps in the clustering.<br />
# Challenges:<br />
## Select a default delta value. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
## Fix volume grading crashing. [<span style="color:orange">TODO</span>]<br />
## Surface needs to be smoothed. [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Extract Atom Types of surface points ===<br />
# Input: A Polydata of a unique protein chain surface and a Polydata of the unique protein chain without Het atoms sorted by radius in descending order<br />
# Output: Atom Types of surface points of the unique protein chain surface<br />
# Algorithmic step:<br />
## Using a KD-tree, for each surface point, find the closest atom: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### If the surface point is within the radius of the closest atom, assign the atom type of the closest point to the surface point<br />
### Else, assign '''0''', which means the surface point is part of the re-entrance surface.<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## Find an ideal offset distance value between the surface points and their closest atoms to improve accuracy. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
<br />
=== (Spiros) Generate the Symmetric Protein chain surfaces ===<br />
# Input: A vector of the unique protein chain surfaces and the Symmetry information<br />
# Output: A vector of all the protein chain surfaces<br />
# Algorithmic step:<br />
## Implement the '''SymmetricProteinChainsSurface''' filter that generates the non unique protein chain surfaces using the Symmetry information. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Assign Model and Chain arrays on SESs ===<br />
# Input: A vector of all the protein chain surfaces & model and chains information<br />
# Output: A vector of all the protein chain surfaces with model and chain arrays<br />
# Algorithmic step:<br />
## Implement the '''AssignModelAndChain''' filter that assigns the model and chain arrays for each protein chain surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Spiros) Merge Protein Chain SESs ===<br />
# Input: A vector of all the protein chain surfaces with model and chain arrays<br />
# Output: Solvent Excluded Surface<br />
# Algorithmic step:<br />
## Implement the '''MergeSurfaces''' filter that merges all the protein-chains surfaces into one and forms the solvent excluded surface of the protein. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
<br />
=== (Angelos/Spiros) Compute ambient occlusion of the SES === <br />
# Input: Solvent Excluded Surface<br />
# Output: Ambient Occlusion values of surface points, normals of surface points<br />
# Algorithmic step:<br />
## (Angelos/Spiros) Using IGL Library, which uses [https://www.embree.org/ Embree] and [https://eigen.tuxfamily.org/dox/ Eigen], calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
## (Angelos/Spiros) Using the CUDA and Optix library, calculate normals of surface points and then calculate Ambient occlusion values for all the surface points. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### '''Note''': GPU parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## (Spiros/Angelos) Eliminate the noise that is created due to the non-smoothed '''PODM3D''' surface by assigning for each vertex the average ambient occlusion value of its neighbors + itself. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
# Ideas that might be used:<br />
## Use the symmetry information to minimize the memory footprint for CPU and GPU [<span style="color:orange">TODO</span>]<br />
<br />
===(Spiros) Compute Surface Statistics ===<br />
# Input: Solvent Excluded Surface<br />
# Output: Statistics of Surface Triangles' Edges<br />
# Algoritmic step:<br />
## Compute min/mean/median/max edge lengths of the surface triangles [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': Shared memory parallelism can be employed [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Angelos/Spiros) Cavity Detection ===<br />
# Input: Solvent Excluded Surface, Ambient Occlusion values of surface points, Surface Statistics<br />
# Output: Protein Cavities<br />
# Algorithmic step:<br />
## (Angelos) Using a modified version of the clustering method '''DBSCAN''', that finds neighbors based on their density '''and''' ambient occlusion values, form clusters that define protein cavities. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### '''Note''': shared memory parallelism can be employed [<span style="color:orange">TODO</span>]<br />
#### Idea 1: [https://link.springer.com/article/10.1023/A:1009884809343 A Fast Parallel Clustering Algorithm for Large Spatial Databases PDBSCAN]<br />
#### Idea 2: Speculative Execution using '''PREMA'''<br />
## (Angelos) Implement a rendering framework to visualize the clustering process using '''VTK'''. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
# Challenges:<br />
## Tune the clustering parameters:<br />
### (Spiros) Search-radius = maxEdgeLength of the surface. [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### (Angelos) Esp = 5. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Spiros/Angelos) Min cavity size = cavityFactor * delta^-2.178. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
#### Default value of cavityFactor = 10.<br />
## Limit the expansion of the cavity by using: <br />
### (Spiros) A relative threshold for the difference an occlusion value from the average occlusion value of the cluster. [<span style="color:green">DONE</span>] [<span style="color:purple">Fall 2020</span>]<br />
### (Angelos) Define weights for the occlusion values, and convert the mean to a weighted mean. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Define occlusion values bounds to detect different types of cavities [<span style="color:orange">TODO</span>]<br />
# Ideas that might be used:<br />
## [https://dl.acm.org/doi/abs/10.1145/304181.304187 OPTICS: ordering points to identify the clustering structure] [<span style="color:orange">TODO</span>]<br />
## Use symmetry information to quickly detect the per-model cavities [<span style="color:orange">TODO</span>]<br />
<br />
=== (Spiros/Angelos) Map Electrostatics Potentials on Surface Points ===<br />
# Input: Solvent Excluded Surface, Electrostatic Potentials Image '''ESP''' of [http://www.poissonboltzmann.org/ APBS], normals of surface points, probe radius, lowerThreshold-upperThreshold<br />
# Output: Electrostatic values of surface points<br />
# Algorithmic steps:<br />
## (Spiros/Angelos) For each surface point: [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
### Scale the point using normals, and probe radius to the solvent-accessible surface, <br />
### Get the value of the closest pixel from the '''ESP''' to the scaled point:<br />
#### If value > upperThreshold, set electrostatic value of point = upperThreshold<br />
#### Else if, value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
#### Else get the interpolated value of the scaled points from the image:<br />
##### If interpolated value > upperThreshold, set electrostatic value of point = upperThreshold<br />
##### Else if, interpolated value < lowerThreshold, set electrostatic value of point = lowerThreshold<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
# Challenges:<br />
## (Angelos/Spiros) Create reader of the [https://www.ics.uci.edu/~dock/manuals/apbs/html/user-guide/x2674.html OpenDx] file format that is used as output by APBS [<span style="color:green">DONE</span>] [<span style="color:purple">Summer 2020</span>]<br />
<br />
=== (Spiros) Select Clustered cavity points ===<br />
# Input: Labeled Solvent Surface Surface<br />
# Output: Labeled clustered cavity points<br />
# Algorithmic step:<br />
## Implement a SelectPoints filter. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
## Select only the points that are clustered at the cavity detection step. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2021</span>]<br />
### Note: this is done to improve the speed of writing the output<br />
<br />
<br />
=== (Angelos/Spiros) Write Results: ===<br />
# Input: Labeled clustered cavity points or Labeled Solvent Surface Surface<br />
# Output: solventSurface.vtp<br />
# Algorithmic step:<br />
## (Angelos) Using a vtkXMLPolyDataWriter, write the output file. [<span style="color:green">DONE</span>] [<span style="color:purple">Spring 2020</span>]<br />
## (Angelos/Spiros) Create a writer, just for the cavities to minimize the IO. [<span style="color:orange">TODO</span>]<br />
### Idea: See '''Fpocket'''.<br />
# Ideas that might be used:<br />
## Note: a Paraview plugin may be developed for visualization purposes. [<span style="color:orange">TODO</span>]<br />
<br />
== Future Work ==<br />
<br />
=== (Angelos/Spiros) Classify cavity type ===<br />
# Input: Solvent Excluded Surface, Atomic Cavities, Atom Types of surface points<br />
# Output: Labeled atomic Cavities<br />
# Algorithm steps:<br />
# Ideas that might be used: <br />
## Ambient occlusion values based classification using mean distance cluster value [<span style="color:orange">TODO</span>]<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
## Analyze the shape of clusters to identify what type of cavity they are. [<span style="color:orange">TODO</span>]<br />
### This can potentially be done using machine learning.<br />
### '''Note''': Shared memory parallelism can be used [<span style="color:orange">TODO</span>]<br />
<br />
== Results ==<br />
=== GPCD vs MOLE ===<br />
{| class="wikitable"<br />
|-<br />
! scope="col" | Feature<br />
! scope="col"| GPCD<br />
! scope="col"| MOLE<br />
|-<br />
! scope="row"| Detect Surface Cavities<br />
| style="background-color: green; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Detect Interior Cavities<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Visualize Exact Channel Shape<br />
| style="background-color: red; color: white;" | NO<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| Works on Very Large Molecules<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO (very slow)<br />
|-<br />
! scope="row"| Works on Multi-Model Files<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Parallelization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|-<br />
! scope="row"| Relative Performance<br />
| style="background-color: green; color: white;" | VERY FAST<br />
| style="background-color: red; color: white;" | SLOW<br />
|-<br />
! scope="row"| Intuitive Visualization<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: green; color: white;" | YES<br />
|-<br />
! scope="row"| False Positives<br />
| style="background-color: orange; color: white;" | YES (more)<br />
| style="background-color: green; color: white;" | YES (less)<br />
|-<br />
! scope="row"| Quantify Major Structural Features<br />
| style="background-color: green; color: white;" | YES<br />
| style="background-color: red; color: white;" | NO<br />
|}<br />
<br />
== Resources ==<br />
<br />
=== Software ===<br />
<br />
* '''[https://www.rbvi.ucsf.edu/chimerax/ UCSF ChimeraX]''': Next-generation molecular visualization program from the Resource for Biocomputing, Visualization, and Informatics (RBVI).<br />
* '''[https://www.ks.uiuc.edu/Research/vmd/ VMD]''': Molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3D graphics and built-in scripting. Used in conjunction with NAMD for visualizing molecular dynamics simulations.<br />
* '''[https://www.ks.uiuc.edu/Research/namd/ NAMD]''': Parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Scales to hundreds of cores for typical simulations and beyond 500,000 cores for the largest simulations.<br />
* '''[https://webchem.ncbr.muni.cz/Platform/App/Mole MOLE]''': Universal toolkit for rapid location and characterization of channels and pores in biomacromolecules.<br />
* '''[https://caver.cz/ CAVER]''': Software tool for the analysis and visualization of tunnels and channels in protein structures.<br />
* '''[https://www.rcsb.org/ RCSB PDB Protein Data Bank]''': The largest and most popular database of protein structures.<br />
<br />
=== Papers ===<br />
<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5839519/ Survey: Geometric Protein Cavity Detection Algorithms]''': Paper surveying the current geometric algorithms for protein cavity detection. This is a great resource for understanding this project.<br />
* '''[https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-5-39 MOLE 2.0]''': Paper describing the MOLE 2.0 protein cavity detection software. Contains a good algorithm that can be used for extracting a good approximation of the surface of a point cloud of atoms.<br />
* '''[https://reader.elsevier.com/reader/sd/pii/S0969212607003759?token=4D0F6192E971D4406E8F7CBA047B0490C8715EBD67579D26427E0E63BD0298BEBF948DB238A28E4EAE2D17EBC6DF4E50 MOLE 1.0]''': Paper describing the MOLE 1.0 algorithm.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/epdf/10.1002/prot.22052 MolAxis]''': Paper describing the MolAxis protein cavity detection software.<br />
* '''[https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002708 CAVER 3.0]''': Paper describing the CAVER 3.0 protein cavity detection software. This algorithm specializes in identifying tunnels.<br />
* '''[https://www.sciencedirect.com/science/article/abs/pii/S0022283606008746 Travel Depth]''': Paper describing the "travel depth" algorithm for protein cavity visualization.<br />
* '''[https://onlinelibrary-wiley-com.proxy.lib.odu.edu/doi/full/10.1111/cgf.12120 GPU Protein Cavity Detection using Ambient Occlusion]''': Paper describing a GPU-based protein cavity detection algorithm that utilizes Ambient Occlusion.<br />
* '''[https://www.aaai.org/Papers/KDD/1996/KDD96-037.pdf DBSCAN]''': Paper describing the DBSCAN clustering algorithm.<br />
* '''[https://link.springer.com/article/10.1023/A:1009884809343 PDBSCAN]''': Paper describing a parallel version of DBSCAN.<br />
* '''[https://pubmed.ncbi.nlm.nih.gov/31133031/ Coronavirus envelope protein]''': Paper describing research on the envelope protein of coronaviruses.<br />
* '''[https://www.cell.com/biophysj/fulltext/S0006-3495(20)33168-4 COVID-19 virion model]''': Paper describing a detail model of the SARS-CoV-2 virus particle.<br />
* '''[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7685317/pdf/nihpp-2020.11.19.390187.pdf COVID-19 simulation]''': Paper describing large-scale COVID-19 simulations.<br />
<br />
=== Organizations/Groups ===<br />
<br />
* '''[https://www.ks.uiuc.edu/ University of Illinois at Urbana-Champaign: Theoretical and Computational Biophysics Group]''': Perhaps the most important group in molecular dynamics. Developers of VMD and NAMD. Ran the simulation for the 200 million atom model of COVID-19.<br />
* '''[https://www.ornl.gov/ Oak Ridge National Laboratory (ORNL)]''': Major national lab with active involvement in drug discovery and design using supercomputers.<br />
* '''[http://rbvi.ucsf.edu/ Resource for Biocomputing, Visualization, and Informatics (RBVI)]''': Developers of UCSF Chimera and UCSF ChimeraX.<br />
* '''[https://ist.ac.at/en/research/edelsbrunner-group/ IST Austria: Edelsbrunner Group]''': Important group in algorithms, computational geometry, and computational topology. Involved with protein cavity detection in the past and potentially involved now. The head of the group, Edelsbrunner created alpha shapes.<br />
* '''[https://www.tacc.utexas.edu/systems/frontera Texas Advanced Computing Center]''': Maintainers of the Frontera supercomputer, the fifth fastest supercomputer in the world (2020). The simulation for the 200 million atom model of COVID-19 was executed on this supercomputer.<br />
* '''[https://webchem.ncbr.muni.cz/ WebChemistry]''': Developed MOLE and many other software tools related to molecular analysis.<br />
<br />
=== Important Links ===<br />
<br />
* '''[https://www.hpcwire.com/off-the-wire/coronavirus-massive-simulations-completed-on-frontera-supercomputer/ COVID-19 200 million atom NAMD simulation]'''<br />
* '''[https://www.ornl.gov/news/early-research-existing-drug-compounds-supercomputing-could-combat-coronavirus/ ORNL COVID-19 spike drug identification]'''<br />
<br />
=== Proteins ===<br />
<br />
* '''[https://www.rcsb.org/structure/1jj2 Large Ribosomal Subunit (98,693 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VXX COVID-19 Spike Glycoprotein, Closed State (23,694 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6Y2E COVID-19 Main Protease (5,370 atoms)]'''<br />
* '''[https://www.rcsb.org/structure/6VWW COVID-19 NSP15 Endoribonuclease (24,036 atoms)]'''<br />
<br />
== References ==<br />
[1] Simões, T., Lopes, D., Dias, S., Fernandes, F., Pereira, J., Jorge, J., Bajaj, C., & Gomes, A. (2017). Geometric Detection Algorithms for Cavities on Protein Surfaces in Molecular Graphics: A Survey. Computer graphics forum : journal of the European Association for Computer Graphics, 36(8), 643–683. https://doi.org/10.1111/cgf.13158<br />
<br />
[2] Yaffe, Fishelovitch, Wolfson, Halperin, & Nussinov. (2008). MolAxis: Efficient and accurate identification of channels in macromolecules. Proteins: Structure, Function, and Bioinformatics, 73(1), 72-86.<br />
<br />
[3] Coleman, R. G., & Sharp, K. A. (2006). Travel depth, a new shape descriptor for macromolecules: application to ligand binding. Journal of molecular biology, 362(3), 441–458. https://doi.org/10.1016/j.jmb.2006.07.022</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Student_Reports&diff=8452Internal:Student Reports2021-04-14T21:17:26Z<p>Spyridon97: /* Kevin */</p>
<hr />
<div>== Eleni, Thomas , Angelos==<br />
[[Internal:ReportCloud | Cloud Report]]<br />
<br />
[[Internal:Preprocess_Medical_Images | Preprocessing of Medical Images]]<br />
<br />
[[Internal:Execution_And_Evaluation_Medical_Images | Execution and Evaluation of Medical Imaging data]]<br />
<br />
== Angelos ==<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
[[Internal:APBNRRExecutionStatus | APBNRR Execution Status]]<br />
<br />
[[Internal:PAPBNRRComparison | Pristine vs New PAPBNRR]]<br />
<br />
[[Internal:APBNRRMachineLearning | APBNRR Machine Learning]]<br />
<br />
[[Internal: SlicerExtensions | Slicer Extensions]]<br />
<br />
== Joi ==<br />
[[Tasks | Tasks ]]<br />
<br />
[[I2M_Conversion_Tool | Image-to-Mesh (I2M) Conversion Tool]]<br />
<br />
[[3D_Slicer_CBC3D_Extension | 3D Slicer CBC3D Extension ]]<br />
<br />
== Christos ==<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/Papers All papers]'''<br />
<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/presentations All presentations]'''<br />
<br />
<hr><br />
[https://www.overleaf.com/6438559258mdmsnbpffztx speculative meshing operations using tasks paper - overleaf]<br />
<br />
[https://www.overleaf.com/3459466599cnnfkhzmkczs Adaptive CDT3D paper - overleaf]<br />
<br />
[https://www.overleaf.com/8242192311zntgrtsphkvv Parallel Grid Adaptation Paper (UGAWG) - overleaf ]<br />
<br />
<br />
<br />
[[Internal:Adaptive_Meshing_For_APBNRR]]<br />
<br />
[https://docs.google.com/presentation/d/1Fjt5PoDPXa715nOAjn0F4IJpdhscIGnlaQvweqFnB0U Adaptivity PPoSS slides]<br />
<br />
[https://docs.google.com/presentation/d/1RDtQufImuo5J1pKioIYDuTWCeX1UtjKyx7kQ2i24Tmo/edit?usp=sharing CNF HPC slides]<br />
<br />
<br />
[https://git-community.cs.odu.edu/nikos/SciTech_2019/blob/draft/main.pdf Scitech Journal paper latest draft]<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/cand_exam.pdf Candidacy exam (document)]<br />
[https://olddominion-my.sharepoint.com/:p:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy_presentation.pptx (presentation .pptx)]<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy%20_presentation.pdf (presentation pdf)]<br />
<br />
[[Internal:Adaptive_CDT3D_Results | Metric Adaptive CDT3D results]]<br />
<br />
[[Internal:Argobots_and_CDT3D | Argobots and CDT3D]]<br />
<br />
[[Internal:4D_Predicates_Benchmark | 4D predicates benchmark]]<br />
<br />
[[CNF:Mesh Sizing for CNF data]]<br />
<br />
<br />
<br />
<br />
[[Internal:ACDT3D | Metric Adaptive CDT3D Notes ]]<br />
<br />
<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/paper.pdf Parallel Mesh Generation review paper]<br />
<br />
<br />
[https://lbpw.larc.nasa.gov/ AIAA Sonic Boom Prediction Workshop (next in 2020)]<br />
<br />
[[Internal:CDT3D_SU2 | SU2 and grid requirements ]]<br />
<br />
[[Internal:Boundary_Layer_References | Boundary Layer References ]]<br />
<br />
[[External:Mesh_Generation_Articles_In_Journals | Some Mesh Generation Journal papers]]<br />
<br />
[https://docs.google.com/document/d/1txyCu6ibheH19hi3nZ10LgHm-WzglUrT5y0AaO__VYk/edit?usp=sharing PhD Tentative Shedule]<br />
<br />
<br />
[https://odu.box.com/s/sfkd5l3d9r5865qbsovx0eo6gwlaiask Poster IMR]<br />
<br />
[https://odu.box.com/s/1sok1edideblq4nctbufg79vud0oiu8f NIA presentation (September 2018) ] <br />
<br />
[[Internal:IMR_2018_Short_Course | IMR 2018 Short Course notes]]<br />
<br />
<br />
<br />
[[Internal:ACDT3D_Scitech2018 | SciTech 2018]]<br />
<br />
<br />
[[Internal:Machine_Learning_CDT3D | Machine Learning and CDT3D]]<br />
<br />
[[Internal:AIAA_2018_CDT3D | CDT3D AIAA Aviation Paper ]]<br />
<br />
[[Internal:PODM4D_predicates | PODM4D fine grain concurrency benchmark ]]<br />
<br />
[[Internal:AIAA_Geometry_and_Mesh_Generation_Workshop | Geometry and Mesh Generation Workshop ]]<br />
<br />
<br />
[[Internal:CDT3D_RoadMap |CDT3D RoadMap]]<br />
<br />
[[Internal:Computational_Resources | High capacity Computational Resources ]]<br />
<br />
[[NDAM:Main | PODM 4D and Anisotropy ]]<br />
<br />
[https://docs.google.com/document/d/1jzTiohDA51VHXc65tfIFkmjrKM_8yQn7YYj2txEyhyU MPI PDR.PODM 16,000 cores data and other]<br />
<br />
== Kevin ==<br />
[[Internal:PDR_AFLR |PDR.AFLR Stability and Scalability]]<br />
<br />
[[Internal:PDR_AFLR_Progress_Report |PDR.AFLR Progress Report]]<br />
<br />
[[AFLR_Point_Spacing/Distribution_Function_Parameters | AFLR Point Spacing/Distribution Function Parameters]]<br />
<br />
[[Internal:PDR_AFLR_Performance_Testing |PDR.AFLR Performance Testing]]<br />
<br />
[[Internal:PDR_AFLR_Performance_AIAA |AIAA PDR.AFLR Performance Data]]<br />
<br />
[[Internal:Report_Status |Report Status]]<br />
<br />
[[Internal:PsC_AFLR_Performance |PsC.AFLR Performance Data]]<br />
<br />
[[Internal:Next_Generation_Boundary_Layer |Next Generation Boundary Layer]]<br />
<br />
[[Internal:CDT3D_GPU_Smoothing | CDT3D GPU Smoothing]]<br />
<br />
== Polykarpos ==<br />
[[PREMA Shared Memory]]<br />
<br />
[[Internal:IMR_2018_Runtime_Systems | IMR 2018 Runtime Systems]]<br />
<br />
[[Intel Xeon Phi and High Bandwidth Memory]]<br />
<br />
[[CNF:Machine_Learning | CNF Machine Learning Project]]<br />
<br />
[[CNF:ML_Data_Denoise | Track denoising with ML]]<br />
<br />
[https://docs.google.com/presentation/d/1rF3rElnBCsKR_Z_66nTCmMqSZa3b51mEzswiLz_3d3Q/edit?usp=sharing CNF HPC Workshop Slides]<br />
<br />
[[PREMA SW4| SW4 Benchmark with PREMA]]<br />
<br />
[[PDR.PODM Distributed Memory| PDR.PODM.PREMA]]<br />
<br />
[https://www.overleaf.com/read/vgwrypshrkqp PREMA-Argobots Paper]<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/g/personal/pthom001_odu_edu/EYiL4h5xKThJgjyvbsFbZe4BH_4pr3rR-9x-gvErpDaq3A?e=hdy7ii Heterogeneousarchitecturesforcomputationalintensiveapplications:Acost-effectivenessanalysis]<br />
<br />
Future Work<br />
<br />
[[Internal:TAPAS]]<br />
<br />
== Spiros ==<br />
'''Image-To-Mesh Conversion Projects'''<br />
<br />
[[Internal:Tessellate_2D | Tessellate 2D]]<br />
<br />
[[Internal:PODM_3D | PODM 3D]]<br />
<br />
[[Internal:PODM_4D | PODM 4D]]<br />
<br />
<br />
'''BioInformatics Projects'''<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
<br />
'''Contrained Mesh Refinement Projects'''<br />
<br />
[[Internal:CDT3D_GPU_Smoothing | CDT3D GPU Smoothing]]<br />
<br />
<br />
'''CNF Related Projects'''<br />
<br />
[[CNF:CNF_Data_Generation | CNF Data Generation]]<br />
<br />
[[CNF:Monte_Carlo_Event_Generator_using_a_Tessellation | Monte Carlo Event Generator using a Tessellation]]<br />
<br />
[[CNF:Smart_LookUp_Table_Utilities | Smart LookUp Table Utilities]]<br />
<br />
[[CNF:PartonsToNRRD | PartonsToNRRD]]<br />
<br />
[[CNF:Weighted_Mesh_To_Image | Weighted Mesh To Image]]<br />
<br />
[[Internal:TetGen_CNF | TetGen CNF]]<br />
<br />
<br />
<br />
'''Utilities Projects'''<br />
<br />
[[Internal:Synthetical_Image_Data_Generator | Synthetical Image Data Generator]]<br />
<br />
[[Internal:MeshIO | MeshIO]]<br />
<br />
<br />
'''Other Projects'''<br />
<br />
[[Internal:PDM_ND | PDM ND]]<br />
<br />
[[Internal:GPU_Usage | GPU Usage]]<br />
<br />
<br />
'''Reports'''<br />
<br />
[[CNF:CNF_Adaptivity | CNF Adaptivity]]<br />
<br />
== General ==<br />
===Conference Slides ===<br />
[https://odu.box.com/s/vev5nqudpd5yq143xfq99y7b8uyf0j4e AI in Nuclear Physics 2020]<br />
<br />
=== Proposal Resources ===<br />
[[Internal:NSF_PPOSS_Funded_Proposals]]<br />
<br />
===Notes ===<br />
[[Internal:CNF_Workshop]]<br />
<br />
[[Internal:CNF_HPC_Workshop_2019]]<br />
<br />
[[Internal:Kokkos_Tutorial_2020_02_19]]<br />
<br />
[[Internal:Intel_OneAPI_Tutorial_2020_02_20]]<br />
<br />
=== Papers ===<br />
[http://www.cs.odu.edu/crtc/internal/papers/O'Connell_et_al._-_2018_-_Application_of_the_Dependency_Inversion_Principle_.pdf T-infinity: The Dependency Inversion Principle for Rapid and Sustainable Multidisciplinary Software Development]<br />
<br />
[https://hal.inria.fr/hal-01927559 Meshless Voronoi on the GPU, Nicolas Ray,Dmitry Sokolov,Sylvain Lefebvre,Bruno Lévy]<br />
<br />
[http://pages.saclay.inria.fr/frederic.alauzet/TetrahedronVI/Slides_Tetrahedron_VI/Thursday/Mike_Park_ugawg-tet-talk.pdf Democratizing Mesh Adaptation forComputational Fluid Dynamics Michael Park]<br />
<br />
[[Internal:Hashing_Meshes]]<br />
<br />
=== Videos ===<br />
[https://www.youtube.com/watch?v=pn7IqhOEGpY Using Graphs for Unstructured Data (Keshav Pingali)]<br />
<br />
[https://www.youtube.com/watch?v=RZ09uZs1ynA Big Data Brain Maps at Argonne National Laboratory (Bobby Kasthuri), ATPESC 2017]<br />
<br />
=== Other ===<br />
http://livingreview.in.tum.de/welcome/<br />
<br />
=== CRTC lab Virtual meetings ===<br />
* 2020.01.18 - Christos<br />
:* Unified Approach for Speculative Operations, [https://docs.google.com/presentation/d/1haL6NVwZzRVG0W71KoWmqLKgxkf6to2sUsdTH5Xn4kw (presentation)], [https://olddominion-my.sharepoint.com/:f:/r/personal/ctsol001_odu_edu/Documents/Parallel_Operations?csf=1&web=1&e=hQ8rPg (shared directory)]<br />
<br />
* 2021.02.03 - Kevin & Spiros<br />
:* Introduction to GPU Parallelism, [https://drive.google.com/file/d/1vW3VJXi04JB2N7XpVoNnXXSNv4i1Pnet/view?usp=sharing (presentation)]<br />
<br />
* 2021.02.10 - Kevin & Spiros<br />
:* GPU Architecture, [https://drive.google.com/file/d/1LfDwsbdscSpTk3eBT74lfIlSyD-W_Zwh/view?usp=sharing (presentation)]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Student_Reports&diff=8451Internal:Student Reports2021-04-14T21:17:08Z<p>Spyridon97: /* Spiros */</p>
<hr />
<div>== Eleni, Thomas , Angelos==<br />
[[Internal:ReportCloud | Cloud Report]]<br />
<br />
[[Internal:Preprocess_Medical_Images | Preprocessing of Medical Images]]<br />
<br />
[[Internal:Execution_And_Evaluation_Medical_Images | Execution and Evaluation of Medical Imaging data]]<br />
<br />
== Angelos ==<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
[[Internal:APBNRRExecutionStatus | APBNRR Execution Status]]<br />
<br />
[[Internal:PAPBNRRComparison | Pristine vs New PAPBNRR]]<br />
<br />
[[Internal:APBNRRMachineLearning | APBNRR Machine Learning]]<br />
<br />
[[Internal: SlicerExtensions | Slicer Extensions]]<br />
<br />
== Joi ==<br />
[[Tasks | Tasks ]]<br />
<br />
[[I2M_Conversion_Tool | Image-to-Mesh (I2M) Conversion Tool]]<br />
<br />
[[3D_Slicer_CBC3D_Extension | 3D Slicer CBC3D Extension ]]<br />
<br />
== Christos ==<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/Papers All papers]'''<br />
<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/presentations All presentations]'''<br />
<br />
<hr><br />
[https://www.overleaf.com/6438559258mdmsnbpffztx speculative meshing operations using tasks paper - overleaf]<br />
<br />
[https://www.overleaf.com/3459466599cnnfkhzmkczs Adaptive CDT3D paper - overleaf]<br />
<br />
[https://www.overleaf.com/8242192311zntgrtsphkvv Parallel Grid Adaptation Paper (UGAWG) - overleaf ]<br />
<br />
<br />
<br />
[[Internal:Adaptive_Meshing_For_APBNRR]]<br />
<br />
[https://docs.google.com/presentation/d/1Fjt5PoDPXa715nOAjn0F4IJpdhscIGnlaQvweqFnB0U Adaptivity PPoSS slides]<br />
<br />
[https://docs.google.com/presentation/d/1RDtQufImuo5J1pKioIYDuTWCeX1UtjKyx7kQ2i24Tmo/edit?usp=sharing CNF HPC slides]<br />
<br />
<br />
[https://git-community.cs.odu.edu/nikos/SciTech_2019/blob/draft/main.pdf Scitech Journal paper latest draft]<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/cand_exam.pdf Candidacy exam (document)]<br />
[https://olddominion-my.sharepoint.com/:p:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy_presentation.pptx (presentation .pptx)]<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy%20_presentation.pdf (presentation pdf)]<br />
<br />
[[Internal:Adaptive_CDT3D_Results | Metric Adaptive CDT3D results]]<br />
<br />
[[Internal:Argobots_and_CDT3D | Argobots and CDT3D]]<br />
<br />
[[Internal:4D_Predicates_Benchmark | 4D predicates benchmark]]<br />
<br />
[[CNF:Mesh Sizing for CNF data]]<br />
<br />
<br />
<br />
<br />
[[Internal:ACDT3D | Metric Adaptive CDT3D Notes ]]<br />
<br />
<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/paper.pdf Parallel Mesh Generation review paper]<br />
<br />
<br />
[https://lbpw.larc.nasa.gov/ AIAA Sonic Boom Prediction Workshop (next in 2020)]<br />
<br />
[[Internal:CDT3D_SU2 | SU2 and grid requirements ]]<br />
<br />
[[Internal:Boundary_Layer_References | Boundary Layer References ]]<br />
<br />
[[External:Mesh_Generation_Articles_In_Journals | Some Mesh Generation Journal papers]]<br />
<br />
[https://docs.google.com/document/d/1txyCu6ibheH19hi3nZ10LgHm-WzglUrT5y0AaO__VYk/edit?usp=sharing PhD Tentative Shedule]<br />
<br />
<br />
[https://odu.box.com/s/sfkd5l3d9r5865qbsovx0eo6gwlaiask Poster IMR]<br />
<br />
[https://odu.box.com/s/1sok1edideblq4nctbufg79vud0oiu8f NIA presentation (September 2018) ] <br />
<br />
[[Internal:IMR_2018_Short_Course | IMR 2018 Short Course notes]]<br />
<br />
<br />
<br />
[[Internal:ACDT3D_Scitech2018 | SciTech 2018]]<br />
<br />
<br />
[[Internal:Machine_Learning_CDT3D | Machine Learning and CDT3D]]<br />
<br />
[[Internal:AIAA_2018_CDT3D | CDT3D AIAA Aviation Paper ]]<br />
<br />
[[Internal:PODM4D_predicates | PODM4D fine grain concurrency benchmark ]]<br />
<br />
[[Internal:AIAA_Geometry_and_Mesh_Generation_Workshop | Geometry and Mesh Generation Workshop ]]<br />
<br />
<br />
[[Internal:CDT3D_RoadMap |CDT3D RoadMap]]<br />
<br />
[[Internal:Computational_Resources | High capacity Computational Resources ]]<br />
<br />
[[NDAM:Main | PODM 4D and Anisotropy ]]<br />
<br />
[https://docs.google.com/document/d/1jzTiohDA51VHXc65tfIFkmjrKM_8yQn7YYj2txEyhyU MPI PDR.PODM 16,000 cores data and other]<br />
<br />
== Kevin ==<br />
[[Internal:PDR_AFLR |PDR.AFLR Stability and Scalability]]<br />
<br />
[[Internal:PDR_AFLR_Progress_Report |PDR.AFLR Progress Report]]<br />
<br />
[[AFLR_Point_Spacing/Distribution_Function_Parameters | AFLR Point Spacing/Distribution Function Parameters]]<br />
<br />
[[Internal:PDR_AFLR_Performance_Testing |PDR.AFLR Performance Testing]]<br />
<br />
[[Internal:PDR_AFLR_Performance_AIAA |AIAA PDR.AFLR Performance Data]]<br />
<br />
[[Internal:Report_Status |Report Status]]<br />
<br />
[[Internal:PsC_AFLR_Performance |PsC.AFLR Performance Data]]<br />
<br />
[[Internal:Next_Generation_Boundary_Layer |Next Generation Boundary Layer]]<br />
<br />
[[Internal:CDT3D | CDT3D]]<br />
<br />
== Polykarpos ==<br />
[[PREMA Shared Memory]]<br />
<br />
[[Internal:IMR_2018_Runtime_Systems | IMR 2018 Runtime Systems]]<br />
<br />
[[Intel Xeon Phi and High Bandwidth Memory]]<br />
<br />
[[CNF:Machine_Learning | CNF Machine Learning Project]]<br />
<br />
[[CNF:ML_Data_Denoise | Track denoising with ML]]<br />
<br />
[https://docs.google.com/presentation/d/1rF3rElnBCsKR_Z_66nTCmMqSZa3b51mEzswiLz_3d3Q/edit?usp=sharing CNF HPC Workshop Slides]<br />
<br />
[[PREMA SW4| SW4 Benchmark with PREMA]]<br />
<br />
[[PDR.PODM Distributed Memory| PDR.PODM.PREMA]]<br />
<br />
[https://www.overleaf.com/read/vgwrypshrkqp PREMA-Argobots Paper]<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/g/personal/pthom001_odu_edu/EYiL4h5xKThJgjyvbsFbZe4BH_4pr3rR-9x-gvErpDaq3A?e=hdy7ii Heterogeneousarchitecturesforcomputationalintensiveapplications:Acost-effectivenessanalysis]<br />
<br />
Future Work<br />
<br />
[[Internal:TAPAS]]<br />
<br />
== Spiros ==<br />
'''Image-To-Mesh Conversion Projects'''<br />
<br />
[[Internal:Tessellate_2D | Tessellate 2D]]<br />
<br />
[[Internal:PODM_3D | PODM 3D]]<br />
<br />
[[Internal:PODM_4D | PODM 4D]]<br />
<br />
<br />
'''BioInformatics Projects'''<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
<br />
'''Contrained Mesh Refinement Projects'''<br />
<br />
[[Internal:CDT3D_GPU_Smoothing | CDT3D GPU Smoothing]]<br />
<br />
<br />
'''CNF Related Projects'''<br />
<br />
[[CNF:CNF_Data_Generation | CNF Data Generation]]<br />
<br />
[[CNF:Monte_Carlo_Event_Generator_using_a_Tessellation | Monte Carlo Event Generator using a Tessellation]]<br />
<br />
[[CNF:Smart_LookUp_Table_Utilities | Smart LookUp Table Utilities]]<br />
<br />
[[CNF:PartonsToNRRD | PartonsToNRRD]]<br />
<br />
[[CNF:Weighted_Mesh_To_Image | Weighted Mesh To Image]]<br />
<br />
[[Internal:TetGen_CNF | TetGen CNF]]<br />
<br />
<br />
<br />
'''Utilities Projects'''<br />
<br />
[[Internal:Synthetical_Image_Data_Generator | Synthetical Image Data Generator]]<br />
<br />
[[Internal:MeshIO | MeshIO]]<br />
<br />
<br />
'''Other Projects'''<br />
<br />
[[Internal:PDM_ND | PDM ND]]<br />
<br />
[[Internal:GPU_Usage | GPU Usage]]<br />
<br />
<br />
'''Reports'''<br />
<br />
[[CNF:CNF_Adaptivity | CNF Adaptivity]]<br />
<br />
== General ==<br />
===Conference Slides ===<br />
[https://odu.box.com/s/vev5nqudpd5yq143xfq99y7b8uyf0j4e AI in Nuclear Physics 2020]<br />
<br />
=== Proposal Resources ===<br />
[[Internal:NSF_PPOSS_Funded_Proposals]]<br />
<br />
===Notes ===<br />
[[Internal:CNF_Workshop]]<br />
<br />
[[Internal:CNF_HPC_Workshop_2019]]<br />
<br />
[[Internal:Kokkos_Tutorial_2020_02_19]]<br />
<br />
[[Internal:Intel_OneAPI_Tutorial_2020_02_20]]<br />
<br />
=== Papers ===<br />
[http://www.cs.odu.edu/crtc/internal/papers/O'Connell_et_al._-_2018_-_Application_of_the_Dependency_Inversion_Principle_.pdf T-infinity: The Dependency Inversion Principle for Rapid and Sustainable Multidisciplinary Software Development]<br />
<br />
[https://hal.inria.fr/hal-01927559 Meshless Voronoi on the GPU, Nicolas Ray,Dmitry Sokolov,Sylvain Lefebvre,Bruno Lévy]<br />
<br />
[http://pages.saclay.inria.fr/frederic.alauzet/TetrahedronVI/Slides_Tetrahedron_VI/Thursday/Mike_Park_ugawg-tet-talk.pdf Democratizing Mesh Adaptation forComputational Fluid Dynamics Michael Park]<br />
<br />
[[Internal:Hashing_Meshes]]<br />
<br />
=== Videos ===<br />
[https://www.youtube.com/watch?v=pn7IqhOEGpY Using Graphs for Unstructured Data (Keshav Pingali)]<br />
<br />
[https://www.youtube.com/watch?v=RZ09uZs1ynA Big Data Brain Maps at Argonne National Laboratory (Bobby Kasthuri), ATPESC 2017]<br />
<br />
=== Other ===<br />
http://livingreview.in.tum.de/welcome/<br />
<br />
=== CRTC lab Virtual meetings ===<br />
* 2020.01.18 - Christos<br />
:* Unified Approach for Speculative Operations, [https://docs.google.com/presentation/d/1haL6NVwZzRVG0W71KoWmqLKgxkf6to2sUsdTH5Xn4kw (presentation)], [https://olddominion-my.sharepoint.com/:f:/r/personal/ctsol001_odu_edu/Documents/Parallel_Operations?csf=1&web=1&e=hQ8rPg (shared directory)]<br />
<br />
* 2021.02.03 - Kevin & Spiros<br />
:* Introduction to GPU Parallelism, [https://drive.google.com/file/d/1vW3VJXi04JB2N7XpVoNnXXSNv4i1Pnet/view?usp=sharing (presentation)]<br />
<br />
* 2021.02.10 - Kevin & Spiros<br />
:* GPU Architecture, [https://drive.google.com/file/d/1LfDwsbdscSpTk3eBT74lfIlSyD-W_Zwh/view?usp=sharing (presentation)]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:CDT3D&diff=8450Internal:CDT3D2021-04-14T21:15:58Z<p>Spyridon97: Spyridon97 moved page Internal:CDT3D to Internal:CDT3D GPU Smoothing</p>
<hr />
<div>#REDIRECT [[Internal:CDT3D GPU Smoothing]]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:CDT3D_GPU_Smoothing&diff=8449Internal:CDT3D GPU Smoothing2021-04-14T21:15:58Z<p>Spyridon97: Spyridon97 moved page Internal:CDT3D to Internal:CDT3D GPU Smoothing</p>
<hr />
<div>__TOC__<br />
<br />
=Presentations=<br />
==Spring 2021==<br />
<br />
# [https://docs.google.com/presentation/d/1haL6NVwZzRVG0W71KoWmqLKgxkf6to2sUsdTH5Xn4kw/edit A Unified Approach for Parallel Mesh Operations] by Christos✔<br />
# [https://docs.google.com/presentation/d/1LfDwsbdscSpTk3eBT74lfIlSyD-W_Zwh/edit GPU Architecture] ✔<br />
<br />
=Code=<br />
==Spring 2021==<br />
<br />
# Discrete Mesh Optimization using the paper: [https://project.inria.fr/imr27/files/2018/09/1003.pdf Discrete Mesh Optimization on GPU], The results below are using the tet_shape metric from the [http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=42FDC437179F12D08EB6C4F84FACD490?doi=10.1.1.483.5380&rep=rep1&type=pdf Verdict library]<br />
## CPU version:<br />
### Sequential:<br />
#### Dihedral Quality of Discrete Vertex smoothing with Global Iterations:✔<br> [[File:Discrete_Vertex_Smoothing_Dihedral_Angle_Quality_Global_Iterations.png|1000px|Discrete_Vertex_Smoothing_Dihedral_Angle_Quality_Global_Iterations.png]]<br />
#### Dihedral Quality of different smoothing methods:✔<br>[[File:Vertex_Smoothing_Methods_Dihedral_Angle_Quality.png|1000px|Vertex_Smoothing_Methods_Dihedral_Angle_Quality.png]]<br />
#### Shape Quality of different smoothing methods:✔<br>[[File:Shape_Quality.png|1000px|Shape_Quality.png]]<br />
#### Distribution of Smoothing Time Per Vertex:✔<br> [[File:Distribution_of_Smoothing_Time_Per_Vertex_CPU.png|1000px|Distribution_of_Smoothing_Time_Per_Vertex_CPU.png]]<br />
### Parallel:<br />
#### Dihedral Quality: Sequential vs Parallel:✔<br>[[File:Discrete_Vertex_Smoothing_Dihedral_Angle_Quality.png|1000px|Discrete_Vertex_Smoothing_Dihedral_Angle_Quality.png]]<br />
#### Speed-Up:✔<br>[[File:Smoothing_Speed-up.png|750px|Smoothing_Speed-up.png]]<br />
#### RollBacks✔:<br>[[File:Percentage_of_Rollbacks_per_Iteration.png|750px|Percentage_of_Rollbacks_per_Iteration.png]]<br />
## GPU version:<br />
### Sequential CPU-wise:<br />
#### Distribution of Smoothing Time Per Vertex:✔<br> [[File:Distribution_of_Smoothing_Time_Per_Vertex_GPU.png|1000px|Distribution_of_Smoothing_Time_Per_Vertex_GPU.png]]<br />
#### Version 1 (all points copied):<br />
##### Steps:<br />
###### Bounding box on GPU, grid(1) block(1)<br />
###### Original point value on GPU, grid(1) block(1)<br />
###### Grid creation on GPU, grid(1) block(1)<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
##### Time total = 18.563 sec (90 sec on CPU sequential)<br />
###### CPU = 0.79182 sec<br />
###### Steps 1 & 2 = 8.31743 sec<br />
###### Step 3 = 1.03768 sec<br />
###### Step 4 = 6.55488 sec (per point)<br />
###### Step 5 = 1.73327 sec<br />
###### MemCopyH2D = 0.12792 sec<br />
###### MemCopyD2H = 0.00002 sec<br />
#### Version 2.0 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
#######!!! Note: access global memory multiple times !!!<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 15.06656 sec<br />
###### CPU = 1.8857 sec (all cavities)<br />
###### Step 3 = 6.43994 sec<br />
###### Step 4 = 6.20552 sec<br />
###### MemCopyH2D = 0.38211 sec<br />
###### MemCopyD2H = 0.15329 sec<br />
#### Version 2.1 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 10.8227 sec<br />
###### CPU = 1.8857 sec<br />
###### Step 3 = 6.47006 sec<br />
###### Step 4 = 1.74147 sec<br />
###### MemCopyH2D = 0.37493 sec<br />
###### MemCopyD2H = 0.15225 sec<br />
#### Version 3 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(8+8) and reduction for the max value and id of a block on GPU, grid(8) block(64)<br />
###### Reduction using global values and ids arrays(8+8) on GPU, grid(1) block(8)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 9.17189 sec<br />
###### CPU = 2.04448 sec<br />
###### Step 3 = 6.40427 sec<br />
###### Step 4 = 0.19378 sec<br />
###### MemCopyH2D = 0.37604 sec<br />
###### MemCopyD2H = 0.15332 sec<br />
#### Version 3.1 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.77189 sec<br />
###### CPU = 2.04448 sec (Data Transformation = 0.194 sec)<br />
###### Step 3 = 6.0427 sec<br />
###### Step 4 = 0.19378 sec<br />
###### MemCopyH2D = 0.37604 sec<br />
###### MemCopyD2H = 0.15332 sec<br />
#### Version 3.2 (allocate what you need at the beginning & copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.34358 sec<br />
###### CPU = 1.58702 sec<br />
###### Step 3 = 6.02387s sec<br />
###### Step 4 = 0.20711ms sec<br />
###### MemCopyH2D = 0.37380 sec<br />
###### MemCopyD2H = 0.15178 sec<br />
#### Version 3.3 (allocate what you need at the beginning & copy what you need & and combine the GPU data):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.06576 sec<br />
###### CPU = 1.466553 sec<br />
###### Step 3 = 5.99690 sec<br />
###### Step 4 = 0.20702 sec<br />
###### MemCopyH2D = 0.25319 sec<br />
###### MemCopyD2H = 0.078917 sec<br />
#### Version 4 (perfom batching with 1 stream per cpu-thread):<br />
##### For a batch of 4 vertices:<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 7.18418 sec<br />
###### CPU = 0.946749 sec<br />
###### Step 3 = 5.94023 sec<br />
###### Step 4 = 0.20709 sec<br />
###### MemCopyH2D = 0.069697 sec<br />
###### MemCopyD2H = 0.020414 sec<br />
##### Batch size:✔ <br>[[File:BATCH-SIZE.v4.png|750px|BATCH-SIZE.v4.png]]<br />
#### Version 4.1 (perfom batching with #streams = batch-size within each cpu-thread):<br />
##### For a batch of 4 vertices<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
#### Version 4.2 (perfom batching with #streams = << batch-size within each cpu-thread):<br />
##### For a batch of 12 vertices with 4 streams<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time:<br />
###### lock time: 0.0190836 sec<br />
###### findCavityCells time: 0.178939 sec<br />
###### findUniqueVertices time: 0.215159 sec<br />
###### dataCollection time: 0.187204 sec<br />
###### boundingBox time: 0.0280833 sec<br />
###### grid time: 0.00122939 sec<br />
###### originalPointValue time: 0.12965 sec<br />
###### assignNewPoint time: 0.00165265 sec<br />
###### unlock time: 0.0117685 sec<br />
###### Total Cpu time: 0.77277 sec<br />
### Parallel CPU-wise:<br />
#### Version 4:<br />
##### Speed-Up:✔<br>[[File:Smoothing_Speed-up-heterogeneous.png|750px|Smoothing_Speed-up-heterogeneous.png]]<br />
##### Rollbacks-Time-Per-Thread:✔<br>[[File:parallel.v4.png|750px|parallel.v4.png]]<br />
##### Profiler: 4 cpu threads, 4 streams✔<br> [[File:profiler.v4.png|750px|profiler.v4.png]]<br />
#### Version 4.1:<br />
##### Profiler: 2 cpu threads, 2(copy)+8(computation) streams✔<br> [[File:profiler.v4.1.png|750px|profiler.v4.1.png]]<br />
#### Version 4.2:<br />
##### Rollbacks-Time-Per-Thread (#vertices = 28,217):✔<br>[[File:parallel.v4.2.png|750px|parallel.v4.2.png]]<br />
##### Rollbacks-Time-Per-Thread (#vertices = 79,809):✔<br>[[File:parallel.v4.2-double-size.png|750px|parallel.v4.2-double-size.png]]<br />
###### CPU (1 thread) 666 sec, CPU (32 threads) 28.29 sec<br />
##### Profiler: 2 cpu threads, 2(copy)+8(computation) streams✔<br> [[File:profiler.v4.2.png|750px|profiler.v4.2.png]]<br />
##### CPU Time Histogram 1 thread:✔<br>[[File:Histogram_of_CPU_Batch_Time_v4.2_threads=1.png|750px|Histogram_of_CPU_Batch_Time_v4.2_threads=1.png]]<br />
##### CPU Time Histogram 2 threads:✔<br>[[File:Histogram_of_CPU_Batch_Time_v4.2_threads=2.png|750px|Histogram_of_CPU_Batch_Time_v4.2_threads=2.png]]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF_Data_Generation&diff=8448CNF:CNF Data Generation2021-04-14T15:24:07Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/cnf-data-generation Gitlab]<br />
# [https://github.com/spyridon97/CNF-Data-Generation Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Set up a MySQL server on Wahab for partons ✔<br />
# Create a container with all the required dependencies ✔<br />
# Split Mitch's Code into:<br />
## A program that splits kinematic values for each core using:<br />
### Uniform Data ✔<br />
### Mesh Points ✔<br />
### Splitting way<br />
#### First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics. ✔<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. ✔<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## A program that computes the Partons Data per core using the kinematics values and the models:<br />
### Compton Form Factors ✔<br />
### Cross-Sections (TODO)<br />
## A program that merges the results of each core into one file ✔<br />
## A program that updates the weights of a mesh using partonsData ✔<br />
# Enable shared memory parallelization using CPU affinity, which means pinning a process to a specific core. ✔<br />
# Create a script that runs partons in parallel using shared memory ✔<br />
# Create a script that runs partons in parallel using distributed + shared memory ✔<br />
# Speedup:<br />
## [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
== Fall 2020 ==<br />
# Fix Axis ordering. ✔<br />
# Generate Images with spacing. ✔<br />
# Fix max connections problem with the same username on the database. ✔<br />
# Parallelize split and merge steps ✔<br />
# Update VTK to 9.0.1 inside the container. (TODO)<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔<br />
# Add Mitch's code for the computation of error and addition of extra points under the required threshold ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF_Data_Generation&diff=8447CNF:CNF Data Generation2021-04-14T15:21:17Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/cnf-data-generation Gitlab]<br />
# [https://github.com/spyridon97/CNF-Data-Generation Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Set up a MySQL server on Wahab for partons ✔<br />
# Create a container with all the required dependencies ✔<br />
# Split Mitch's Code into:<br />
## A program that splits kinematic values for each core using:<br />
### Uniform Data ✔<br />
### Mesh Points ✔<br />
### Splitting way<br />
#### First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics. ✔<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. ✔<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## A program that computes the Partons Data per core using the kinematics values and the models:<br />
### Compton Form Factors ✔<br />
### Cross-Sections (TODO)<br />
## A program that merges the results of each core into one file ✔<br />
## A program that updates the weights of a mesh using partonsData ✔<br />
# Enable shared memory parallelization using CPU affinity, which means pinning a process to a specific core. ✔<br />
# Create a script that runs partons in parallel using shared memory ✔<br />
# Create a script that runs partons in parallel using distributed + shared memory ✔<br />
# Speedup:<br />
## [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
== Fall 2020 ==<br />
# Fix Axis ordering. ✔<br />
# Generate Images with spacing. ✔<br />
# Fix max connections problem with the same username on the database. ✔<br />
# Parallelize split and merge steps ✔<br />
# Update VTK to 9.0.1 inside the container. (TODO)<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔<br />
# Add code for the computation of error and addition of extra points under the required threshold ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Weighted_Mesh_To_Image&diff=8446CNF:Weighted Mesh To Image2021-04-13T16:10:07Z<p>Spyridon97: /* Repository */</p>
<hr />
<div>__TOC__<br />
<br />
=Repository=<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/weightedmeshtoimage Gitlab]<br />
<br />
=Algorithms=<br />
==Summer 2020==<br />
Create a new image using the weights of:<br />
# k-nearest-neighbors points as values in a weighted-mean, were weight[i] = 1 / distance(pixel-point, neighbor-point[i]) ^ power.<br />
# cell points as values in a weighted-mean, were weight[i] = 1 / distance(pixel-point, cell-point[i]) ^ power.<br />
## Note: If the pixel point is not inside a cell, then the algorithm `1` will be used.<br />
# the intersection set of k-nearest-neighbors points and cell points as values in a weighted-mean, were weight[i] = 1 / distance(pixel-point, intersection-point[i]) ^ power.<br />
## Note 1: If the Intersection set is empty, then the algorithm `2` will be used.<br />
## Note 2: If the pixel point is not inside a cell, then the algorithm `1` will be used.<br />
# cell points as values in a weighted-mean, were weight[i] = interpolation-value[i] based on the location of each pixel point inside a cell. Interpolation-values are the result of the FindCell function.<br />
## Note: If the pixel point is not inside a cell, then the algorithm `1` will be used.<br />
<br />
=Comparison Metrics=<br />
==Summer 2020==<br />
# Let a and b be the sets of pixel values of input and output image, then the mean weighted pixel difference value is:<br />
## [[File:MeanWeightedPixelValueFormula.png|300px|File:MeanWeightedPixelValueFormula.png]]<br />
# Average Hausdorff Distance (itk filter)<br />
# Similarity Index (itk filter)<br />
<br />
=Comparison Examples=<br />
==Summer 2020==<br />
<br />
'''Left: Image, Middle: Algorithm Result. Right: Mesh''' <br />
<br />
=== KNN, number of points = 4, power = 1.5 ===<br />
<br />
<gallery mode="packed" heights=400px><br />
File:NT-image.png<br />
File:NT-a_1-n_4-p_1.5.png<br />
File:NT-mesh.png<br />
</gallery><br />
<br />
# Mean Weighted pixel value difference: 0.000395461<br />
# Average Hausdorff Distance: 0.030885<br />
# Similarity Index: 0.986463<br />
<br />
=== Cell Points, number of points = 4, power = 1.5 ===<br />
<gallery mode="packed" heights=400px><br />
File:NT-image.png<br />
NT-a_2-n_4-p_1.5.png<br />
File:NT-mesh.png<br />
</gallery><br />
<br />
# Mean Weighted pixel value difference: 0.0002258<br />
# Average Hausdorff Distance: 0.030885<br />
# Similarity Index: 0.986463<br />
<br />
=== Intersection of KNN and Cell points, number of points = 5, power = 1.5 ===<br />
<gallery mode="packed" heights=400px><br />
File:NT-image.png<br />
File:NT-a_3-n_5-p_1.5.png<br />
File:NT-mesh.png<br />
</gallery><br />
<br />
# Mean Weighted pixel value difference: 0.000273839<br />
# Average Hausdorff Distance: 0.030885<br />
# Similarity Index: 0.986463<br />
<br />
=== Cell points with interpolation weights, number of points = 4, power = 1.5 ===<br />
<gallery mode="packed" heights=400px><br />
File:NT-image.png<br />
File:NT-a_4-n_4-p_1.5.png<br />
File:NT-mesh.png<br />
</gallery><br />
<br />
# Mean Weighted pixel value difference: 9.17551e-05<br />
# Average Hausdorff Distance: 0.030885<br />
# Similarity Index: 0.986463<br />
<br />
===Best Algorithm===<br />
Evidently, the best algorithm both visually and by looking into the metrics is algorithm 4.<br />
<br />
=Examples of PODM with changed weights=<br />
==Summer 2020==<br />
<br />
<gallery mode="packed" heights=300px><br />
File:NT-image.png<br />
File:NT-mesh.png<br />
File:NT_140519,d=5,wl=0.07,me=1.png<br />
</gallery><br />
<br />
===Modifying Area based on X coordinate===<br />
'''Algorithm 4 will be used'''<br />
<br />
<syntaxhighlight lang="c++" line='line'><br />
auto maxWeight = *std::max_element(pointWeights.begin(), pointWeights.end());<br />
for (size_t i = 0; i < numberOfPoints; ++i) {<br />
if (points[i][0] >= 15 && points[i][0] <= 20)<br />
pointWeights[i] = maxWeight + pointWeights[i];<br />
}<br />
</syntaxhighlight><br />
<br />
<gallery mode="packed" heights=400px><br />
File:NT-x_15_20_image.png<br />
File:NT-x_15_20_mesh.png<br />
</gallery><br />
<br />
===Modifying Area based on Y coordinate===<br />
'''Algorithm 4 will be used'''<br />
<br />
<syntaxhighlight lang="c++" line='line'><br />
auto maxWeight = *std::max_element(pointWeights.begin(), pointWeights.end());<br />
for (size_t i = 0; i < numberOfPoints; ++i) {<br />
if (points[i][1] >= 95 && points[i][1] <= 101)<br />
pointWeights[i] = maxWeight + pointWeights[i];<br />
}<br />
</syntaxhighlight><br />
<br />
<gallery mode="packed" heights=450px><br />
File:NT-y_95_101_image.png<br />
File:NT-y_95_101_mesh.png<br />
</gallery><br />
<br />
===Modifying Area based on Z coordinate===<br />
'''Algorithm 4 will be used'''<br />
<br />
<syntaxhighlight lang="c++" line='line'><br />
auto maxWeight = *std::max_element(pointWeights.begin(), pointWeights.end());<br />
for (size_t i = 0; i < numberOfPoints; ++i) {<br />
if (points[i][2] >= 43 && points[i][2] <= 53)<br />
pointWeights[i] = maxWeight + pointWeights[i];<br />
}<br />
</syntaxhighlight><br />
<br />
<gallery mode="packed" heights=400px><br />
File:NT-z_43_53_image.png<br />
File:NT-z_43_53_mesh.png<br />
</gallery></div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Smart_LookUp_Table_Utilities&diff=8445CNF:Smart LookUp Table Utilities2021-04-13T16:09:21Z<p>Spyridon97: /* Fall 2020 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/smartlookuptableutilities Gitlab]<br />
# [https://github.com/spyridon97/SmartLookUpTableUtilities Github]<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Find Cell that includes a point (2D/3D)✔<br />
# Get points of a cell (2D/3D/4D) ✔<br />
# Get points weights of a cell (2D/3D/4D) ✔<br />
# Get the volume of a cell (2D/3D/4D) ✔<br />
# Get centroid coordinates of a cell (2D/3D/4D) ✔<br />
# Get the interpolated weight of a point inside a cell (2D/3D/4D) ✔<br />
<br />
== Fall 2020 ==<br />
# Get neighbors of a cell (3D) ✔<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Smart_LookUp_Table_Utilities&diff=8444CNF:Smart LookUp Table Utilities2021-04-13T16:08:46Z<p>Spyridon97: /* Repository */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/smartlookuptableutilities Gitlab]<br />
# [https://github.com/spyridon97/SmartLookUpTableUtilities Github]<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Find Cell that includes a point (2D/3D)✔<br />
# Get points of a cell (2D/3D/4D) ✔<br />
# Get points weights of a cell (2D/3D/4D) ✔<br />
# Get the volume of a cell (2D/3D/4D) ✔<br />
# Get centroid coordinates of a cell (2D/3D/4D) ✔<br />
# Get the interpolated weight of a point inside a cell (2D/3D/4D) ✔<br />
<br />
== Fall 2020 ==<br />
# Get neighbors of a cell (3D) ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF_Data_Generation&diff=8443CNF:CNF Data Generation2021-04-13T16:08:10Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/cnf-data-generation Gitlab]<br />
# [https://github.com/spyridon97/CNF-Data-Generation Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Set up a MySQL server on Wahab for partons ✔<br />
# Create a container with all the required dependencies ✔<br />
# Split Mitch's Code into:<br />
## A program that splits kinematic values for each core using:<br />
### Uniform Data ✔<br />
### Mesh Points ✔<br />
### Splitting way<br />
#### First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics. ✔<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. ✔<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## A program that computes the Partons Data per core using the kinematics values and the models:<br />
### Compton Form Factors ✔<br />
### Cross-Sections (TODO)<br />
## A program that merges the results of each core into one file ✔<br />
## A program that updates the weights of a mesh using partonsData ✔<br />
# Enable shared memory parallelization using CPU affinity, which means pinning a process to a specific core. ✔<br />
# Create a script that runs partons in parallel using shared memory ✔<br />
# Create a script that runs partons in parallel using distributed + shared memory ✔<br />
# Speedup:<br />
## [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
== Fall 2020 ==<br />
# Fix Axis ordering. ✔<br />
# Generate Images with spacing. ✔<br />
# Fix max connections problem with the same username on the database. ✔<br />
# Parallelize split and merge steps ✔<br />
# Update VTK to 9.0.1 inside the container. (TODO)<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Monte_Carlo_Event_Generator_using_a_Tessellation&diff=8442CNF:Monte Carlo Event Generator using a Tessellation2021-04-13T16:07:53Z<p>Spyridon97: /* Fall 2020 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/mcegut Gitlab]<br />
# [https://github.com/spyridon97/MCEGUT Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Organize Gagik's code ✔<br />
# Add command-line interface ✔<br />
# Template code for 2-3 dimensions using: <br />
## [https://math.stackexchange.com/questions/18686/uniform-random-point-in-triangle Generate Random Points in a triangle] ✔<br />
## [http://vcg.isti.cnr.it/jgt/tetra.htm Generate Random Points in a tetrahedron] ✔<br />
# Replace linear search (O(n)) of tetrahedrons with given weight threshold with binary search (O(log(n))) ✔<br />
# Parallelize code using OpenMP ✔<br />
== Fall 2020 ==<br />
# Use Lebesque integral ✔<br />
# Incorporate Smart Look-up table utilities ✔<br />
# Extend Monte Carlo Event Generation to work with negative values ✔<br />
# Generate randomly uniform points inside a cell using: ✔<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
== Spring 2021 ==<br />
# Use meshes with double precision weights array ✔<br />
<br />
= References =<br />
[https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:Monte_Carlo_Event_Generator_using_a_Tessellation&diff=8441CNF:Monte Carlo Event Generator using a Tessellation2021-04-13T16:07:12Z<p>Spyridon97: /* Repository */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/mcegut Gitlab]<br />
# [https://github.com/spyridon97/MCEGUT Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Organize Gagik's code ✔<br />
# Add command-line interface ✔<br />
# Template code for 2-3 dimensions using: <br />
## [https://math.stackexchange.com/questions/18686/uniform-random-point-in-triangle Generate Random Points in a triangle] ✔<br />
## [http://vcg.isti.cnr.it/jgt/tetra.htm Generate Random Points in a tetrahedron] ✔<br />
# Replace linear search (O(n)) of tetrahedrons with given weight threshold with binary search (O(log(n))) ✔<br />
# Parallelize code using OpenMP ✔<br />
== Fall 2020 ==<br />
# Use Lebesque integral ✔<br />
# Incorporate Smart Look-up table utilities ✔<br />
# Extend Monte Carlo Event Generation to work with negative values ✔<br />
# Generate randomly uniform points inside a cell using: ✔<br />
## [https://www.researchgate.net/publication/275348534_Picking_a_Uniformly_Random_Point_from_an_Arbitrary_Simplex Picking a Uniformly Random Point from an Arbitrary Simplex]<br />
<br />
= References =<br />
[https://github.com/gavalian/clas12ai/blob/add-crtc-ml/cnf/main.cc Gagik's MC kernel]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF_Data_Generation&diff=8440CNF:CNF Data Generation2021-04-13T16:06:25Z<p>Spyridon97: /* Repository */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/cnf-data-generation Gitlab]<br />
# [https://github.com/spyridon97/CNF-Data-Generation Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Set up a MySQL server on Wahab for partons ✔<br />
# Create a container with all the required dependencies ✔<br />
# Split Mitch's Code into:<br />
## A program that splits kinematic values for each core using:<br />
### Uniform Data ✔<br />
### Mesh Points ✔<br />
### Splitting way<br />
#### First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics. ✔<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. ✔<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## A program that computes the Partons Data per core using the kinematics values and the models:<br />
### Compton Form Factors ✔<br />
### Cross-Sections (TODO)<br />
## A program that merges the results of each core into one file ✔<br />
## A program that updates the weights of a mesh using partonsData ✔<br />
# Enable shared memory parallelization using CPU affinity, which means pinning a process to a specific core. ✔<br />
# Create a script that runs partons in parallel using shared memory ✔<br />
# Create a script that runs partons in parallel using distributed + shared memory ✔<br />
# Speedup:<br />
## [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
== Fall 2020 ==<br />
# Fix Axis ordering. ✔<br />
# Generate Images with spacing. ✔<br />
# Fix max connections problem with the same username on the database. ✔<br />
# Parallelize split and merge steps ✔<br />
# Update VTK to 9.0.1 inside the container. (TODO)<br />
<br />
== Spring 2021 ==<br />
# Accept double instead of float type weights array of meshes ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:CNF_Data_Generation&diff=8439CNF:CNF Data Generation2021-04-13T16:05:28Z<p>Spyridon97: /* Fall 2020 */</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://github.com/spyridon97/CNF-Data-Generation Github]<br />
<br />
= Documentation =<br />
== Summer 2020 ==<br />
# Add README.md file ✔<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Set up a MySQL server on Wahab for partons ✔<br />
# Create a container with all the required dependencies ✔<br />
# Split Mitch's Code into:<br />
## A program that splits kinematic values for each core using:<br />
### Uniform Data ✔<br />
### Mesh Points ✔<br />
### Splitting way<br />
#### First implementation: split the kinematics dataset like. This way we create '''areas''' of kinematics. ✔<br />
##### N0_0... N0_m-1, <br />
##### N1_0... N1_m-1, <br />
##### N2_0... N2_m-1, <br />
##### ..., <br />
##### Nn-1_0... Nn-1_m-1. <br />
#### Second implementation: split the kinematics dataset by creating chunks of size 1 until each kinematics dataset for each node is filled. This way all nodes will have points of similar kinematics. ✔<br />
##### N0_0, N1_0, N2_0 .. Nn-1_0, <br />
##### N0_1, N1_1, N2_1 .. Nn-1_1, <br />
##### N0_2, N1_2, N2_2 .. Nn-1_2, <br />
##### ..., <br />
##### N0_m-1, N1_m-1, N2_m-1 .. Nn-1_m-1<br />
## A program that computes the Partons Data per core using the kinematics values and the models:<br />
### Compton Form Factors ✔<br />
### Cross-Sections (TODO)<br />
## A program that merges the results of each core into one file ✔<br />
## A program that updates the weights of a mesh using partonsData ✔<br />
# Enable shared memory parallelization using CPU affinity, which means pinning a process to a specific core. ✔<br />
# Create a script that runs partons in parallel using shared memory ✔<br />
# Create a script that runs partons in parallel using distributed + shared memory ✔<br />
# Speedup:<br />
## [[File:partons-speedup.png|500px|File:partons-speedup.png]]<br />
<br />
== Fall 2020 ==<br />
# Fix Axis ordering. ✔<br />
# Generate Images with spacing. ✔<br />
# Fix max connections problem with the same username on the database. ✔<br />
# Parallelize split and merge steps ✔<br />
# Update VTK to 9.0.1 inside the container. (TODO)<br />
<br />
== Spring 2021 ==<br />
# Accept double instead of float type weights array of meshes ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Tessellate_2D&diff=8438Internal:Tessellate 2D2021-04-13T16:04:41Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Spring 2020==<br />
# Write a proper README.md file✔<br />
# Create a [https://crtc.cs.odu.edu/I2M_Suite User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Spring 2020==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
=Code=<br />
==Spring 2020==<br />
# Restructure repository of Tessellate 2D the same way as PODM3D<br />
# Add binary file support<br />
# Improve Text output<br />
<br />
==Summer 2020==<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Spring 2021==<br />
# Accept double instead of float type precision images ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:PODM_3D&diff=8437Internal:PODM 3D2021-04-13T16:04:33Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Fall 2019==<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_56.pdf PODM3D] paper✔<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_58.pdf Guaranteed Quality Tetrahedral Delaunay Meshing for Medical Images] paper (Sect. 9) (TODO)<br />
# Write a proper README.md file✔<br />
# Add the [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] into the Appendix of the November Report✔<br />
# Create [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity CNF Adaptivity] Report<br />
<br />
==Spring 2020==<br />
# Create a [https://crtc.cs.odu.edu/CRTC_I2M User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Fall 2019==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
==Spring 2020==<br />
# Create meshes for the Medical Image Computing part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Medical_Imaging_Example_Meshes Medical Imaging Example Meshes] ✔<br />
<br />
==Summer 2020==<br />
# Create meshes for the Bioinformatics part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Bioinformatics_Example_Meshes Bioinformatics Example Meshes] ✔<br />
<br />
=Code=<br />
==Fall 2019==<br />
# Restructure repository of PODM3D based on Software engineering standards✔<br />
# Decouple Args from PODM✔<br />
# Create setArguments function✔<br />
# Create setInputImage(ImageType::Pointer)✔<br />
# Remove IOs from PODM✔<br />
# Implement a sizing function that will work as an extra rule and it will use the points of the tetrahedron that is tested✔<br />
## Create UML diagrams to understand the code and the Meshing Procedure in general✔<br />
## Create a list of functions that need to be modified to implement the extra Rule ✔<br />
## Modify the created list ([https://docs.google.com/spreadsheets/d/13M6qLRtS3wMH5B8vXwcqA-gP2b9p8QGwbhrn68VXoj0/edit#gid=0 Extra Rule sizing Function list]) of functions✔<br />
## Create the following test sizing function to understand if modifications have been done correctly✔ <syntaxhighlight lang="c++" line='line'><br />
bool checkSizingFunction(double coordinatesOfTetrahedron**)<br />
{<br />
double volume = volume(coordinatesOfTetrahedron);<br />
double* baryCenter = baryCenter(coordinatesOfTetrahedron)<br />
<br />
// if x axis <= 55 and volume of tetrahedron > 10<br />
if (baryCenter[0] <= 55 && volume > 10)<br />
return true;<br />
else<br />
return false<br />
}<br />
</syntaxhighlight><br />
## Create the infrastructure to support Sizing Function pointer✔<br />
## Incorporate Function Pointer to PODM by creating a setSizingFunction()✔<br />
## Replace the test sizing function with the Adaptivity function that was created by Christos (The algorithm is described [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity here])✔<br />
### Understand that the current implementation supports negative weights✔<br />
### Change the current implementation to use the minimum-maximum edge length of the element instead of its volume✔<br />
### Identify an issue of checking sample points outside the image and solve it according to Christos's suggestions✔<br />
# Separate the interface, the IOs and the filters from PODM✔<br />
<br />
==Spring 2020==<br />
# Create a general PODM driver✔<br />
# Set a warning if delta > min-spacing * min-dimension-size / 5✔<br />
# Set a default delta: ✔<br />
## if sizing-function is off then default delta = min-spacing * min-dimension-size / 100 ✔<br />
## if sizing-function is on then default delta = min-spacing * min-dimension-size / 20 ✔<br />
# Remove labels array if there is only 1 label of interest ✔<br />
# Print a warning if a label is not recovered✔<br />
# Add Numa flag in CmakeLists.txt ✔<br />
# Add binary file support ✔<br />
# Improve Text output ✔<br />
# Create label-filter for images ✔<br />
## Preserve a set of labels of an image ✔<br />
## Exclude a set of labels of an image ✔<br />
# Remove redundant volume threshold checks that don't contribute anything to the meshing procedure ✔<br />
<br />
==Summer 2020==<br />
# Libraries:<br />
## Update ITK to v5.1.0 ✔<br />
## Update VTK to v9.0.1 ✔<br />
## Remove Boost from PODM (preserve it only for CGAL) ✔<br />
# Save the label of a cell after its creation and Remove the expensive label recalculation in GetCleanMesh/GetCleanMeshParallel✔<br />
# Compute Mesh Results in Parallel (TODO)<br />
## Implement Parallel version of GetCleanMesh function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:mesh-results-speed-up.png|500px|File:mesh-results-speed-up.png]] ✔<br />
## Assigned as default ✔<br />
# Upgrade the define NO_IO_WRITING<br />
## Rename it to IO_WRITING ✔<br />
## Create functions for loading and writing of auxiliary files ✔<br />
## Create a compile flag for it, which will also enable the creation of the auxiliary files folder ✔<br />
## Fix truncation error when loading box ✔<br />
# Compute Box, Resized image and surface in Parallel ✔<br />
## Implement Parallel version of ComputeBoxAndSurface function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:box-surface-speed-up.png|500px|File:box-surface-speed-up.png]]✔<br />
## Assigned as default ✔<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Fall 2020==<br />
# Fix shift of origin if delta not multiple of the spacing ✔<br />
# Handle CNF images with spacing different than 1, and origin different than {0, 0, 0} ✔<br />
## Upgrade Image Segmentation ✔<br />
## Upgrade Sizing function ✔<br />
## Upgrade Linear Interpolation ✔<br />
# Change ImageSegmentation to create an image inside the requested grid domain ✔<br />
<br />
==Spring 2021==<br />
# [Christos] Addition of Extra points at the beginning of meshing ✔<br />
# [Polykarpos] Loading and saving the state of PODM mesher ✔<br />
# Load Dynamically Sizing Functions and their initialization as a shared Library using function pointers ✔<br />
# Detect special cases of Christos and Polykarpos additions ✔<br />
# Accept double instead of float type precision images ✔<br />
# Fix PODM not working on Wahab (TODO)<br />
# Remeshing by using a mesh instead of an image as input and background mesh (TODO)<br />
# Create upscale-image-filter (TODO)<br />
# Replace CGAL Predicates with: (TODO)<br />
## Predicates Construction Kit:<br />
### [https://hal.inria.fr/hal-01225202 Paper]<br />
### [https://gforge.inria.fr/frs/?group_id=5833 Code]<br />
### Benchmark: [https://crtc.cs.odu.edu/Internal:4D_Predicates_Benchmark PCK vs CGAL Predicates Benchmark]<br />
### Note: see Geogram branch<br />
## Adaptive Precision Floating-Point Arithmetic and Fast Robust Predicates for Computational Geometry:<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-predicates.ps Paper]<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/code/predicates.c Code]<br />
# Smoothing (TODO)<br />
## [https://hal.inria.fr/inria-00413248/document Feature preserving Delaunay mesh generation from 3D multi-material images]<br />
## CGAL Sources: <br />
### https://doc.cgal.org/latest/Mesh_3/index.html<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3protectionimage3D<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3HybridDomainExample<br />
## PODM3D citations:<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999101968160 Volume conserving smoothing for piecewise linear curves, surfaces, and triple lines]<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999108004981 An anisotropic scale-invariant unstructured mesh generator suitable for volumetric imaging data]<br />
### [https://pubmed.ncbi.nlm.nih.gov/20161102/ Variational generation of prismatic boundary-layer meshes for biomedical computing, International Journal for Numerical Methods]<br />
## Prof, Andrey Suggestions:<br />
### [https://people.eecs.berkeley.edu/~jrs/papers/stuffing.pdf Isosurface Stuffing: Fast Tetrahedral Meshes with Good Dihedral Angles]<br />
### [https://imr.sandia.gov/_assets/documents/2019_IMR_Papers/3B.2-Chernikov.pdf CERTIFIED FUNCTIONS FOR MESH GENERATION]<br />
### [https://crtc.cs.odu.edu/pub/papers/journal_49.pdf GENERALIZED INSERTION REGION GUIDES FOR DELAUNAY MESH REFINEMENT]<br />
# Μultiple δ (TODO)<br />
## Implement multiple δ (TODO)<br />
## Use different δ values (possibly uniform based) according to the surface of each label of a segmented image (TODO)<br />
# Produce the Voronoi Mesh out of the Delaunay (TODO)<br />
## Check what TetGen does (TODO)<br />
## Research conversion methods (TODO)</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Tessellate_2D&diff=8436Internal:Tessellate 2D2021-04-13T16:03:59Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Spring 2020==<br />
# Write a proper README.md file✔<br />
# Create a [https://crtc.cs.odu.edu/I2M_Suite User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Spring 2020==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
=Code=<br />
==Spring 2020==<br />
# Restructure repository of Tessellate 2D the same way as PODM3D<br />
# Add binary file support<br />
# Improve Text output<br />
<br />
==Summer 2020==<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Spring 2021==<br />
# Use double instead of float type precision images</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Tessellate_2D&diff=8435Internal:Tessellate 2D2021-04-13T16:03:40Z<p>Spyridon97: /* Summer 2020 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Spring 2020==<br />
# Write a proper README.md file✔<br />
# Create a [https://crtc.cs.odu.edu/I2M_Suite User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Spring 2020==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
=Code=<br />
==Spring 2020==<br />
# Restructure repository of Tessellate 2D the same way as PODM3D<br />
# Add binary file support<br />
# Improve Text output<br />
<br />
==Summer 2020==<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Spring 2021==<br />
Use double instead of float type precision images</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=CNF:PartonsToNRRD&diff=8434CNF:PartonsToNRRD2021-04-13T16:02:51Z<p>Spyridon97: Created page with "__TOC__ = Repository = # [https://git-community.cs.odu.edu/crtc/nuclear-physics/partonstonrrd/ Gitlab] # [https://github.com/spyridon97/PartonsToNRRD/ Github] = Code = == Su..."</p>
<hr />
<div>__TOC__<br />
<br />
= Repository =<br />
# [https://git-community.cs.odu.edu/crtc/nuclear-physics/partonstonrrd/ Gitlab]<br />
# [https://github.com/spyridon97/PartonsToNRRD/ Github]<br />
<br />
= Code =<br />
== Summer 2020 ==<br />
# Create tool to generate NRRD images using partons data ✔<br />
<br />
== Fall 2020 ==<br />
# Enable the creation of images using specific spacing ✔<br />
<br />
== Spring 2021 ==<br />
# Create double instead of float type images ✔</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:Student_Reports&diff=8433Internal:Student Reports2021-04-13T15:57:46Z<p>Spyridon97: /* Spiros */</p>
<hr />
<div>== Eleni, Thomas , Angelos==<br />
[[Internal:ReportCloud | Cloud Report]]<br />
<br />
[[Internal:Preprocess_Medical_Images | Preprocessing of Medical Images]]<br />
<br />
[[Internal:Execution_And_Evaluation_Medical_Images | Execution and Evaluation of Medical Imaging data]]<br />
<br />
== Angelos ==<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
[[Internal:APBNRRExecutionStatus | APBNRR Execution Status]]<br />
<br />
[[Internal:PAPBNRRComparison | Pristine vs New PAPBNRR]]<br />
<br />
[[Internal:APBNRRMachineLearning | APBNRR Machine Learning]]<br />
<br />
[[Internal: SlicerExtensions | Slicer Extensions]]<br />
<br />
== Joi ==<br />
[[Tasks | Tasks ]]<br />
<br />
[[I2M_Conversion_Tool | Image-to-Mesh (I2M) Conversion Tool]]<br />
<br />
[[3D_Slicer_CBC3D_Extension | 3D Slicer CBC3D Extension ]]<br />
<br />
== Christos ==<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/Papers All papers]'''<br />
<br />
'''[https://olddominion-my.sharepoint.com/:f:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/presentations All presentations]'''<br />
<br />
<hr><br />
[https://www.overleaf.com/6438559258mdmsnbpffztx speculative meshing operations using tasks paper - overleaf]<br />
<br />
[https://www.overleaf.com/3459466599cnnfkhzmkczs Adaptive CDT3D paper - overleaf]<br />
<br />
[https://www.overleaf.com/8242192311zntgrtsphkvv Parallel Grid Adaptation Paper (UGAWG) - overleaf ]<br />
<br />
<br />
<br />
[[Internal:Adaptive_Meshing_For_APBNRR]]<br />
<br />
[https://docs.google.com/presentation/d/1Fjt5PoDPXa715nOAjn0F4IJpdhscIGnlaQvweqFnB0U Adaptivity PPoSS slides]<br />
<br />
[https://docs.google.com/presentation/d/1RDtQufImuo5J1pKioIYDuTWCeX1UtjKyx7kQ2i24Tmo/edit?usp=sharing CNF HPC slides]<br />
<br />
<br />
[https://git-community.cs.odu.edu/nikos/SciTech_2019/blob/draft/main.pdf Scitech Journal paper latest draft]<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/cand_exam.pdf Candidacy exam (document)]<br />
[https://olddominion-my.sharepoint.com/:p:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy_presentation.pptx (presentation .pptx)]<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/PhD_Candidacy%20_presentation.pdf (presentation pdf)]<br />
<br />
[[Internal:Adaptive_CDT3D_Results | Metric Adaptive CDT3D results]]<br />
<br />
[[Internal:Argobots_and_CDT3D | Argobots and CDT3D]]<br />
<br />
[[Internal:4D_Predicates_Benchmark | 4D predicates benchmark]]<br />
<br />
[[CNF:Mesh Sizing for CNF data]]<br />
<br />
<br />
<br />
<br />
[[Internal:ACDT3D | Metric Adaptive CDT3D Notes ]]<br />
<br />
<br />
<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/frombox/CRTC_Students/ChristosT/candidacy_exam/paper.pdf Parallel Mesh Generation review paper]<br />
<br />
<br />
[https://lbpw.larc.nasa.gov/ AIAA Sonic Boom Prediction Workshop (next in 2020)]<br />
<br />
[[Internal:CDT3D_SU2 | SU2 and grid requirements ]]<br />
<br />
[[Internal:Boundary_Layer_References | Boundary Layer References ]]<br />
<br />
[[External:Mesh_Generation_Articles_In_Journals | Some Mesh Generation Journal papers]]<br />
<br />
[https://docs.google.com/document/d/1txyCu6ibheH19hi3nZ10LgHm-WzglUrT5y0AaO__VYk/edit?usp=sharing PhD Tentative Shedule]<br />
<br />
<br />
[https://odu.box.com/s/sfkd5l3d9r5865qbsovx0eo6gwlaiask Poster IMR]<br />
<br />
[https://odu.box.com/s/1sok1edideblq4nctbufg79vud0oiu8f NIA presentation (September 2018) ] <br />
<br />
[[Internal:IMR_2018_Short_Course | IMR 2018 Short Course notes]]<br />
<br />
<br />
<br />
[[Internal:ACDT3D_Scitech2018 | SciTech 2018]]<br />
<br />
<br />
[[Internal:Machine_Learning_CDT3D | Machine Learning and CDT3D]]<br />
<br />
[[Internal:AIAA_2018_CDT3D | CDT3D AIAA Aviation Paper ]]<br />
<br />
[[Internal:PODM4D_predicates | PODM4D fine grain concurrency benchmark ]]<br />
<br />
[[Internal:AIAA_Geometry_and_Mesh_Generation_Workshop | Geometry and Mesh Generation Workshop ]]<br />
<br />
<br />
[[Internal:CDT3D_RoadMap |CDT3D RoadMap]]<br />
<br />
[[Internal:Computational_Resources | High capacity Computational Resources ]]<br />
<br />
[[NDAM:Main | PODM 4D and Anisotropy ]]<br />
<br />
[https://docs.google.com/document/d/1jzTiohDA51VHXc65tfIFkmjrKM_8yQn7YYj2txEyhyU MPI PDR.PODM 16,000 cores data and other]<br />
<br />
== Kevin ==<br />
[[Internal:PDR_AFLR |PDR.AFLR Stability and Scalability]]<br />
<br />
[[Internal:PDR_AFLR_Progress_Report |PDR.AFLR Progress Report]]<br />
<br />
[[AFLR_Point_Spacing/Distribution_Function_Parameters | AFLR Point Spacing/Distribution Function Parameters]]<br />
<br />
[[Internal:PDR_AFLR_Performance_Testing |PDR.AFLR Performance Testing]]<br />
<br />
[[Internal:PDR_AFLR_Performance_AIAA |AIAA PDR.AFLR Performance Data]]<br />
<br />
[[Internal:Report_Status |Report Status]]<br />
<br />
[[Internal:PsC_AFLR_Performance |PsC.AFLR Performance Data]]<br />
<br />
[[Internal:Next_Generation_Boundary_Layer |Next Generation Boundary Layer]]<br />
<br />
[[Internal:CDT3D | CDT3D]]<br />
<br />
== Polykarpos ==<br />
[[PREMA Shared Memory]]<br />
<br />
[[Internal:IMR_2018_Runtime_Systems | IMR 2018 Runtime Systems]]<br />
<br />
[[Intel Xeon Phi and High Bandwidth Memory]]<br />
<br />
[[CNF:Machine_Learning | CNF Machine Learning Project]]<br />
<br />
[[CNF:ML_Data_Denoise | Track denoising with ML]]<br />
<br />
[https://docs.google.com/presentation/d/1rF3rElnBCsKR_Z_66nTCmMqSZa3b51mEzswiLz_3d3Q/edit?usp=sharing CNF HPC Workshop Slides]<br />
<br />
[[PREMA SW4| SW4 Benchmark with PREMA]]<br />
<br />
[[PDR.PODM Distributed Memory| PDR.PODM.PREMA]]<br />
<br />
[https://www.overleaf.com/read/vgwrypshrkqp PREMA-Argobots Paper]<br />
<br />
[https://olddominion-my.sharepoint.com/:b:/g/personal/pthom001_odu_edu/EYiL4h5xKThJgjyvbsFbZe4BH_4pr3rR-9x-gvErpDaq3A?e=hdy7ii Heterogeneousarchitecturesforcomputationalintensiveapplications:Acost-effectivenessanalysis]<br />
<br />
Future Work<br />
<br />
[[Internal:TAPAS]]<br />
<br />
== Spiros ==<br />
'''Image-To-Mesh Conversion Projects'''<br />
<br />
[[Internal:Tessellate_2D | Tessellate 2D]]<br />
<br />
[[Internal:PODM_3D | PODM 3D]]<br />
<br />
[[Internal:PODM_4D | PODM 4D]]<br />
<br />
<br />
'''BioInformatics Projects'''<br />
<br />
[[Internal:GPCD | GPCD]]<br />
<br />
<br />
'''Contrained Mesh Refinement Projects'''<br />
<br />
[[Internal:CDT3D | CDT3D]]<br />
<br />
<br />
'''CNF Related Projects'''<br />
<br />
[[CNF:CNF_Data_Generation | CNF Data Generation]]<br />
<br />
[[CNF:Monte_Carlo_Event_Generator_using_a_Tessellation | Monte Carlo Event Generator using a Tessellation]]<br />
<br />
[[CNF:Smart_LookUp_Table_Utilities | Smart LookUp Table Utilities]]<br />
<br />
[[CNF:PartonsToNRRD | PartonsToNRRD]]<br />
<br />
[[CNF:Weighted_Mesh_To_Image | Weighted Mesh To Image]]<br />
<br />
[[Internal:TetGen_CNF | TetGen CNF]]<br />
<br />
<br />
<br />
'''Utilities Projects'''<br />
<br />
[[Internal:Synthetical_Image_Data_Generator | Synthetical Image Data Generator]]<br />
<br />
[[Internal:MeshIO | MeshIO]]<br />
<br />
<br />
'''Other Projects'''<br />
<br />
[[Internal:PDM_ND | PDM ND]]<br />
<br />
[[Internal:GPU_Usage | GPU Usage]]<br />
<br />
<br />
'''Reports'''<br />
<br />
[[CNF:CNF_Adaptivity | CNF Adaptivity]]<br />
<br />
== General ==<br />
===Conference Slides ===<br />
[https://odu.box.com/s/vev5nqudpd5yq143xfq99y7b8uyf0j4e AI in Nuclear Physics 2020]<br />
<br />
=== Proposal Resources ===<br />
[[Internal:NSF_PPOSS_Funded_Proposals]]<br />
<br />
===Notes ===<br />
[[Internal:CNF_Workshop]]<br />
<br />
[[Internal:CNF_HPC_Workshop_2019]]<br />
<br />
[[Internal:Kokkos_Tutorial_2020_02_19]]<br />
<br />
[[Internal:Intel_OneAPI_Tutorial_2020_02_20]]<br />
<br />
=== Papers ===<br />
[http://www.cs.odu.edu/crtc/internal/papers/O'Connell_et_al._-_2018_-_Application_of_the_Dependency_Inversion_Principle_.pdf T-infinity: The Dependency Inversion Principle for Rapid and Sustainable Multidisciplinary Software Development]<br />
<br />
[https://hal.inria.fr/hal-01927559 Meshless Voronoi on the GPU, Nicolas Ray,Dmitry Sokolov,Sylvain Lefebvre,Bruno Lévy]<br />
<br />
[http://pages.saclay.inria.fr/frederic.alauzet/TetrahedronVI/Slides_Tetrahedron_VI/Thursday/Mike_Park_ugawg-tet-talk.pdf Democratizing Mesh Adaptation forComputational Fluid Dynamics Michael Park]<br />
<br />
[[Internal:Hashing_Meshes]]<br />
<br />
=== Videos ===<br />
[https://www.youtube.com/watch?v=pn7IqhOEGpY Using Graphs for Unstructured Data (Keshav Pingali)]<br />
<br />
[https://www.youtube.com/watch?v=RZ09uZs1ynA Big Data Brain Maps at Argonne National Laboratory (Bobby Kasthuri), ATPESC 2017]<br />
<br />
=== Other ===<br />
http://livingreview.in.tum.de/welcome/<br />
<br />
=== CRTC lab Virtual meetings ===<br />
* 2020.01.18 - Christos<br />
:* Unified Approach for Speculative Operations, [https://docs.google.com/presentation/d/1haL6NVwZzRVG0W71KoWmqLKgxkf6to2sUsdTH5Xn4kw (presentation)], [https://olddominion-my.sharepoint.com/:f:/r/personal/ctsol001_odu_edu/Documents/Parallel_Operations?csf=1&web=1&e=hQ8rPg (shared directory)]<br />
<br />
* 2021.02.03 - Kevin & Spiros<br />
:* Introduction to GPU Parallelism, [https://drive.google.com/file/d/1vW3VJXi04JB2N7XpVoNnXXSNv4i1Pnet/view?usp=sharing (presentation)]<br />
<br />
* 2021.02.10 - Kevin & Spiros<br />
:* GPU Architecture, [https://drive.google.com/file/d/1LfDwsbdscSpTk3eBT74lfIlSyD-W_Zwh/view?usp=sharing (presentation)]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:PODM_3D&diff=8432Internal:PODM 3D2021-04-13T15:55:36Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Documentation=<br />
==Fall 2019==<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_56.pdf PODM3D] paper✔<br />
# Read [https://crtc.cs.odu.edu/pub/papers/journal_58.pdf Guaranteed Quality Tetrahedral Delaunay Meshing for Medical Images] paper (Sect. 9) (TODO)<br />
# Write a proper README.md file✔<br />
# Add the [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] into the Appendix of the November Report✔<br />
# Create [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity CNF Adaptivity] Report<br />
<br />
==Spring 2020==<br />
# Create a [https://crtc.cs.odu.edu/CRTC_I2M User Manual] for PODM 3D and Tessellate2D within the fields of Medical Image Computing and Computational Nuclear Femptography ✔<br />
<br />
=Execution Examples=<br />
==Fall 2019==<br />
# Create meshes for the Computational Nuclear Femtography part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/CNF_Example_Meshes CNF Example Meshes] ✔<br />
<br />
==Spring 2020==<br />
# Create meshes for the Medical Image Computing part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Medical_Imaging_Example_Meshes Medical Imaging Example Meshes] ✔<br />
<br />
==Summer 2020==<br />
# Create meshes for the Bioinformatics part of CRTC_I2M_Suite Documentation. [https://crtc.cs.odu.edu/Bioinformatics_Example_Meshes Bioinformatics Example Meshes] ✔<br />
<br />
=Code=<br />
==Fall 2019==<br />
# Restructure repository of PODM3D based on Software engineering standards✔<br />
# Decouple Args from PODM✔<br />
# Create setArguments function✔<br />
# Create setInputImage(ImageType::Pointer)✔<br />
# Remove IOs from PODM✔<br />
# Implement a sizing function that will work as an extra rule and it will use the points of the tetrahedron that is tested✔<br />
## Create UML diagrams to understand the code and the Meshing Procedure in general✔<br />
## Create a list of functions that need to be modified to implement the extra Rule ✔<br />
## Modify the created list ([https://docs.google.com/spreadsheets/d/13M6qLRtS3wMH5B8vXwcqA-gP2b9p8QGwbhrn68VXoj0/edit#gid=0 Extra Rule sizing Function list]) of functions✔<br />
## Create the following test sizing function to understand if modifications have been done correctly✔ <syntaxhighlight lang="c++" line='line'><br />
bool checkSizingFunction(double coordinatesOfTetrahedron**)<br />
{<br />
double volume = volume(coordinatesOfTetrahedron);<br />
double* baryCenter = baryCenter(coordinatesOfTetrahedron)<br />
<br />
// if x axis <= 55 and volume of tetrahedron > 10<br />
if (baryCenter[0] <= 55 && volume > 10)<br />
return true;<br />
else<br />
return false<br />
}<br />
</syntaxhighlight><br />
## Create the infrastructure to support Sizing Function pointer✔<br />
## Incorporate Function Pointer to PODM by creating a setSizingFunction()✔<br />
## Replace the test sizing function with the Adaptivity function that was created by Christos (The algorithm is described [https://crtc.cs.odu.edu/Internal:CNF_Adaptivity here])✔<br />
### Understand that the current implementation supports negative weights✔<br />
### Change the current implementation to use the minimum-maximum edge length of the element instead of its volume✔<br />
### Identify an issue of checking sample points outside the image and solve it according to Christos's suggestions✔<br />
# Separate the interface, the IOs and the filters from PODM✔<br />
<br />
==Spring 2020==<br />
# Create a general PODM driver✔<br />
# Set a warning if delta > min-spacing * min-dimension-size / 5✔<br />
# Set a default delta: ✔<br />
## if sizing-function is off then default delta = min-spacing * min-dimension-size / 100 ✔<br />
## if sizing-function is on then default delta = min-spacing * min-dimension-size / 20 ✔<br />
# Remove labels array if there is only 1 label of interest ✔<br />
# Print a warning if a label is not recovered✔<br />
# Add Numa flag in CmakeLists.txt ✔<br />
# Add binary file support ✔<br />
# Improve Text output ✔<br />
# Create label-filter for images ✔<br />
## Preserve a set of labels of an image ✔<br />
## Exclude a set of labels of an image ✔<br />
# Remove redundant volume threshold checks that don't contribute anything to the meshing procedure ✔<br />
<br />
==Summer 2020==<br />
# Libraries:<br />
## Update ITK to v5.1.0 ✔<br />
## Update VTK to v9.0.1 ✔<br />
## Remove Boost from PODM (preserve it only for CGAL) ✔<br />
# Save the label of a cell after its creation and Remove the expensive label recalculation in GetCleanMesh/GetCleanMeshParallel✔<br />
# Compute Mesh Results in Parallel (TODO)<br />
## Implement Parallel version of GetCleanMesh function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:mesh-results-speed-up.png|500px|File:mesh-results-speed-up.png]] ✔<br />
## Assigned as default ✔<br />
# Upgrade the define NO_IO_WRITING<br />
## Rename it to IO_WRITING ✔<br />
## Create functions for loading and writing of auxiliary files ✔<br />
## Create a compile flag for it, which will also enable the creation of the auxiliary files folder ✔<br />
## Fix truncation error when loading box ✔<br />
# Compute Box, Resized image and surface in Parallel ✔<br />
## Implement Parallel version of ComputeBoxAndSurface function ✔<br />
## Testing ✔<br />
## Speed-Up [[File:box-surface-speed-up.png|500px|File:box-surface-speed-up.png]]✔<br />
## Assigned as default ✔<br />
# Sizing function:<br />
## Control min edge and max edge using a percentage of the diagonal ✔<br />
## Find values of samples points using linear interpolation ✔<br />
<br />
==Fall 2020==<br />
# Fix shift of origin if delta not multiple of the spacing ✔<br />
# Handle CNF images with spacing different than 1, and origin different than {0, 0, 0} ✔<br />
## Upgrade Image Segmentation ✔<br />
## Upgrade Sizing function ✔<br />
## Upgrade Linear Interpolation ✔<br />
# Change ImageSegmentation to create an image inside the requested grid domain ✔<br />
<br />
==Spring 2021==<br />
# [Christos] Addition of Extra points at the beginning of meshing ✔<br />
# [Polykarpos] Loading and saving the state of PODM mesher ✔<br />
# Load Dynamically Sizing Functions and their initialization as a shared Library using function pointers ✔<br />
# Detect special cases of Christos and Polykarpos additions ✔<br />
# Convert PODM to accept double precision CNF images ✔<br />
# Fix PODM not working on Wahab (TODO)<br />
# Remeshing by using a mesh instead of an image as input and background mesh (TODO)<br />
# Create upscale-image-filter (TODO)<br />
# Replace CGAL Predicates with: (TODO)<br />
## Predicates Construction Kit:<br />
### [https://hal.inria.fr/hal-01225202 Paper]<br />
### [https://gforge.inria.fr/frs/?group_id=5833 Code]<br />
### Benchmark: [https://crtc.cs.odu.edu/Internal:4D_Predicates_Benchmark PCK vs CGAL Predicates Benchmark]<br />
### Note: see Geogram branch<br />
## Adaptive Precision Floating-Point Arithmetic and Fast Robust Predicates for Computational Geometry:<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/papers/robust-predicates.ps Paper]<br />
### [https://www.cs.cmu.edu/afs/cs/project/quake/public/code/predicates.c Code]<br />
# Smoothing (TODO)<br />
## [https://hal.inria.fr/inria-00413248/document Feature preserving Delaunay mesh generation from 3D multi-material images]<br />
## CGAL Sources: <br />
### https://doc.cgal.org/latest/Mesh_3/index.html<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3protectionimage3D<br />
### https://doc.cgal.org/latest/Mesh_3/index.html#fig__mesh3HybridDomainExample<br />
## PODM3D citations:<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999101968160 Volume conserving smoothing for piecewise linear curves, surfaces, and triple lines]<br />
### [https://www-sciencedirect-com.proxy.lib.odu.edu/science/article/pii/S0021999108004981 An anisotropic scale-invariant unstructured mesh generator suitable for volumetric imaging data]<br />
### [https://pubmed.ncbi.nlm.nih.gov/20161102/ Variational generation of prismatic boundary-layer meshes for biomedical computing, International Journal for Numerical Methods]<br />
## Prof, Andrey Suggestions:<br />
### [https://people.eecs.berkeley.edu/~jrs/papers/stuffing.pdf Isosurface Stuffing: Fast Tetrahedral Meshes with Good Dihedral Angles]<br />
### [https://imr.sandia.gov/_assets/documents/2019_IMR_Papers/3B.2-Chernikov.pdf CERTIFIED FUNCTIONS FOR MESH GENERATION]<br />
### [https://crtc.cs.odu.edu/pub/papers/journal_49.pdf GENERALIZED INSERTION REGION GUIDES FOR DELAUNAY MESH REFINEMENT]<br />
# Μultiple δ (TODO)<br />
## Implement multiple δ (TODO)<br />
## Use different δ values (possibly uniform based) according to the surface of each label of a segmented image (TODO)<br />
# Produce the Voronoi Mesh out of the Delaunay (TODO)<br />
## Check what TetGen does (TODO)<br />
## Research conversion methods (TODO)</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=Internal:CDT3D_GPU_Smoothing&diff=8431Internal:CDT3D GPU Smoothing2021-04-13T03:05:27Z<p>Spyridon97: /* Spring 2021 */</p>
<hr />
<div>__TOC__<br />
<br />
=Presentations=<br />
==Spring 2021==<br />
<br />
# [https://docs.google.com/presentation/d/1haL6NVwZzRVG0W71KoWmqLKgxkf6to2sUsdTH5Xn4kw/edit A Unified Approach for Parallel Mesh Operations] by Christos✔<br />
# [https://docs.google.com/presentation/d/1LfDwsbdscSpTk3eBT74lfIlSyD-W_Zwh/edit GPU Architecture] ✔<br />
<br />
=Code=<br />
==Spring 2021==<br />
<br />
# Discrete Mesh Optimization using the paper: [https://project.inria.fr/imr27/files/2018/09/1003.pdf Discrete Mesh Optimization on GPU], The results below are using the tet_shape metric from the [http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=42FDC437179F12D08EB6C4F84FACD490?doi=10.1.1.483.5380&rep=rep1&type=pdf Verdict library]<br />
## CPU version:<br />
### Sequential:<br />
#### Dihedral Quality of Discrete Vertex smoothing with Global Iterations:✔<br> [[File:Discrete_Vertex_Smoothing_Dihedral_Angle_Quality_Global_Iterations.png|1000px|Discrete_Vertex_Smoothing_Dihedral_Angle_Quality_Global_Iterations.png]]<br />
#### Dihedral Quality of different smoothing methods:✔<br>[[File:Vertex_Smoothing_Methods_Dihedral_Angle_Quality.png|1000px|Vertex_Smoothing_Methods_Dihedral_Angle_Quality.png]]<br />
#### Shape Quality of different smoothing methods:✔<br>[[File:Shape_Quality.png|1000px|Shape_Quality.png]]<br />
#### Distribution of Smoothing Time Per Vertex:✔<br> [[File:Distribution_of_Smoothing_Time_Per_Vertex_CPU.png|1000px|Distribution_of_Smoothing_Time_Per_Vertex_CPU.png]]<br />
### Parallel:<br />
#### Dihedral Quality: Sequential vs Parallel:✔<br>[[File:Discrete_Vertex_Smoothing_Dihedral_Angle_Quality.png|1000px|Discrete_Vertex_Smoothing_Dihedral_Angle_Quality.png]]<br />
#### Speed-Up:✔<br>[[File:Smoothing_Speed-up.png|750px|Smoothing_Speed-up.png]]<br />
#### RollBacks✔:<br>[[File:Percentage_of_Rollbacks_per_Iteration.png|750px|Percentage_of_Rollbacks_per_Iteration.png]]<br />
## GPU version:<br />
### Sequential CPU-wise:<br />
#### Distribution of Smoothing Time Per Vertex:✔<br> [[File:Distribution_of_Smoothing_Time_Per_Vertex_GPU.png|1000px|Distribution_of_Smoothing_Time_Per_Vertex_GPU.png]]<br />
#### Version 1 (all points copied):<br />
##### Steps:<br />
###### Bounding box on GPU, grid(1) block(1)<br />
###### Original point value on GPU, grid(1) block(1)<br />
###### Grid creation on GPU, grid(1) block(1)<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
##### Time total = 18.563 sec (90 sec on CPU sequential)<br />
###### CPU = 0.79182 sec<br />
###### Steps 1 & 2 = 8.31743 sec<br />
###### Step 3 = 1.03768 sec<br />
###### Step 4 = 6.55488 sec (per point)<br />
###### Step 5 = 1.73327 sec<br />
###### MemCopyH2D = 0.12792 sec<br />
###### MemCopyD2H = 0.00002 sec<br />
#### Version 2.0 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
#######!!! Note: access global memory multiple times !!!<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 15.06656 sec<br />
###### CPU = 1.8857 sec (all cavities)<br />
###### Step 3 = 6.43994 sec<br />
###### Step 4 = 6.20552 sec<br />
###### MemCopyH2D = 0.38211 sec<br />
###### MemCopyD2H = 0.15329 sec<br />
#### Version 2.1 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values array(512) on GPU, grid(2, 2, 2) block(4, 4, 4)<br />
###### Reduction using global values array(512) on GPU, grid(1) block(1)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 10.8227 sec<br />
###### CPU = 1.8857 sec<br />
###### Step 3 = 6.47006 sec<br />
###### Step 4 = 1.74147 sec<br />
###### MemCopyH2D = 0.37493 sec<br />
###### MemCopyD2H = 0.15225 sec<br />
#### Version 3 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(8+8) and reduction for the max value and id of a block on GPU, grid(8) block(64)<br />
###### Reduction using global values and ids arrays(8+8) on GPU, grid(1) block(8)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 9.17189 sec<br />
###### CPU = 2.04448 sec<br />
###### Step 3 = 6.40427 sec<br />
###### Step 4 = 0.19378 sec<br />
###### MemCopyH2D = 0.37604 sec<br />
###### MemCopyD2H = 0.15332 sec<br />
#### Version 3.1 (copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.77189 sec<br />
###### CPU = 2.04448 sec (Data Transformation = 0.194 sec)<br />
###### Step 3 = 6.0427 sec<br />
###### Step 4 = 0.19378 sec<br />
###### MemCopyH2D = 0.37604 sec<br />
###### MemCopyD2H = 0.15332 sec<br />
#### Version 3.2 (allocate what you need at the beginning & copy what you need):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.34358 sec<br />
###### CPU = 1.58702 sec<br />
###### Step 3 = 6.02387s sec<br />
###### Step 4 = 0.20711ms sec<br />
###### MemCopyH2D = 0.37380 sec<br />
###### MemCopyD2H = 0.15178 sec<br />
#### Version 3.3 (allocate what you need at the beginning & copy what you need & and combine the GPU data):<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 8.06576 sec<br />
###### CPU = 1.466553 sec<br />
###### Step 3 = 5.99690 sec<br />
###### Step 4 = 0.20702 sec<br />
###### MemCopyH2D = 0.25319 sec<br />
###### MemCopyD2H = 0.078917 sec<br />
#### Version 4 (perfom batching with 1 stream per cpu-thread):<br />
##### For a batch of 4 vertices:<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time total = 7.18418 sec<br />
###### CPU = 0.946749 sec<br />
###### Step 3 = 5.94023 sec<br />
###### Step 4 = 0.20709 sec<br />
###### MemCopyH2D = 0.069697 sec<br />
###### MemCopyD2H = 0.020414 sec<br />
##### Batch size:✔ <br>[[File:BATCH-SIZE.v4.png|750px|BATCH-SIZE.v4.png]]<br />
#### Version 4.1 (perfom batching with #streams = batch-size within each cpu-thread):<br />
##### For a batch of 4 vertices<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
#### Version 4.2 (perfom batching with #streams = << batch-size within each cpu-thread):<br />
##### For a batch of 12 vertices with 4 streams<br />
##### Steps:<br />
###### Bounding box on CPU<br />
###### Grid creation on CPU<br />
###### Gridpoint values using global values and ids arrays(16+16) and reduction for the max value and id of a block on GPU, grid(16) block(32)<br />
###### Reduction using global values and ids arrays(16+16) on GPU, grid(1) block(16)<br />
###### Original point value on CPU concurrently with steps 3 & 4<br />
##### Time:<br />
###### lock time: 0.0190836 sec<br />
###### findCavityCells time: 0.178939 sec<br />
###### findUniqueVertices time: 0.215159 sec<br />
###### dataCollection time: 0.187204 sec<br />
###### boundingBox time: 0.0280833 sec<br />
###### grid time: 0.00122939 sec<br />
###### originalPointValue time: 0.12965 sec<br />
###### assignNewPoint time: 0.00165265 sec<br />
###### unlock time: 0.0117685 sec<br />
###### Total Cpu time: 0.77277 sec<br />
### Parallel CPU-wise:<br />
#### Version 4:<br />
##### Speed-Up:✔<br>[[File:Smoothing_Speed-up-heterogeneous.png|750px|Smoothing_Speed-up-heterogeneous.png]]<br />
##### Rollbacks-Time-Per-Thread:✔<br>[[File:parallel.v4.png|750px|parallel.v4.png]]<br />
##### Profiler: 4 cpu threads, 4 streams✔<br> [[File:profiler.v4.png|750px|profiler.v4.png]]<br />
#### Version 4.1:<br />
##### Profiler: 2 cpu threads, 2(copy)+8(computation) streams✔<br> [[File:profiler.v4.1.png|750px|profiler.v4.1.png]]<br />
#### Version 4.2:<br />
##### Rollbacks-Time-Per-Thread (#vertices = 28,217):✔<br>[[File:parallel.v4.2.png|750px|parallel.v4.2.png]]<br />
##### Rollbacks-Time-Per-Thread (#vertices = 79,809):✔<br>[[File:parallel.v4.2-double-size.png|750px|parallel.v4.2-double-size.png]]<br />
###### CPU (1 thread) 666 sec, CPU (32 threads) 28.29 sec<br />
##### Profiler: 2 cpu threads, 2(copy)+8(computation) streams✔<br> [[File:profiler.v4.2.png|750px|profiler.v4.2.png]]<br />
##### CPU Time Histogram 1 thread:✔<br>[[File:Histogram_of_CPU_Batch_Time_v4.2_threads=1.png|750px|Histogram_of_CPU_Batch_Time_v4.2_threads=1.png]]<br />
##### CPU Time Histogram 2 threads:✔<br>[[File:Histogram_of_CPU_Batch_Time_v4.2_threads=2.png|750px|Histogram_of_CPU_Batch_Time_v4.2_threads=2.png]]</div>Spyridon97https://crtc.cs.odu.edu/index.php?title=File:Parallel.v4.2-double-size.png&diff=8430File:Parallel.v4.2-double-size.png2021-04-13T01:40:49Z<p>Spyridon97: </p>
<hr />
<div></div>Spyridon97