Internal:Metric Adaptive CDT3D

From crtc.cs.odu.edu
Revision as of 09:00, 28 May 2018 by Ctsolakis (talk | contribs) (→Scalable complexity (In-memory): comment out turing)
Jump to navigation Jump to search

Delta Wing (Scitech 2018)

500k Complexity

Starting from a 50K Complexity volume mesh

Updated on 2018.05.22

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,516,917 926,822 2.22 28 20.95 19.44 1.65 2.85 665.49 712.60 1.31
2 5,557,272 933,556 2.41 28 11.36 11.85 1.82 3.47 332.76 363.67 1.7
4 5,551,696 932,709 2.48 30 6.22 6.86 1.74 3.57 177.65 198.52 1.43
6 5,561,503 934,363 2.39 26 3.95 4.37 1.55 3.13 111.55 126.94 1.46
8 5,563,801 934,630 2.36 27 3.27 3.93 1.66 3.32 90.57 105.11 1.47
12 5,563,857 934,699 2.44 28 2.27 2.99 1.62 3.23 62.54 75.08 1.58
16 5,558,467 933,729 2.68 27 1.76 2.88 1.59 3.26 46.78 58.96 1.5
24 5,558,489 933,876 2.43 27 1.31 2.64 1.63 3.11 32.19 43.31 1.81
24 + 12HT 5,553,184 932,833 2.45 27 1.15 2.62 1.55 3.13 27.71 38.62 2.26
24 + 24HT 5,567,259 935,267 2.35 27 1.04 2.84 1.81 3.28 24.42 35.75 2.72
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,926.36 6,926.36 1,154.25 1,154.25
2 13,682.94 6,841.47 2,280.23 1,140.11
4 25,037.78 6,259.44 4,172.90 1,043.22
6 39,233.55 6,538.93 6,538.98 1,089.83
8 47,403.72 5,925.46 7,899.58 987.45
12 66,364.69 5,530.39 11,060.12 921.68
16 84,417.76 5,276.11 14,067.57 879.22
24 114,922.49 4,788.44 19,154.26 798.09
24 + 12HT 128,741.27 — 21,453.34 —
24 + 24HT 139,470.29 — 23,243.69 —


This approach offers no boundary adaptation. #Contents

Starting for a coarse surface mesh

Updated on 2018.05.20

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Intitialize Bg Mesh Boundary Adaptation (MMGS) Boundary Recovery Mesh Refinement Total Time

(without MMGS)

Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,572,743 941,913 1.47 29.77 6.74 25 17.82 83.22 1.63 2.17 667.87 811.21 1.96
6 5,598,669 946,401 1.36 30.03 5.61 26 3.56 16.21 1.73 2.62 108.01 169.65 1.61
12 5,597,829 946,215 1.45 30.02 5.82 27 1.99 8.47 1.76 2.52 59.71 112.25 1.71
24 5,596,756 946,067 1.24 29.88 6.82 28 1.16 4.54 1.67 2.51 31.17 79.52 1.95
24 + 12HT 5,607,155 947,817 1.22 30.03 6.28 27 0.96 3.76 1.72 2.44 26.45 73.40 2.33
24 + 24HT 5,617,086 949,525 1.47 30.01 6.54 27 0.89 3.53 1.71 2.55 24.05 71.26 2.78
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,869.67 6,869.67 1,161.12 1,161.12
6 33,001.29 5,500.22 5,578.55 929.76
12 49,869.30 4,155.78 8,429.53 702.46
24 70,381.74 2,932.57 11,897.22 495.72
24 + 12HT 76,391.76 — 12,913.04 —
24 + 24HT 78,825.23 — 13,324.80 —

#Contents

Notes 2

  • Speed 35.1K test/sec (40.6K tets/sec w/o body adaptation) and 5.9K ps/sec. The mesh is about 15.2 times larger in terms of test and the speed is 1.56 larger!
  • In speed table of volume reconstruction initial number of tetrahedra and vertices is subtracted.
  • In speed table of with surface adaptation MMGS time is not included.
  • Target is 1M vertices.
  • Quality comparison starting from surface vs volume
  • in some cases boundary recovery reached a point of non-developed code. Rerunning created a different surface mesh and CDT3D was able to continue.
  • commit fb0f106c6b65107
  • common parameters --met=1 --mrecm2=1 --mrec4=1 --malign=0 --mpp=2 --qstatr=1 --outmesh=${NAME}.meshb --ndfs=0 --nthreads=$THREADS --nbuckets=20*NTHREADS --nthreadsaf=20*NTHREADS --nbucketsaf=20*NTHREADS --blockmul=500000 --nqual=0 --nminpts=30 --ordt=1 --pardeact=1 --mdse=0 --cdfn=0.55
  • non-common parameters
    • reconstruct --meval=1 --reconstruct=1
    • surf adapt --brec=1 --meval=3 --nadapt=2

#Contents

1M Complexity (~12M Tetrahedra)

Starting from a 50K Complexity volume mesh

Updated on 2018.05.20

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 11,130,150 1,865,647 2.15 29 1.94 6.88 3.53 8.25 52.12 74.88 3.53
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 140,878.13 2,934.96 23,522.20 490.05

#Contents

10M Complexity (~120M Tetrahedra)

Starting from a 50K Complexity volume mesh

Updated on 2018.05.20

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 109,884,436 18,365,004 2.41 102 30.51 77.69 39.25 246.50 883.71 1280.07 25.91
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 85,388.49 — 14,265.39 —

#Contents

100M Complexity (~1.2B Tetrahedra)

Starting from a 50K Complexity volume mesh

Updated on 2018.05.20

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 1,096,524,832 183,097,400 2.90 236 509.81 950.33 389.19 4.731.73 14,991.60 21,575.50 247.17
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 50,795.75 1,058.24 8,481.52 176.70

#Contents

Scalable complexity

# Threads # Initial Complexity # Target Complexity
1 50k 500k
2 500k 1M
4 1M 2M
8 2M 4M
16 4M 8M
32 8M 16M

coreV2-23-himem (Turing cluster)

All time data are given in seconds ( himem node on turing).
# Threads Initial #Tetrahedra Inital #Vertices # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 581,196 104,305 5,516,764 926,798 2.68 28 26.15 21.28 1.62 3.29 806.01 861.03 1.83
2 5,516,764 926,798 11,539,471 1,933,053 35.36 17 25.25 24.78 3.12 6.01 598.95 693.48 4.06
4 11,539,471 1,933,053 23,144,340 3,871,241 70.92 19 31.84 27.21 5.85 11.88 596.05 743.76 6.93
8 23,144,340 3,871,241 46,382,851 7,749,804 204.57 22 46.69 23.43 7.33 17.03 535.20 834.24 14.20
16 46,382,851 7,749,804 92,749,306 15,487,503 317.11 27 70.50 40.30 27.93 65.46 696.53 1,217.84 29.08
32 92,749,306 15,487,503 185,408,908 30,946,344 618.62 56 136.99 85.39 50.42 216.3 883.8 1,991.52 59.46
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,732.17 5,732.17 955.24 955.24
2 8,684.76 4,342.38 1,451.02 725.51
4 15,602.98 3,900.74 2,605.93 651.48
8 27,855.91 3,481.99 4,649.22 581.15
16 38,072.70 2,379.54 6,353.63 397.10
32 46,527.08 1,453.97 7,762.33 242.57

crtc-gpu

All time data are given in seconds (crtc-gpu in lab).
# Threads Initial #Tetrahedra Inital #Vertices # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 581,196 104,305 5,516,897 926,820 2.24 28 21.14 19.25 1.55 3.07 676.20 723.45 1.82
2 5,516,897 926,820 11,542,316 1,933,496 30.28 17 17.66 18.13 1.94 3.41 416.84 488.26 3.94
4 11,542,316 1,933,496 23,159,373 3,873,738 61.72 18 22.58 20.86 4.93 10.48 418.44 539.00 6.96
8 23,159,373 3,873,738 46,405,798 7,753,398 168.19 23 36.93 26.19 8.29 20.88 475.23 735.71 14.34
16 46,405,798 7,753,398 92,786,435 15,493,130 305.83 32 62.40 35.86 15.78 50.83 561.35 1,032.06 29.41
24 92,786,435 15,493,130 144,584,441 24,132,712 546.54 36 134.26 41.90 23.26 76.29 493.90 1,316.15 45.19
24 + 12HT 144,584,441 24,132,712 215,865,522 36,034,558 1,172.70 34 216.46 57.87 30.66 109.77 585.90 2,173.36 67.39
24 + 24HT 215,865,522 36,034,558 322,632,552 53,862,057 1,447.57 50 284.30 94.22 36.90 239.08 885.71 2,987.80 100.32
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,822.45 6,822.45 1,136.93 1,136.93
2 12,340.60 6,170.30 2,061.76 1,030.88
4 21,552.98 5,388.25 3,599.71 899.93
8 31,597.27 3,949.66 5,273.35 659.17
16 44,939.86 2,808.74 7,499.30 468.71
24 39,355.70 1,639.82 6,564.28 273.51
24 + 12HT 32,797.64 — 5,476.24 —
24 + 24HT 35,734.33 — 5,966.76 —

Specifications

hostname nodes memory (GB) cache (MB) Model Name Turbo Cores per node Threads per core sockets Cores per socket NUMA-nodes
coreV2-23-himem-* 4 757 16 Intel® Xeon® E5-4610 v2 @ 2.30GHz 2.70GHz 32 1 4 8 [0-7],[8-15],[16-23],[24-31]
crtc-gpu 1 757 30 Intel® Xeon® E5-2697 v2 @ 2.70GHz 3.50GHz 24 2 2 12 [0,2,..46],[1,3..47]

Notes

In speed table the initial number of tetrahedra and vertices is subtracted.

#Contents

Scalable complexity (In-memory)

# Threads # Initial Complexity # Target Complexity
1 50k 500k
2 500k 1M
4 1M 2M
8 2M 4M
16 4M 8M
32 8M 16M

crtc-gpu

All time data are given in seconds (crtc-gpu in lab).
# Threads Initial #Tetrahedra Inital #Vertices # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 581,196 104,305 5,516,897 926,820 2.24 28 21.14 19.25 1.55 3.07 676.20 723.45 1.82
2 5,516,897 926,820 11,542,316 1,933,496 30.28 17 17.66 18.13 1.94 3.41 416.84 488.26 3.94
4 11,542,316 1,933,496 23,159,373 3,873,738 61.72 18 22.58 20.86 4.93 10.48 418.44 539.00 6.96
8 23,159,373 3,873,738 46,405,798 7,753,398 168.19 23 36.93 26.19 8.29 20.88 475.23 735.71 14.34
16 46,405,798 7,753,398 92,786,435 15,493,130 305.83 32 62.40 35.86 15.78 50.83 561.35 1,032.06 29.41
24 92,786,435 15,493,130 144,584,441 24,132,712 546.54 36 134.26 41.90 23.26 76.29 493.90 1,316.15 45.19
24 + 12HT 144,584,441 24,132,712 215,865,522 36,034,558 1,172.70 34 216.46 57.87 30.66 109.77 585.90 2,173.36 67.39
24 + 24HT 215,865,522 36,034,558 322,632,552 53,862,057 1,447.57 50 284.30 94.22 36.90 239.08 885.71 2,987.80 100.32
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,822.45 6,822.45 1,136.93 1,136.93
2 12,340.60 6,170.30 2,061.76 1,030.88
4 21,552.98 5,388.25 3,599.71 899.93
8 31,597.27 3,949.66 5,273.35 659.17
16 44,939.86 2,808.74 7,499.30 468.71
24 39,355.70 1,639.82 6,564.28 273.51
24 + 12HT 32,797.64 — 5,476.24 —
24 + 24HT 35,734.33 — 5,966.76 —

Cube (polar-2 metric) (Scitech 2018)

Starting from a 8K Complexity volume mesh

Updated on 2018.05.22

Size of intial mesh

Vertices : 17,767

Tetrahedra: 91,414

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,824,738 981,297 0.47 37 29.47 31.40 2.27 6.38 978.43 1,048.43 1.85
2 5,849,206 985,342 0.86 39 15.98 18.39 2.01 6.54 468.29 512.08 1.77
4 5,858,367 986,927 0.77 39 8.32 9.92 1.94 6.86 244.80 272.62 1.43
8 5,877,203 990,141 0.73 42 4.64 5.97 1.87 7.33 134.06 154.6 1.55
16 5,878,983 990,525 0.74 49 2.84 4.40 1.85 8.14 80.86 98.82 1.63
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,468.49 5,468.49 919.02 919.02
2 11,243.93 5,621.97 1,889.50 944.75
4 21,153.81 5,288.45 3,554.98 888.75
8 37,424.25 4,678.03 6,289.61 786.20
16 58,566.78 3,660.42 9,843.74 615.23


This approach offers no boundary adaptation.

General Notes

#Contents

Stability for 500k case

Comparison of Quality Metric using different amount of cores and surface adaptation

#Contents

Comparison of Quality Metric using different amount of cores and mesh reconstruction

#Contents

Comparison Surface Adaptation vs Mesh Reconstruction (reproducibility)

TODO needs update

Surface adaptation offers better edge lengths but mesh reconstruction slightly better mean ratio quality

Need update

50k Complexity

Starting from a coarse surface mesh

TODO needs update

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Intitialize Bg Mesh Boundary Adaptation (MMGS) Boundary Recovery Mesh Refinement Total Time

(without MMGS)

Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 357,165 62,177 1.44 6.32 0.36 20 2.09 11.92 0.09 1.26 57.35 80.97 0.44
6 361,037 62,795 1.49 6.23 0.35 21 0.51 2.78 0.11 1.45 14.06 27.12 0.63
12 363,278 63,198 1.41 6.27 0.33 22 0.32 1.55 0.11 1.62 8.24 19.99 0.86
24 363,756 63,270 1.49 6.26 0.33 22 0.2 0.88 0.11 1.6 5.13 16.15 1.31

#Contents

Notes 1

  • Speed 22.5K test/sec and 3.9 ps/sec
  • according to Mike's email mesh should be around 100k vertices
  • Metric Input Data available at https://github.com/UGAWG/parallel-adapt-results
  • Input surface is the coase mesh available at https://github.com/UGAWG/solution-adapt-cases
  • The experiments performed on a DELL workstation with Linux Red Hat Enterprise, 24 hardware cores Intel® Xeon® CPU E5-2697v2@2.70 GHz, and 757 GB RAM
  • Memory is measured at the end of Mesh refinement and thus it may reach a slightly higher value before the point of measurement
  • parameters --brec=2 --met=1 --meval=3 --mrecm2=1 --mrec4=1 --nadapt=2 --malign=0 --mpp=2 --ndfs=0 --nthreads=$THREADS --nbuckets=20*$THREADS --nthreadsaf=20*$THREADS --nbucketsaf=20*${THREADS} --blockmul=500000 --nqual=0 --nminpts=30 --ordt=0 --pardeact=1 --mdse=0
  • using commit bb7819619eb9b8831ac1d4515b9605869df72b47 or tag Delta-wiki

#Contents

2M Complexity

Starting from a 50K Complexity volume mesh

needs update for ordt=1

Size of initial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Intitialize Bg Mesh Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 21,865,792 3,660,818 1.22 2.02 26 176.15 515.30 7.25 149.05 7,643.51 8,495.57 6.97
6 21,953,024 3,675,913 1.28 2.04 32 53.90 124.95 7.40 231.48 1,340.84 1,762.97 6.49
12 21,960,752 3,677,185 1.16 2.02 32 29.69 70.13 7.01 239.09 702.43 1,052.64 6.41
24 21,966,402 3,678,089 1.34 2.02 29 15.04 38.25 6.33 214.75 332.75 611.56 6.68


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 2,573.72 2,573.72 418.63 418.63
6 12,451.97 2,075.33 2,025.90 337.65
12 20,862.00 1,738.50 3,394.21 282.85
24 35,917.69 1,496.57 5,843.72 243.49

#Contents

Starting from a coarse Surface mesh

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Intitialize Bg Mesh Boundary Adaptation (MMGS) Boundary Recovery Mesh Refinement Total Time

(without MMGS)

Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 21,972,547 3,697,482 1.44 75.89 36.64 28 167.11 402.53 7.59 144.66 7,305.68 8,143.17 7.04
6 22,097,857 3,718,978 1.47 75.71 36.00 31 51.16 99.82 8.98 198.95 1,331.57 1,805.32 5.93
12 22,096,517 3,718,207 1.46 75.68 38.99 32 28.09 54.64 8.3 214.43 685.94 1,109.18 6.0
24 22,094,983 3,718,117 1.36 76.06 38.96 30 14.43 29.23 7.38 201.29 331.19 701.51 6.04


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 2,698.28 2,698.28 454.06 454.06
6 12,240.41 2,040.07 2,060.01 343.34
12 19,921.49 1,660.12 3,352.21 279.35
24 31,496.32 1,312.35 5,300.16 220.84

#Contents

Notes 3

  • In speed table of volume reconstruction the initial number of tetrahedra and vertices is subtracted.
  • In speed table with surface adaptation MMGS time is not included.
  • parameters same as above
Retrieved from "https://crtc.cs.odu.edu/index.php?title=Internal:Metric_Adaptive_CDT3D&oldid=1438"