Internal:ACDT3D Scitech2018 strong scaling (Delta Wing): Difference between revisions

From crtc.cs.odu.edu
Jump to navigation Jump to search
← Older editNewer edit →
Ctsolakis (talk | contribs)
BrainMapping, CNF, HPC, I2MConversion, NDAM, Segmentation, UltraSound3D, Bureaucrats, external, internal, Administrators
2,152 edits
Ctsolakis (talk | contribs)
BrainMapping, CNF, HPC, I2MConversion, NDAM, Segmentation, UltraSound3D, Bureaucrats, external, internal, Administrators
2,152 edits
No edit summary
Line 481: Line 481:
[[#Contents | Back To Top]]
[[#Contents | Back To Top]]


= 1M Complexity (~12M Tetrahedra) =
''' Updated on 2018.05.29'''
Size of intial mesh
Vertices : 104,305
Tetrahedra: 581,196
== crtc-gpu ==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds.(crtc-gpu)
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,122,131
| 1,864,227 
| 2.38     
| 28
| 2.53     
| 5.95     
| 3.48     
| 7.73     
| 69.20     
| 91.29     
| 3.15
|-
| 24 +12HT         
| 11,125,744
| 1,864,711
| 2.27     
| 30
| 2.23     
| 6.75     
| 3.62     
| 7.94     
| 62.07   
| 84.89   
| 3.26
|-
| 24 + 24HT
| 11,130,150                           
| 1,865,647                                                                                                             
| 2.15                                                           
| 29                                                           
| 1.94                                                           
| 6.88                                                           
| 3.53                                                           
| 8.25                                                           
| 52.12                                                           
| 74.88                                                           
| 3.53
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
| 24
| 115,466.48
| 4,811.10
| 19,278.37
| 803.27
|-
| 24+ 12HT
| 124,214.25
| —
| 20,737.50
| —
|-
| 24 + 24HT
| 140,878.13
| —
| 23,522.20
| —
|}
==turing coreV2-himem==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds ( turing coreV2-himem).
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,119,089.
| 1,863,716
| 2.64     
| 28
| 4.59     
| 7.26     
| 4.58     
| 9.93     
| 83.8     
| 112.8     
| 3.13
|-
| 32
| 11,114,943
| 1,862,960
| 2.60     
| 29
| 3.78     
| 7.53     
| 4.63     
| 10.47     
| 62.41     
| 91.44     
| 2.95
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
|24
|93,421.04
|3,892.54
|15,597.62
|649.90
|-
|32
|115,198.46
|3,599.95
|19,232.88
|601.03
|}
== bridges.psc ==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds (  bridges.psc).
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,128,838
| 1,865,236
| 2.05
| 26
| 6.58
| 6.95
| 4.43
| 8.78
| 202.82
| 231.6
| 2.91
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
|24
| 45,542.50
| 1,897.60
| 7,603.33
| 316.81
|}


= Specifications =
= Specifications =
Line 1,039: Line 831:
| 8,481.52
| 8,481.52
| 176.70
| 176.70
|}
= NOT USED in paper =
= 1M Complexity (~12M Tetrahedra) =
''' Updated on 2018.05.29'''
Size of intial mesh
Vertices : 104,305
Tetrahedra: 581,196
== crtc-gpu ==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds.(crtc-gpu)
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,122,131
| 1,864,227 
| 2.38     
| 28
| 2.53     
| 5.95     
| 3.48     
| 7.73     
| 69.20     
| 91.29     
| 3.15
|-
| 24 +12HT         
| 11,125,744
| 1,864,711
| 2.27     
| 30
| 2.23     
| 6.75     
| 3.62     
| 7.94     
| 62.07   
| 84.89   
| 3.26
|-
| 24 + 24HT
| 11,130,150                           
| 1,865,647                                                                                                             
| 2.15                                                           
| 29                                                           
| 1.94                                                           
| 6.88                                                           
| 3.53                                                           
| 8.25                                                           
| 52.12                                                           
| 74.88                                                           
| 3.53
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
| 24
| 115,466.48
| 4,811.10
| 19,278.37
| 803.27
|-
| 24+ 12HT
| 124,214.25
| —
| 20,737.50
| —
|-
| 24 + 24HT
| 140,878.13
| —
| 23,522.20
| —
|}
==turing coreV2-himem==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds ( turing coreV2-himem).
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,119,089.
| 1,863,716
| 2.64     
| 28
| 4.59     
| 7.26     
| 4.58     
| 9.93     
| 83.8     
| 112.8     
| 3.13
|-
| 32
| 11,114,943
| 1,862,960
| 2.60     
| 29
| 3.78     
| 7.53     
| 4.63     
| 10.47     
| 62.41     
| 91.44     
| 2.95
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
|24
|93,421.04
|3,892.54
|15,597.62
|649.90
|-
|32
|115,198.46
|3,599.95
|19,232.88
|601.03
|}
== bridges.psc ==
{| class="wikitable" style="text-align: right;"
|+ All time data are given in seconds (  bridges.psc).
! rowspan="2" | # Threads
! rowspan="2" | # Tetrahedra
! rowspan="2" | # Vertices
! rowspan="2" | Reconstruct Mesh
! colspan="6" style="text-align: center;" | Mesh Refinement
! rowspan="2" | Total Time
! rowspan="2" | Memory (GB)
|-
! # Iterations
! Par. Deactivation
! Par. Vertex Creation
! Vertex Insertion
! Partition
! Par. Reconnection
|-
| 24
| 11,128,838
| 1,865,236
| 2.05
| 26
| 6.58
| 6.95
| 4.43
| 8.78
| 202.82
| 231.6
| 2.91
|}
{| class="wikitable" style="text-align: right;"
|+ Speed
! # Threads
! # Tetrahedra/sec
! # Tetrahedra/sec/core
! # Vertices/sec
! # Vertices/sec/core
|-
|24
| 45,542.50
| 1,897.60
| 7,603.33
| 316.81
|}
|}
[[#Contents | Back To Top]]
[[#Contents | Back To Top]]

Revision as of 10:24, 2 June 2018

500k Complexity (~6M Tetrahedra)

crtc.gpu

Updated on 2018.05.22

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,516,917 926,822 2.22 28 20.95 19.44 1.65 2.85 665.49 712.60 1.31
2 5,557,272 933,556 2.41 28 11.36 11.85 1.82 3.47 332.76 363.67 1.7
4 5,551,696 932,709 2.48 30 6.22 6.86 1.74 3.57 177.65 198.52 1.43
6 5,561,503 934,363 2.39 26 3.95 4.37 1.55 3.13 111.55 126.94 1.46
8 5,563,801 934,630 2.36 27 3.27 3.93 1.66 3.32 90.57 105.11 1.47
12 5,563,857 934,699 2.44 28 2.27 2.99 1.62 3.23 62.54 75.08 1.58
16 5,558,467 933,729 2.68 27 1.76 2.88 1.59 3.26 46.78 58.96 1.5
24 5,558,489 933,876 2.43 27 1.31 2.64 1.63 3.11 32.19 43.31 1.81
24 + 12HT 5,553,184 932,833 2.45 27 1.15 2.62 1.55 3.13 27.71 38.62 2.26
24 + 24HT 5,567,259 935,267 2.35 27 1.04 2.84 1.81 3.28 24.42 35.75 2.72
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,926.36 6,926.36 1,154.25 1,154.25
2 13,682.94 6,841.47 2,280.23 1,140.11
4 25,037.78 6,259.44 4,172.90 1,043.22
6 39,233.55 6,538.93 6,538.98 1,089.83
8 47,403.72 5,925.46 7,899.58 987.45
12 66,364.69 5,530.39 11,060.12 921.68
16 84,417.76 5,276.11 14,067.57 879.22
24 114,922.49 4,788.44 19,154.26 798.09
24 + 12HT 128,741.27 — 21,453.34 —
24 + 24HT 139,470.29 — 23,243.69 —


This approach offers no boundary adaptation. Back To Top

turing (core2-himem)

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,516,843 926,811 2.58 28 25.97 21.25 1.62 3.24 809.79 864.45 1.32
2 5,550,102 932,383 2.59 26 13.35 11.28 1.59 2.98 413.39 445.19 1.32
4 5,557,347 933,669 2.58 28 7.16 6.30 1.53 2.99 219.60 240.16 1.43
8 5,562,016 934,415 2.61 27 3.64 3.42 1.54 2.96 109.37 123.56 1.47
12 5,558,551 933,711 2.59 26 2.80 3.40 2.08 3.39 75.13 89.40 1.58
16 5,560,912 934,132 2.58 29 2.36 3.40 2.08 3.91 59.41 73.74 1.51
24 5,555,808 933,354 2.61 26 2.23 4.00 2.07 4.15 38.49 53.56 1.81
32 5,558,347 933,748 2.58 25 1.86 3.80 2.13 4.22 28.29 42.89 2.10


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,709.58 5,709.58 951.48 951.48
2 11,161.32 5,580.66 1,860.06 930.03
4 20,720.15 5,180.04 3,453.38 863.35
8 40,310.94 5,038.87 6,718.27 839.78
12 55,675.11 4,639.59 9,277.47 773.12
16 67,530.73 4,220.67 11,253.42 703.34
24 92,879.24 3,869.97 15,478.88 644.95
32 116,044.56 3,626.39 19,338.84 604.34

bridges.psc

All time data are given in seconds.(brdiges.psc)
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 5,566,176 935,010 2.07 27 3.67 3.42 2.06 3.79 105.89 120.91 2.72


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 41,228.85 1,717.87 6,870.44 286.27

This approach offers no boundary adaptation. Back To Top

Notes 2

  • Speed 35.1K test/sec (40.6K tets/sec w/o body adaptation) and 5.9K ps/sec. The mesh is about 15.2 times larger in terms of test and the speed is 1.56 larger!
  • In speed table of volume reconstruction initial number of tetrahedra and vertices is subtracted.
  • In speed table of with surface adaptation MMGS time is not included.
  • Target is 1M vertices.
  • Quality comparison starting from surface vs volume
  • in some cases boundary recovery reached a point of non-developed code. Rerunning created a different surface mesh and CDT3D was able to continue.
  • commit fb0f106c6b65107
  • common parameters --met=1 --mrecm2=1 --mrec4=1 --malign=0 --mpp=2 --qstatr=1 --outmesh=${NAME}.meshb --ndfs=0 --nthreads=$THREADS --nbuckets=20*NTHREADS --nthreadsaf=20*NTHREADS --nbucketsaf=20*NTHREADS --blockmul=500000 --nqual=0 --nminpts=30 --ordt=1 --pardeact=1 --mdse=0 --cdfn=0.55
  • non-common parameters
    • reconstruct --meval=1 --reconstruct=1
    • surf adapt --brec=1 --meval=3 --nadapt=2
  • same experiment * using Surface adaptation

Back To Top


Specifications

hostname nodes memory (GB) cache (MB) Model Name Turbo Cores per node Threads per core sockets Cores per socket NUMA-nodes
coreV2-23-himem-* 4 757 16 Intel® Xeon® E5-4610 v2 @ 2.30GHz 2.70GHz 32 1 4 8 [0-7],[8-15],[16-23],[24-31]
crtc-gpu 1 757 30 Intel® Xeon® E5-2697 v2 @ 2.70GHz 3.50GHz 24 2 2 12 [0,2,..46],[1,3..47]
bridges.psc (LM nodes) 1 3TB 40 Intel® Xeon® CPU E7-8860 v3 @ 2.20GHz 3.20GHz 64 1 4 16 [0-7,32-39],[8-15,40-47],[16-23,48-55],[24-31,56-63]


Back To Top

10M Complexity (~120M Tetrahedra)

turing (core2-himem)

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 109,348,728 18,274,996 2.66 41 577.51 493.65 38.33 88.46 19,931.90 21,132.50 23.20
2 109,711,517 18,336,151 2.63 40 306.91 270.48 37.35 86.05 10,751.80 11,455.20 23.99
4 109,753,007 18,342,871 2.62 39 160.32 143.95 36.19 83.39 5,561.60 5,988.07 24.66
8 109,806,497 18,351,711 2.6 44 85.21 82.94 34.75 88.4 2,883.09 3,177.01 24.74
12 109,856,184 18,360,261 2.6 47 70.37 82.36 49.08 113.46 2,173.66 2,491.54 25.11
16 109,832,107 18,356,023 2.65 48 56.34 81.76 49.65 126.22 1,670.84 1,987.46 25.08
24 109,848,657 18,358,545 2.63 46 48.75 87.93 54.16 139.97 1,119.65 1,453.10 25.26
32 109,869,072 18,362,358 2.64 50 41.74 89.09 53.57 156.57 851.18 1,194.79 25.37


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,146.93 5,146.93 859.85 859.85
2 9,526.71 4,763.35 1,591.58 795.79
4 18,231.55 4,557.89 3,045.82 761.45
8 34,379.90 4,297.49 5,743.58 717.95
12 43,858.41 3,654.87 7,327.18 610.60
16 54,970.12 3,435.63 9,183.44 573.96
24 75,196.11 3,133.17 12,562.27 523.43
32 91,470.36 2,858.45 15,281.39 477.54

crtc-gpu

Updated on 2018.05.20

Size of initial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 109,884,436 18,365,004 2.41 102 30.51 77.69 39.25 246.50 883.71 1280.07 25.91
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 85,388.49 — 14,265.39 —

Back To Top

100M Complexity (~1.2B Tetrahedra)

Starting from a 50K Complexity volume mesh

Updated on 2018.05.20

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 1,096,524,832 183,097,400 2.90 236 509.81 950.33 389.19 4.731.73 14,991.60 21,575.50 247.17
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 50,795.75 1,058.24 8,481.52 176.70


NOT USED in paper

1M Complexity (~12M Tetrahedra)

Updated on 2018.05.29

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

crtc-gpu

All time data are given in seconds.(crtc-gpu)
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,122,131 1,864,227 2.38 28 2.53 5.95 3.48 7.73 69.20 91.29 3.15
24 +12HT 11,125,744 1,864,711 2.27 30 2.23 6.75 3.62 7.94 62.07 84.89 3.26
24 + 24HT 11,130,150 1,865,647 2.15 29 1.94 6.88 3.53 8.25 52.12 74.88 3.53
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 115,466.48 4,811.10 19,278.37 803.27
24+ 12HT 124,214.25 — 20,737.50 —
24 + 24HT 140,878.13 — 23,522.20 —

turing coreV2-himem

All time data are given in seconds ( turing coreV2-himem).
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,119,089. 1,863,716 2.64 28 4.59 7.26 4.58 9.93 83.8 112.8 3.13
32 11,114,943 1,862,960 2.60 29 3.78 7.53 4.63 10.47 62.41 91.44 2.95


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 93,421.04 3,892.54 15,597.62 649.90
32 115,198.46 3,599.95 19,232.88 601.03

bridges.psc

All time data are given in seconds ( bridges.psc).
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,128,838 1,865,236 2.05 26 6.58 6.95 4.43 8.78 202.82 231.6 2.91


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 45,542.50 1,897.60 7,603.33 316.81

Back To Top

Retrieved from "https://crtc.cs.odu.edu/index.php?title=Internal:ACDT3D_Scitech2018_strong_scaling_(Delta_Wing)&oldid=1582"