Internal:ACDT3D Scitech2018 strong scaling (Delta Wing): Difference between revisions

From crtc.cs.odu.edu
Jump to navigation Jump to search
← Older edit
Ctsolakis (talk | contribs)
BrainMapping, CNF, HPC, I2MConversion, NDAM, Segmentation, UltraSound3D, Bureaucrats, external, internal, Administrators
2,152 edits
Ctsolakis (talk | contribs)
BrainMapping, CNF, HPC, I2MConversion, NDAM, Segmentation, UltraSound3D, Bureaucrats, external, internal, Administrators
2,152 edits
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
<span id="Contents"></span>
<span id="Contents"></span>
= Specifications =
{| class="wikitable" style="text-align: center;
!width="11%"| hostname
!width="4%"| nodes
!width="7%"| memory (GB)
!width="7%"| cache (MB)
!width="20%"| Model Name
!width="4%"| Turbo
!width="8%"| Cores per node
!width="9%"| Threads per core
!width="5%"| sockets
!width="9%"| Cores per socket
!width="10%"| NUMA-nodes
|-
| coreV2-23-himem-*
| 4
| 757
| 16
| Intel® Xeon® E5-4610 v2 @ 2.30GHz
| 2.70GHz
| 32
| 1
| 4
| 8
| [0-7],[8-15],[16-23],[24-31]
|-
| crtc-gpu
| 1
| 757
| 30
| Intel® Xeon® E5-2697 v2 @ 2.70GHz
| 3.50GHz
| 24
| 2
| 2
| 12
| [0,2,..46],[1,3..47]
|-
| bridges.psc (LM nodes)
| 1
| 3TB
| 40
| Intel® Xeon® CPU E7-8860 v3 @ 2.20GHz
| 3.20GHz
| 64
| 1
| 4
| 16
| [0-7,32-39],[8-15,40-47],[16-23,48-55],[24-31,56-63]
|}
[[#Contents | Back To Top]]
= 500k Complexity (~6M Tetrahedra) =
= 500k Complexity (~6M Tetrahedra) =
==crtc.gpu ==
==crtc.gpu ==
Line 481: Line 536:
[[#Contents | Back To Top]]
[[#Contents | Back To Top]]


= Specifications =
{| class="wikitable" style="text-align: center;
!width="11%"| hostname
!width="4%"| nodes
!width="7%"| memory (GB)
!width="7%"| cache (MB)
!width="20%"| Model Name
!width="4%"| Turbo
!width="8%"| Cores per node
!width="9%"| Threads per core
!width="5%"| sockets
!width="9%"| Cores per socket
!width="10%"| NUMA-nodes
|-
| coreV2-23-himem-*
| 4
| 757
| 16
| Intel® Xeon® E5-4610 v2 @ 2.30GHz
| 2.70GHz
| 32
| 1
| 4
| 8
| [0-7],[8-15],[16-23],[24-31]
|-
| crtc-gpu
| 1
| 757
| 30
| Intel® Xeon® E5-2697 v2 @ 2.70GHz
| 3.50GHz
| 24
| 2
| 2
| 12
| [0,2,..46],[1,3..47]
|-
| bridges.psc (LM nodes)
| 1
| 3TB
| 40
| Intel® Xeon® CPU E7-8860 v3 @ 2.20GHz
| 3.20GHz
| 64
| 1
| 4
| 16
| [0-7,32-39],[8-15,40-47],[16-23,48-55],[24-31,56-63]
|}
[[#Contents | Back To Top]]


=10M Complexity (~120M Tetrahedra) =
=10M Complexity (~120M Tetrahedra) =
== turing (core2-himem) ==
== turing (core2-himem) ==
''' Updated on 2018.06.02'''
{| class="wikitable" style="text-align: right;"
{| class="wikitable" style="text-align: right;"
  |+ All time data are given in seconds.
  |+ All time data are given in seconds.
Line 745: Line 746:
! Partition
! Partition
! Par. Reconnection
! Par. Reconnection
|-
| 1
| 109,349,201
| 18,275,080
| 2.29
| 41
| 474.49
| 427.15
| 37.61
| 91.94
| 16,834.80
| 17,868.30
| 33.61
|-
| 2
| 109,696,641
| 18,333,599
| 2.2
| 42
| 280.01
| 266.33
| 45.26
| 110.70
| 9,478.53
| 10,183.00
| 30.16
|-
| 4
| 109,730,063
| 18,339,113
| 2.22
| 45
| 150.98
| 150.97
| 43.25
| 139.24
| 4,947.61
| 5,434.26
| 24.57
|-
| 8
| 109,824,486
| 18,355,407
| 2.32
| 50
| 82.52
| 98.17
| 41.74
| 146.66
| 2,632.98
| 3,004.40
| 24.83
|-
| 16
| 109,821,714
| 18,354,479
| 2.19
| 49
| 44.18
| 72.66
| 41.69
| 148.71
| 1,349.91
| 1,659.33
| 24.94
|-
| 24
| 109,831,473
| 1,8356,406
| 2.15
| 50
| 32.38
| 68.19
| 41.72
| 153.36
| 935.34
| 1,233.15
| 25.28
|-
| 24 + 12HT
| 109,855,934
| 18,360,514
| 2.34
| 47
| 25.88
| 67.41
| 38.82
| 144.35
| 747.55
| 1,026.35
| 25.58
|-
|-
| 24 + 24HT
| 24 + 24HT
| 109,884,436
| 109,828,538
| 18,365,004
| 18,356,104
| 2.41
| 2.22
| 102
| 70
| 30.51
| 26.63
| 77.69
| 75.4
| 39.25
| 40.09
| 246.50
| 192.84
| 883.71
| 769.09
| 1280.07
| 1,106.26
| 25.91
| 25.73
 
|}
|}


Line 768: Line 859:
! # Vertices/sec
! # Vertices/sec
! # Vertices/sec/core
! # Vertices/sec/core
|-
| 1
| 6,087.20
| 6,087.20
| 1,016.93
| 1,016.93
|-
| 2
| 10,715.45
| 5,357.73
| 1,790.17
| 895.08
|-
| 4
| 20,085.32
| 5,021.33
| 3,355.53
| 838.88
|-
| 8
| 36,361.10
| 4,545.14
| 6,074.79
| 759.35
|-
| 16
| 65,834.11
| 4,114.63
| 10,998.52
| 687.41
|-
| 24
| 88,594.48
| 3,691.44
| 14,801.20
| 616.72
|-
| 24 + 12HT
| 106,469.27
| &mdash;
| 17,787.51
| &mdash;
|-
|-
| 24 + 24HT
| 24 + 24HT
| 85,388.49
| 98,753.77
| &mdash;
| &mdash;
| 14,265.39
| 16,498.65
| &mdash;
| &mdash;
|}
|}
[[#Contents | Back To Top]]
[[#Contents | Back To Top]]


Latest revision as of 11:33, 24 October 2018

Specifications

hostname nodes memory (GB) cache (MB) Model Name Turbo Cores per node Threads per core sockets Cores per socket NUMA-nodes
coreV2-23-himem-* 4 757 16 Intel® Xeon® E5-4610 v2 @ 2.30GHz 2.70GHz 32 1 4 8 [0-7],[8-15],[16-23],[24-31]
crtc-gpu 1 757 30 Intel® Xeon® E5-2697 v2 @ 2.70GHz 3.50GHz 24 2 2 12 [0,2,..46],[1,3..47]
bridges.psc (LM nodes) 1 3TB 40 Intel® Xeon® CPU E7-8860 v3 @ 2.20GHz 3.20GHz 64 1 4 16 [0-7,32-39],[8-15,40-47],[16-23,48-55],[24-31,56-63]


Back To Top

500k Complexity (~6M Tetrahedra)

crtc.gpu

Updated on 2018.05.22

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,516,917 926,822 2.22 28 20.95 19.44 1.65 2.85 665.49 712.60 1.31
2 5,557,272 933,556 2.41 28 11.36 11.85 1.82 3.47 332.76 363.67 1.7
4 5,551,696 932,709 2.48 30 6.22 6.86 1.74 3.57 177.65 198.52 1.43
6 5,561,503 934,363 2.39 26 3.95 4.37 1.55 3.13 111.55 126.94 1.46
8 5,563,801 934,630 2.36 27 3.27 3.93 1.66 3.32 90.57 105.11 1.47
12 5,563,857 934,699 2.44 28 2.27 2.99 1.62 3.23 62.54 75.08 1.58
16 5,558,467 933,729 2.68 27 1.76 2.88 1.59 3.26 46.78 58.96 1.5
24 5,558,489 933,876 2.43 27 1.31 2.64 1.63 3.11 32.19 43.31 1.81
24 + 12HT 5,553,184 932,833 2.45 27 1.15 2.62 1.55 3.13 27.71 38.62 2.26
24 + 24HT 5,567,259 935,267 2.35 27 1.04 2.84 1.81 3.28 24.42 35.75 2.72
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,926.36 6,926.36 1,154.25 1,154.25
2 13,682.94 6,841.47 2,280.23 1,140.11
4 25,037.78 6,259.44 4,172.90 1,043.22
6 39,233.55 6,538.93 6,538.98 1,089.83
8 47,403.72 5,925.46 7,899.58 987.45
12 66,364.69 5,530.39 11,060.12 921.68
16 84,417.76 5,276.11 14,067.57 879.22
24 114,922.49 4,788.44 19,154.26 798.09
24 + 12HT 128,741.27 — 21,453.34 —
24 + 24HT 139,470.29 — 23,243.69 —


This approach offers no boundary adaptation. Back To Top

turing (core2-himem)

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 5,516,843 926,811 2.58 28 25.97 21.25 1.62 3.24 809.79 864.45 1.32
2 5,550,102 932,383 2.59 26 13.35 11.28 1.59 2.98 413.39 445.19 1.32
4 5,557,347 933,669 2.58 28 7.16 6.30 1.53 2.99 219.60 240.16 1.43
8 5,562,016 934,415 2.61 27 3.64 3.42 1.54 2.96 109.37 123.56 1.47
12 5,558,551 933,711 2.59 26 2.80 3.40 2.08 3.39 75.13 89.40 1.58
16 5,560,912 934,132 2.58 29 2.36 3.40 2.08 3.91 59.41 73.74 1.51
24 5,555,808 933,354 2.61 26 2.23 4.00 2.07 4.15 38.49 53.56 1.81
32 5,558,347 933,748 2.58 25 1.86 3.80 2.13 4.22 28.29 42.89 2.10


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,709.58 5,709.58 951.48 951.48
2 11,161.32 5,580.66 1,860.06 930.03
4 20,720.15 5,180.04 3,453.38 863.35
8 40,310.94 5,038.87 6,718.27 839.78
12 55,675.11 4,639.59 9,277.47 773.12
16 67,530.73 4,220.67 11,253.42 703.34
24 92,879.24 3,869.97 15,478.88 644.95
32 116,044.56 3,626.39 19,338.84 604.34

bridges.psc

All time data are given in seconds.(brdiges.psc)
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 5,566,176 935,010 2.07 27 3.67 3.42 2.06 3.79 105.89 120.91 2.72


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 41,228.85 1,717.87 6,870.44 286.27

This approach offers no boundary adaptation. Back To Top

Notes 2

  • Speed 35.1K test/sec (40.6K tets/sec w/o body adaptation) and 5.9K ps/sec. The mesh is about 15.2 times larger in terms of test and the speed is 1.56 larger!
  • In speed table of volume reconstruction initial number of tetrahedra and vertices is subtracted.
  • In speed table of with surface adaptation MMGS time is not included.
  • Target is 1M vertices.
  • Quality comparison starting from surface vs volume
  • in some cases boundary recovery reached a point of non-developed code. Rerunning created a different surface mesh and CDT3D was able to continue.
  • commit fb0f106c6b65107
  • common parameters --met=1 --mrecm2=1 --mrec4=1 --malign=0 --mpp=2 --qstatr=1 --outmesh=${NAME}.meshb --ndfs=0 --nthreads=$THREADS --nbuckets=20*NTHREADS --nthreadsaf=20*NTHREADS --nbucketsaf=20*NTHREADS --blockmul=500000 --nqual=0 --nminpts=30 --ordt=1 --pardeact=1 --mdse=0 --cdfn=0.55
  • non-common parameters
    • reconstruct --meval=1 --reconstruct=1
    • surf adapt --brec=1 --meval=3 --nadapt=2
  • same experiment * using Surface adaptation

Back To Top


10M Complexity (~120M Tetrahedra)

turing (core2-himem)

Updated on 2018.06.02

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 109,348,728 18,274,996 2.66 41 577.51 493.65 38.33 88.46 19,931.90 21,132.50 23.20
2 109,711,517 18,336,151 2.63 40 306.91 270.48 37.35 86.05 10,751.80 11,455.20 23.99
4 109,753,007 18,342,871 2.62 39 160.32 143.95 36.19 83.39 5,561.60 5,988.07 24.66
8 109,806,497 18,351,711 2.6 44 85.21 82.94 34.75 88.4 2,883.09 3,177.01 24.74
12 109,856,184 18,360,261 2.6 47 70.37 82.36 49.08 113.46 2,173.66 2,491.54 25.11
16 109,832,107 18,356,023 2.65 48 56.34 81.76 49.65 126.22 1,670.84 1,987.46 25.08
24 109,848,657 18,358,545 2.63 46 48.75 87.93 54.16 139.97 1,119.65 1,453.10 25.26
32 109,869,072 18,362,358 2.64 50 41.74 89.09 53.57 156.57 851.18 1,194.79 25.37


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 5,146.93 5,146.93 859.85 859.85
2 9,526.71 4,763.35 1,591.58 795.79
4 18,231.55 4,557.89 3,045.82 761.45
8 34,379.90 4,297.49 5,743.58 717.95
12 43,858.41 3,654.87 7,327.18 610.60
16 54,970.12 3,435.63 9,183.44 573.96
24 75,196.11 3,133.17 12,562.27 523.43
32 91,470.36 2,858.45 15,281.39 477.54

crtc-gpu

Updated on 2018.05.20

Size of initial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
1 109,349,201 18,275,080 2.29 41 474.49 427.15 37.61 91.94 16,834.80 17,868.30 33.61
2 109,696,641 18,333,599 2.2 42 280.01 266.33 45.26 110.70 9,478.53 10,183.00 30.16
4 109,730,063 18,339,113 2.22 45 150.98 150.97 43.25 139.24 4,947.61 5,434.26 24.57
8 109,824,486 18,355,407 2.32 50 82.52 98.17 41.74 146.66 2,632.98 3,004.40 24.83
16 109,821,714 18,354,479 2.19 49 44.18 72.66 41.69 148.71 1,349.91 1,659.33 24.94
24 109,831,473 1,8356,406 2.15 50 32.38 68.19 41.72 153.36 935.34 1,233.15 25.28
24 + 12HT 109,855,934 18,360,514 2.34 47 25.88 67.41 38.82 144.35 747.55 1,026.35 25.58
24 + 24HT 109,828,538 18,356,104 2.22 70 26.63 75.4 40.09 192.84 769.09 1,106.26 25.73
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
1 6,087.20 6,087.20 1,016.93 1,016.93
2 10,715.45 5,357.73 1,790.17 895.08
4 20,085.32 5,021.33 3,355.53 838.88
8 36,361.10 4,545.14 6,074.79 759.35
16 65,834.11 4,114.63 10,998.52 687.41
24 88,594.48 3,691.44 14,801.20 616.72
24 + 12HT 106,469.27 — 17,787.51 —
24 + 24HT 98,753.77 — 16,498.65 —

Back To Top

100M Complexity (~1.2B Tetrahedra)

Starting from a 50K Complexity volume mesh

Updated on 2018.05.20

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

All time data are given in seconds.
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 + 24HT 1,096,524,832 183,097,400 2.90 236 509.81 950.33 389.19 4.731.73 14,991.60 21,575.50 247.17
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 + 24HT 50,795.75 1,058.24 8,481.52 176.70


NOT USED in paper

1M Complexity (~12M Tetrahedra)

Updated on 2018.05.29

Size of intial mesh

Vertices : 104,305

Tetrahedra: 581,196

crtc-gpu

All time data are given in seconds.(crtc-gpu)
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,122,131 1,864,227 2.38 28 2.53 5.95 3.48 7.73 69.20 91.29 3.15
24 +12HT 11,125,744 1,864,711 2.27 30 2.23 6.75 3.62 7.94 62.07 84.89 3.26
24 + 24HT 11,130,150 1,865,647 2.15 29 1.94 6.88 3.53 8.25 52.12 74.88 3.53
Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 115,466.48 4,811.10 19,278.37 803.27
24+ 12HT 124,214.25 — 20,737.50 —
24 + 24HT 140,878.13 — 23,522.20 —

turing coreV2-himem

All time data are given in seconds ( turing coreV2-himem).
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,119,089. 1,863,716 2.64 28 4.59 7.26 4.58 9.93 83.8 112.8 3.13
32 11,114,943 1,862,960 2.60 29 3.78 7.53 4.63 10.47 62.41 91.44 2.95


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 93,421.04 3,892.54 15,597.62 649.90
32 115,198.46 3,599.95 19,232.88 601.03

bridges.psc

All time data are given in seconds ( bridges.psc).
# Threads # Tetrahedra # Vertices Reconstruct Mesh Mesh Refinement Total Time Memory (GB)
# Iterations Par. Deactivation Par. Vertex Creation Vertex Insertion Partition Par. Reconnection
24 11,128,838 1,865,236 2.05 26 6.58 6.95 4.43 8.78 202.82 231.6 2.91


Speed
# Threads # Tetrahedra/sec # Tetrahedra/sec/core # Vertices/sec # Vertices/sec/core
24 45,542.50 1,897.60 7,603.33 316.81

Back To Top

Retrieved from "https://crtc.cs.odu.edu/index.php?title=Internal:ACDT3D_Scitech2018_strong_scaling_(Delta_Wing)&oldid=2547"