Intel Xeon Phi and High Bandwidth Memory - Revision history

Pthomadakis: /* Resources */

2018-06-11T16:34:36Z

‎Resources

2018-06-11T16:32:43Z

2018-06-01T18:26:24Z

‎KNL Mesh Interconnect

2018-06-01T17:26:47Z

‎KNL Mesh Interconnect

2018-06-01T17:15:36Z

‎KNL Mesh Interconnect

2018-06-01T17:15:22Z

‎KNL Mesh Interconnect

2018-06-01T17:14:57Z

‎KNL Mesh Interconnect

2018-06-01T16:58:55Z

‎KNL Mesh Interconnect

2018-06-01T16:57:28Z

‎Intel Xeon Phi

2018-06-01T16:37:55Z

‎Accessing MCDRAM in Flat Mode

← Older revision		Revision as of 16:34, 11 June 2018
Line 101:		Line 101:
	*[https://arxiv.org/pdf/1704.08273.pdf Exploring the Performance Benefit of Hybrid Memory System on HPC Environments (Paper)]		*[https://arxiv.org/pdf/1704.08273.pdf Exploring the Performance Benefit of Hybrid Memory System on HPC Environments (Paper)]
	*[http://pages.cs.wisc.edu/~david/courses/cs758/Fall2016/handouts/restricted/Knights-landing.pdf KNL Paper]		*[http://pages.cs.wisc.edu/~david/courses/cs758/Fall2016/handouts/restricted/Knights-landing.pdf KNL Paper]
		+	*[https://software.intel.com/en-us/articles/mcdram-high-bandwidth-memory-on-knights-landing-analysis-methods-tools MCDRAM]

← Older revision		Revision as of 16:32, 11 June 2018
Line 97:		Line 97:

	<div align="center">'''The numbers in the paths represent the following: (1) L2 cache miss (2) Directrory Access (3) Memory Access (4) Data return'''</div>		<div align="center">'''The numbers in the paths represent the following: (1) L2 cache miss (2) Directrory Access (3) Memory Access (4) Data return'''</div>
		+
		+	== Resources ==
		+	*[https://arxiv.org/pdf/1704.08273.pdf Exploring the Performance Benefit of Hybrid Memory System on HPC Environments (Paper)]
		+	*[http://pages.cs.wisc.edu/~david/courses/cs758/Fall2016/handouts/restricted/Knights-landing.pdf KNL Paper]

@@ Line 86: / Line 86: @@
 === Sub-NUMA Clustering Mode ===
 In this mode, each quadrant is exposed as a separate NUMA domain to the operating system. Memory affinity can now be guaranteed between tile, directory, and memory. However, the application needs to be optimized for NUMA in order to get benefit. When optimized, it provides the lowest latency and highest bandwidth from all modes, by keeping the communication local to a single quadrant.
-{{multiple_image
-| align = left
-| direction = horizontal
-| image1 = mesh_all_to_all.png
-| image2 = mesh_quadrant.png
+<gallery class="center" mode=packed widths="480px" heights="320px" caption="The diagrams below show the path followed in order to retrive data in case of a cache miss">
-| image3 = mesh_subnuma.png
+File:mesh_all_to_all.png|'''(a) All-to-All Mode'''
-}}
+File:mesh_quadrant.png|'''(b) Quadrant Mode'''

← Older revision		Revision as of 17:26, 1 June 2018
Line 86:		Line 86:
	=== Sub-NUMA Clustering Mode ===		=== Sub-NUMA Clustering Mode ===
	In this mode, each quadrant is exposed as a separate NUMA domain to the operating system. Memory affinity can now be guaranteed between tile, directory, and memory. However, the application needs to be optimized for NUMA in order to get benefit. When optimized, it provides the lowest latency and highest bandwidth from all modes, by keeping the communication local to a single quadrant.		In this mode, each quadrant is exposed as a separate NUMA domain to the operating system. Memory affinity can now be guaranteed between tile, directory, and memory. However, the application needs to be optimized for NUMA in order to get benefit. When optimized, it provides the lowest latency and highest bandwidth from all modes, by keeping the communication local to a single quadrant.
		+	{{multiple_image
		+	\| align = left
		+	\| direction = horizontal
		+	\| image1 = mesh_all_to_all.png
		+	\| image2 = mesh_quadrant.png
		+	\| image3 = mesh_subnuma.png
		+	}}

@@ Line 75: / Line 75: @@
 == KNL Mesh Interconnect ==
-[[File:knl_mesh.png|4800px|right|thumb|'''KNL Mesh Interconnect''']]
+[[File:knl_mesh.png|480px|right|thumb|'''KNL Mesh Interconnect''']]
 KNL uses a mesh interconnect for the connection of the tiles containing the cores. The interconnect provides cache coherency using a distributed directory to filter snoops. It offers 3 clustering modes all-to-all, quadrant and sub-NUMA.

← Older revision		Revision as of 16:58, 1 June 2018
Line 75:		Line 75:

	== KNL Mesh Interconnect ==		== KNL Mesh Interconnect ==
		+	[[File:knl_mesh.png\|320px\|right\|thumb\|'''KNL Mesh Interconnect'''
	KNL uses a mesh interconnect for the connection of the tiles containing the cores. The interconnect provides cache coherency using a distributed directory to filter snoops. It offers 3 clustering modes all-to-all, quadrant and sub-NUMA.		KNL uses a mesh interconnect for the connection of the tiles containing the cores. The interconnect provides cache coherency using a distributed directory to filter snoops. It offers 3 clustering modes all-to-all, quadrant and sub-NUMA.

← Older revision		Revision as of 16:57, 1 June 2018
Line 73:		Line 73:
	=== Using memkind library ===		=== Using memkind library ===
	Memkind library is a user-extensible heap manager built on top of jemalloc, a C library for general-purpose memory allocation functions. The library is generalizable to any NUMA architecture, but for Knights Landing processors it is used primarily for manual allocation to HBM using special allocators for C/C++		Memkind library is a user-extensible heap manager built on top of jemalloc, a C library for general-purpose memory allocation functions. The library is generalizable to any NUMA architecture, but for Knights Landing processors it is used primarily for manual allocation to HBM using special allocators for C/C++
		+
		+	== KNL Mesh Interconnect ==
		+	KNL uses a mesh interconnect for the connection of the tiles containing the cores. The interconnect provides cache coherency using a distributed directory to filter snoops. It offers 3 clustering modes all-to-all, quadrant and sub-NUMA.

@@ Line 65: / Line 65: @@
 The memory is used in part as cache and the rest as a separate memory module. Has benefits and disadvantages of both. The portion that is assigned to its functionality is determined in boot time.
-=== Accessing MCDRAM in Flat Mode ===
+== Accessing MCDRAM in Flat Mode ==
 There are 2 options to control MCDRAM in flat mode with less or more control over individual allocations numactl and memkind library.
-==== Using numactl ====
+=== Using numactl ===
 One can use numactl to configure the application to reserve memory from the MCDRAM without changing the code itself. Works better when the whole application can fit in the MCDRAM, but can still be used for bigger applications, by moving allocations that do not fit in MCDRAM to DDR. Another mode is to interleave allocation between the memory nodes.
-==== Using memkind library ====
+=== Using memkind library ===
 Memkind library is a user-extensible heap manager built on top of jemalloc, a C library for general-purpose memory allocation functions. The library is generalizable to any NUMA architecture, but for Knights Landing processors it is used primarily for manual allocation to HBM using special allocators for C/C++