2018 10 16 10:25-12:10

2018 10 16 10:25-12:10

1. 9 25

2. 10 2

l

3. 10 9

l

4. 10 16

l

5. 10 23

l 2

6. 10 30

l -

l

8. 11 20

l -

9. 11 27

l

10. 12 4

l l

11. 12 11

l

12. 12 18 ??

l

13. 1 8

l RB-H

• MPI

1.

2.

3.

4.

5.

6.

1 1

1

1

• →

100

l

8 9

0 1 2 3 4 6 7

( )

10

6

0 2

10

6

0 2

32 /

112 2

1M /2

32 /

112 2

1M /2

(L3: Cache )

64 65 66 67

…

2 2 2 2

…

Intel OmniPath Architecture

12.

…

(L3: Cache )

64 65 66 67

…

2 2 2 2

(L3: Cache )

64 65 66 67

…

2 2 2 2

(L3: Cache )

64 65 66 67

…

2 2 2 2

(L3: Cache )

64 65 66 67

…

2 2 2 2

(L3: Cache )

64 65 66 67

…

2 2 2 2

(L3: Cache )

64 65 66 67

…

2 2 2 2

• Knights Landing Overview 1 1

Chip: 36 Tiles interconnected by 2D Mesh Tile: 2 Cores + 2 VPU/core + 1 MB L2

Memory: MCDRAM: 16 GB on-package; High BW DDR4: 6 channels @ 2400 up to 384GB IO: 36 lanes PCIe Gen3. 4 lanes of DMI for chipset Node: 1-Socket only

Fabric: Omni-Path on-package (not shown)

Vector Peak Perf: 3+TF DP and 6+TF SP Flops Scalar Perf: ~3x over Knights Corner

Streams Triad (GB/s): MCDRAM : 400+; DDR: 90+

4

Core L2 Core

Package

Source Intel: All products, computer systems, dates and figures specified are preliminary based on current expectations, and are subject to change without notice. KNL data are preliminary based on current expectations and are subject to change without notice. 1Binary Compatible with Intel Xeon processors using Haswell Instruction Set (except TSX). ²Bandwidth numbers are based on STREAM-like memory access pattern when MCDRAM used as flat memory. Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.

Omni-path not shown

EDC EDC

PCIe Gen 3

EDC ^EDC

Tile

DDR MC DDR MC

EDC EDC misc ^{EDC EDC}

36 Tiles connected by

2D Mesh Interconnect

MCDRAM MCDRAM MCDRAM MCDRAM

3 D D R 4 C H A N N E L S

3 D D R 4 C H A N N E L S

MCDRAM MCDRAM MCDRAM MCDRAM

D M

I 2 x16

1 x4

X4 DMI

HotChips27 KNL

Potential future options subject to change without notice.

All timeframes, features, products and dates are preliminary forecasts and subject to change without further notification.

Three products

KNL Self-Boot KNL Self-Boot w/ Fabric KNL Card (Baseline) (Fabric Integrated) (PCIe-Card)

2 VPU 2 VPU

Core 1MB Core L2

MCDRAM: 490GB/

DDR4: 115.2 GB/

=(8Byte 2400MHz 6 channel)

( ) MCDRAM:

16GB

+ DDR4

+ 8 A +56 + = 5 A ? 6A = =

. D

2 18 A

2 38 A )

+56

4 3 ( 3+

3+

l (

•

•

• 0.4

•

•

•

•

• 5

•

•

0.4 0.4 0.4 0.4 0.4

•

• 2 1

• 2.4 2

• 2.8 3

• 3.2 4

• 3.4 5

• 3.8 6

• 0.63

•

0.4

•

1.

•

• 1.

2.

2.

•

• 1.

2.

•

• -

for (j=0; j<n; j++) for (i=0; i<n; i++) {

y[j] += A[j][i] * x[i] ; }

•

A[0][0] x[0] A[0][0]*

x[0] y[0]

A[0][1] x[1] A[0][0]*

x[1] y[0]

A[0][2] x[2]

•

•

A[0][0] x[0] A[0][0]*

x[0] y[0]

A[0][1] x[1] A[0][0]*

x[1] y[0]

A[0][2] x[2] A[0][2]*

x[2] y[0]

A[0][3] x[3] A[0][3]*

x[3] y[0]

A[0][4] x[4] A[0][2]*

x[4] y[0]

… l

N

→N

•

•

•

1.

→CPU CPU→

i, j

2.

3.

•

1.

2.

•

• Intel Pentium4

•

•

• CPU CPU

Xeon Phi: KNC 7 ,

• Broadwell 14-19 (?)

• KNL 14

• 32

• AVX512 8

16

• AVX512

• 16 2 AVX512 32

• 1.4

•

1.4 GHz* 32 = 44.8 GFLOPS /

• 68

44.8 * 68 = 3046.6 GFLOPS /

= 3.04 TFLOPS /

•

1.

2.

•

•

•

•

•

•

•

l k- 2 (n 2 )

for (i=0; i<n; i++) for (j=0; j<n; j++)

for (k=0; k<n; k+=2)

C[i][j] += A[i][k] *B[k][j] + A[i][k+ ]*B[k+ ][j];

Ø k- /2

l j- 2 (n 2 )

for (i=0; i<n; i++) for (j=0; j<n; j+=2)

for (k=0; k<n; k++) {

C[i][j ] += A[i][k] *B[k][j ];

C[i][j+ ] += A[i][k] *B[k][j+ ];

}

Ø A[i][k]

l i- 2 (n 2 )

for (i=0; i<n; i+=2) for (j=0; j<n; j++)

for (k=0; k<n; k++) {

C[i ][j] += A[i ][k] *B[k][j];

C[i+ ][j] += A[i+ ][k] *B[k][j];

}

Ø B[i][j]

l i- j- 2

(n )

for (i=0; i<n; i+=2) for (j=0; j<n; j+=2)

for (k=0; k<n; k++) {

C[i ][j ] += A[i ][k] *B[k][j ];

C[i ][j+ ] += A[i ][k] *B[k][j+ ];

C[i+ ][j ] += A[i+ ][k] *B[k][j ];

C[i+ ][j+ ] += A[i+ ][k] *B[k][j + ];

}

Ø A[i][j], A[i+ ][k],B[k][j],B[k][j+ ]

l

for (i=0; i<n; i+=2) for (j=0; j<n; j+=2) {

dc00 = C[i ][j ]; dc01 = C[i ][j+ ];

dc10 = C[i+ ][j ]; dc11 = C[i+ ][j+ ] ; for (k=0; k<n; k++) {

da0= A[i ][k] ; da1= A[i+ ][k] ; db0= B[k][j ]; db1= B[k][j+ ];

dc00 += da0 *db0; dc01 += da0 *db1;

dc10 += da1 *db0; dc11 += da1 *db1;

} C[i ][j ] = dc00; C[i ][j+ ] = dc01;

C[i+ ][j ] = dc10; C[i+ ][j+ ] = dc11;

}

l k- 2 (n 2 ) do i= , n

do j= , n

do k= , n, 2

C(i, j) = C(i, j) +A(i, k) *B(k, j) &

+ A(i, k+ )*B(k+ , j) enddo

enddo enddo

Ø k- /2

l j- 2 (n 2 )

do i= , n

do j= , n, 2 do k= , n

C(i, j ) = C(i, j ) +A(i, k) * B(k, j ) C(i, j+ ) = C(i, j+ ) +A(i, k) * B(k, j+ ) enddo

enddo enddo

Ø A(i, k)

l i- 2 (n 2 )

do i= , n, 2 do j= , n

do k= , n

C(i , j) = C(i , j) +A(i , k) * B(k , j) C(i+ , j) = C(i+ , j) +A(i+ , k) * B(k , j) enddo

enddo enddo

Ø B(i, j)

l i- j- 2

(n )

do i= , n, 2 do j= , n, 2

do k= , n

C(i , j ) = C(i , j ) +A(i , k) *B(k, j ) C(i , j+ ) = C(i , j+ ) +A(i , k) *B(k, j+ ) C(i+ , j ) = C(i+ , j ) +A(i+ , k) *B(k, j ) C(i+ , j+ )=C(i+ , j+ ) +A(i+ , k) *B(k, j + ) enddo; enddo; enddo;

Ø A(i,j), A(i+ ,k),B(k,j),B(k,j+ )

l

do i= , n, 2 do j= , n, 2

dc00 = C(i ,j ); dc01 = C(i ,j+ ) dc10 = C(i+ ,j ); dc11 = C(i+ ,j+ )

do k= , n

da0= A(i ,k); da1= A(i+ , k) db0= B(k ,j ); db1= B(k, j+ )

dc00 = dc00+da0 *db0; dc01 = dc01+da0 *db1;

dc10 = dc10+da1 *db0; dc11 = dc11+da1 *db1;

enddo

C(i , j ) = dc00; C(i , j+ ) = dc01

C(i+ , j ) = dc10; C(i+ , j+ ) = dc11

enddo; enddo

•

i j

}

i, j

1 5 9 13

2 6 10 14

3 7 11 15

4 8 12 16

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

i

j

i

j

•

•

•

•

•

•

•

… …

•

•

•

•

i

…

…

2.

•

• →

•

•

•

•

•

•

•

•

•

•

•

•

•

•

2

•

1.

• (&A)++

• 2.

3.

• Sparc64 Iv L1 2Way

• OpenMP

• KNL cache L3 Direct Mapping(=1 Way)

• OpenMP Static

Dynamic(Cyclic) L2

L1

• !$OMP DO SCHEDULE(static,1)

•

•

SACSIS2012

http://sacsis.hpcc.jp/2012/files/SACSIS2012-tutorial1-pub.pdf

-

• emacs emacs

• ^x ^s control

• ^x ^c

^z

• ^g :

• ^k :

• ^y : ^k

• ^s :

• ^M x goto-line :

• rm

• rm *~ : test.c~ ~

*~

• ls :

• cd

• cd .. :

• cd ~

• cat

• make : Makefile

• make clean :

clean Makefile

• less (cat

• : 1

• / :

• q

• C Fortran

Mat-Mat-noopt-ofp.tar.gz

• mat-mat-noopt.bash

lecture-flat lecture7-flat

gt00 gt17 pjsub

• lecture-flat :

• lecture7-flat:

• $ cd /work/gt17/t170xx

$ cp /work/gt17/z30105/Mat-Mat-noopt-ofp.tar.gz ./

$ tar xvfz Mat-Mat-noopt-ofp.tar.gz

$ cd Mat-Mat-noopt

•

$ cd C : C

$ cd F :

• $ make

• $ pjsub mat-mat-noopt.bash

• $ cat mat-mat-noopt.bash.oXXXX (XXXX )

• (C )

N = 512

Mat-Mat time = 12.511196 [sec.]

21.455619 [MFLOPS]

OK!

N = 512

Mat-Mat time = 13.501827 [sec.]

19.881417 [MFLOPS]

OK!

DDR4

MCDRAM

• (Fortran )

N = 512

Mat-Mat time[sec.] = 24.4274609088898 MFLOPS = 10.9890854527813

OK!

N = 512

Mat-Mat time[sec.] = 27.0449259281158 MFLOPS = 9.92553856630092

OK!

DDR4

MCDRAM

• (DDR4 )

• mpiexec.hydra -n ${PJM_MPI_PROC} ./mat-mat-noopt

• MCDRAM

• mpiexec.hydra -n ${PJM_MPI_PROC} numactl -m 1

./mat-mat-noopt

• #define N 512

• #define DEBUG -

• MyMatMat

• N N

•

integer, parameter :: NN=512

• MyMatMat

•

•

•

1. [L ] -

2. [L ] - i, j, k

•L00:

•L10

•L20

•L30

•L40

•L50

L

1.

,

2. Kevin Dowd

C

,