Skip to content

Instantly share code, notes, and snippets.

@fac2003
Created July 11, 2016 14:18
Show Gist options
  • Save fac2003/b21556e327f66cd03ec2675de4341a12 to your computer and use it in GitHub Desktop.
Save fac2003/b21556e327f66cd03ec2675de4341a12 to your computer and use it in GitHub Desktop.
head -1000 GPU-benchmark-0.4.0-2.txt
o.n.n.NativeOps - Number of threads used for linear algebra 32
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [0]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [1]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [2]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [3]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [4]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [5]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [6]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.a.c.i.BasicContextPool - Creating new stream for thread: [1], device: [7]...
o.n.j.c.CudaAffinityManager - Single device is forced, mapping to device [0]
o.n.j.c.CudaAffinityManager - Manually mapping thread [27] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [28] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [29] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [30] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [31] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [32] to device [0], out of [8] devices...
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 4
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 2
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 1
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 0
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 5
o.n.j.h.i.CudaZeroHandler - Creating bucketID: 3
Using existing text file at /tmp/Shakespeare.txt
o.n.j.c.CudaAffinityManager - Mapping thread [120] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [121] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [144] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [148] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [125] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [136] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [149] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [126] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [123] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [146] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [138] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [141] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [124] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [147] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [132] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [134] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [145] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [135] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [139] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [133] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [128] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [129] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [137] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [131] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [119] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [143] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [130] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [127] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [142] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [140] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [118] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Mapping thread [122] to device [7], out of [8] devices...
o.d.s.i.p.ParameterAveragingTrainingMaster - Starting training of split 1 of 1. workerMiniBatchSize=8, averagingFreq=3, Configured for 32 workers
o.n.j.c.CudaAffinityManager - Manually mapping thread [414] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [413] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [417] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [416] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [415] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [412] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [418] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [421] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [420] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [419] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [422] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [423] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [424] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [425] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [426] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [427] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [428] to device [5], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [429] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [431] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [432] to device [0], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [430] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [433] to device [7], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [434] to device [2], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [435] to device [6], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [438] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [439] to device [4], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [440] to device [1], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [441] to device [3], out of [8] devices...
o.n.j.c.CudaAffinityManager - Manually mapping thread [442] to device [2], out of [8] devices...
Failed on [140223034093040] -> [35649840128], size: [307692], direction: [0], result: [77]
Failed on [140223369915152] -> [35863749632], size: [307692], direction: [0], result: [77]
Failed on [140222429192528] -> [35627512320], size: [307692], direction: [0], result: [77]
o.a.s.e.Executor - Exception in task 27.0 in stage 4.0 (TID 155)
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalStateException: MemcpyAsync H2H failed: [140223034093040] -> [35649840128]
at org.nd4j.linalg.api.ndarray.BaseNDArrayProxy.readObject(BaseNDArrayProxy.java:68) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) ~[na:1.8.0_91]
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: MemcpyAsync H2H failed: [140223034093040] -> [35649840128]
at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.read(BaseCudaDataBuffer.java:737) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.api.buffer.BaseDataBuffer.read(BaseDataBuffer.java:1129) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.api.ndarray.BaseNDArrayProxy.read(BaseNDArrayProxy.java:75) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.api.ndarray.BaseNDArrayProxy.readObject(BaseNDArrayProxy.java:66) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source) ~[na:na]
Caused by: java.lang.IllegalStateException: MemcpyAsync H2H failed: [140223034093040] -> [35649840128]
at org.nd4j.jita.handler.impl.CudaZeroHandler.memcpyAsync(CudaZeroHandler.java:510) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.allocator.impl.AtomicAllocator.memcpyAsync(AtomicAllocator.java:774) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.set(BaseCudaDataBuffer.java:324) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.setData(BaseCudaDataBuffer.java:371) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.read(BaseCudaDataBuffer.java:728) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.s.TaskSetManager - Task 27 in stage 4.0 failed 1 times; aborting job
Failed on [140223704424800] -> [35766232064], size: [307692], direction: [0], result: [77]
Failed on [140223436538976] -> [35776410112], size: [307692], direction: [0], result: [77]
Failed on [140224307442384] -> [35769377792], size: [307692], direction: [0], result: [77]
Failed on [140223167836352] -> [35735207936], size: [307692], direction: [0], result: [77]
Failed on [140224375761200] -> [35897737216], size: [307692], direction: [0], result: [77]
Failed on [35981361152] -> [180699398144], size: [2239508], direction: [1], result: [77]
o.a.s.e.Executor - Exception in task 3.0 in stage 4.0 (TID 131)
java.lang.IllegalStateException: MemcpyAsync relocate H2D failed: [35981361152] -> [180699398144]
at org.nd4j.jita.handler.impl.CudaZeroHandler.relocate(CudaZeroHandler.java:366) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.getDevicePointer(CudaZeroHandler.java:733) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.allocator.impl.AtomicAllocator.getPointer(AtomicAllocator.java:256) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.jcublas.ops.executioner.JCudaExecutioner.invoke(JCudaExecutioner.java:1001) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.linalg.jcublas.ops.executioner.JCudaExecutioner.exec(JCudaExecutioner.java:552) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 25.0 in stage 4.0 (TID 153)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 118
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 1.0 in stage 4.0 (TID 129)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 145
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 13.0 in stage 4.0 (TID 141)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 123
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 4.0 in stage 4.0 (TID 132)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 149
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.s.TaskSchedulerImpl - Exception in statusUpdate
java.util.concurrent.RejectedExecutionException: Task org.apache.spark.scheduler.TaskResultGetter$$anon$3@7e2a786 rejected from java.util.concurrent.ThreadPoolExecutor@56cc9f29[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 135]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) ~[na:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) ~[na:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) ~[na:1.8.0_91]
at org.apache.spark.scheduler.TaskResultGetter.enqueueFailedTask(TaskResultGetter.scala:104) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.apache.spark.scheduler.TaskSchedulerImpl.liftedTree2$1(TaskSchedulerImpl.scala:350) [dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 9.0 in stage 4.0 (TID 137)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 127
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 26.0 in stage 4.0 (TID 154)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 132
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
o.a.s.e.Executor - Exception in task 2.0 in stage 4.0 (TID 130)
java.lang.RuntimeException: Can't allocate [HOST] memory: 2239508; threadId: 148
at org.nd4j.jita.memory.impl.CudaDirectProvider.malloc(CudaDirectProvider.java:56) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaCachingZeroProvider.malloc(CudaCachingZeroProvider.java:113) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.memory.impl.CudaFullCachingProvider.malloc(CudaFullCachingProvider.java:77) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:218) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
at org.nd4j.jita.handler.impl.CudaZeroHandler.alloc(CudaZeroHandler.java:239) ~[dl4j-spark-cdh5-examples-1.0-SNAPSHOT.jar:na]
Failed on [140224106755440] -> [35841114112], size: [307692], direction: [0], result: [77]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment