Skip to content

Instantly share code, notes, and snippets.

@astrofrog

astrofrog/da_block_slow.py

Created February 12, 2020 13:29
Show Gist options
  • Save astrofrog/d833250615bf45104f3f67f7920a5c7f to your computer and use it in GitHub Desktop.
Save astrofrog/d833250615bf45104f3f67f7920a5c7f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
da_block_slow.py
import time
import uuid
import numpy as np
import dask.array as da
class ArrayLikeObject:
def __init__(self):
self._array = np.ones((20, 30), dtype=float)
def __getitem__(self, item):
return self._array[item]
@property
def shape(self):
return self._array.shape
@property
def ndim(self):
return self._array.ndim
@property
def size(self):
return self._array.size
@property
def dtype(self):
return self._array.dtype
meta = np.zeros((0,), dtype=float)
start = time.time()
chunks = [[[[da.from_array(ArrayLikeObject(), name=str(uuid.uuid4()), meta=meta)] * 269] * 6] * 4]
print(time.time() - start)
data = da.block(chunks)
print(time.time() - start)
print(data)
@Cadair
Copy link

Cadair commented Feb 12, 2020

When I run this I get:

dask.array<concatenate, shape=(1, 4, 120, 8070), dtype=float64, chunksize=(1, 1, 20, 30), chunktype=numpy.ndarray>

the shape is wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment