numpy - How to do elementwise processing (first int, then pairwise absolute length) to avoid memory problems? (python) -
i want process big list of uint numbers (test1) , chunks of "length". need them signed int , need absolute length each pait of , odd values in list.
but want rid of 2 problems:
- it uses a lot of ram
- it takes ages!
so how make faster? trick? use numpy, no problem in doing so.
thanks in advance!
test2 = -127 + test1[i:i+length*2048000*2 + 2048000*2*1] test3 = (test2[::2]**2 + test2[1::2]**2)**0.5
an efficient way try use numpy functions, e.g:
n = 10 ff = np.random.randint(0, 255, n) # generate data ff2 = ff.reshape(n/2, 2) # new view on ff (only makes copy if needed) l_ff = np.linalg.norm(ff2, axis=1) # calculate vector length of each row note when modifying entry in ff2 ff change , vice versa.
internally, numpy stores data contiguous memory blocks. there further methods besides np.reshape() exploit structure. efficient conversion of data types, can try:
dd_s = np.arange(-5, 10, dtype=np.int8) dd_u = dd_s.astype(np.uint8) # conversion signed unsigned
Comments
Post a Comment