numpy - How to do elementwise processing (first int, then pairwise absolute length) to avoid memory problems? (python) -


i want process big list of uint numbers (test1) , chunks of "length". need them signed int , need absolute length each pait of , odd values in list.

but want rid of 2 problems:

  1. it uses a lot of ram
  2. it takes ages!

so how make faster? trick? use numpy, no problem in doing so.

thanks in advance!

test2 = -127 + test1[i:i+length*2048000*2 + 2048000*2*1] test3 = (test2[::2]**2 + test2[1::2]**2)**0.5 

an efficient way try use numpy functions, e.g:

n = 10 ff = np.random.randint(0, 255, n)  # generate data  ff2 = ff.reshape(n/2, 2)  # new view on ff (only makes copy if needed) l_ff = np.linalg.norm(ff2, axis=1)  # calculate vector length of each row 

note when modifying entry in ff2 ff change , vice versa.

internally, numpy stores data contiguous memory blocks. there further methods besides np.reshape() exploit structure. efficient conversion of data types, can try:

dd_s = np.arange(-5, 10, dtype=np.int8) dd_u = dd_s.astype(np.uint8) # conversion signed unsigned 

Comments

Popular posts from this blog

get url and add instance to a model with prefilled foreign key :django admin -

android - Keyboard hides my half of edit-text and button below it even in scroll view -

css - Make div keyboard-scrollable in jQuery Mobile? -