parallel processing - How to use pmap on a single large Matrix -

i have 1 large matrix m (around 5 gig) , have perform operation f: column -> column on every column of m. suppose should use pmap (correct me if wrong), understand should give list of matrices. how process m in order pass pmap?

the second question if preferable f can take multiple columns @ once or not.

i think might idea try sharedarray this. better multithreading instead of julia's current multiprocessing, isn't released yet.

f should take reference matrix, , list of columns, rather columns themselves, avoid copying.

edit: here attempt @ sharedarray example - i've never used myself before, written poorly.

addprocs(3)  @everywhere rows = 10000 @everywhere cols = 100 data = sharedarray(float64, (rows,cols))  @everywhere function f(col, data)     row = 1:rows         new_val = rand()*col         dowork = 1:10000             new_val = sqrt(new_val)^2         end         data[row,col] = new_val     end end  tic() pmap(g->f(g...), [(col,data) col in 1:cols]) toc()  = 1:10:cols     println(i, " ", mean(data[:,i]), " ", 0.5*i) end  tic() map(g->f(g...), [(col,data) col in 1:cols]) toc() 

with output

elapsed time: 24.454875168 seconds 1 0.49883655930753457 0.5 11 5.480063271913496 5.5 21 10.495998948926 10.5 31 15.480227440365235 15.5 41 20.70105670567518 20.5 51 25.300540822213783 25.5 61 30.427728439076436 30.5 71 35.5280001975307 35.5 81 41.06101008798742 40.5 91 45.72394376323945 45.5 elapsed time: 69.651211534 seconds 

so getting approximately 3x speedup, hoped for. it'll approach ideal closer longer jobs runs, there jit warmup time.