javascript - Optimize comparing huge amounts of data with itself in NodeJS -


i have big multidimensional object in nodejs, 50 mb worth of json. contains variations of biological data. think can sufficiently simplify so:

{     lads : {         // lad         lad4515643 : {             brains  : {                 // brain                 brain1256251 : {                     var01 : 'lala',                     var02 : 'jaja',                     var99 : 'haha',                 },                 // brain                 brain3567432 : {},                 brain4867321 : {},                 brain5145621 : {} // etc             },             var01 : 'foo',             var02 : 'bar',             var99 : 'baz'         },         // lad         lad4555672 : {},         lad5625627 : {},         lad7457255 : {} // etc     } } 

i need compare combinations of lads brains lads brains see ones "better", in order make kind of hierarchy. parent lads keys weigh in on brains comparison.

i figured, using iterations on objects using references can assign ids of better ones. take quick glance on code (comments) , see mean:

// iterate on lads (var ladid in obj.lads) {     if (obj.lads.hasownproperty(ladid)) {         var lad = obj.lads[ladid];          // iterate on brains         (var brainid in lad.brains) {             if (lad.brains.hasownproperty(brainid)) {                 var brain = lad.brains[brainid];                  // iterate on lads again                 (var lad2id in obj.lads) {                     if (obj.lads.hasownproperty(lad2id)) {                         var lad2 = obj.lads[lad2id];                          // iterate on lads' brains                         (var brain2id in lad2.brains) {                             if (lad2.brains.hasownproperty(brain2id)) {                                 var brain2 = lad2.brains[brain2id];                                  // 1 lad+brain combination                                 var drone1 = {                                     lad : lad,                                     brain : brain                                 };                                  // lad+brain combination                                 var drone2 = {                                     lad : lad2,                                     brain : brain2                                     ladid : lad2id, // required store reference if better                                     brainid : brain2id // required store reference if better                                 };                                  // comparison unless comparing ourselves                                 if (brain != brain2) {                                     // objects passed reference, convenient:                                     judge(drone1, drone2);                                 }                             }                         }                     }                 }             }         }     } }  // judge better function judge(drone1, drone2) {     // magic compares lad+brain combos     if (magic) {         // add list of better versions         drone1.brain.better = drone1.brain.better || [];          // passed reference - can modify original brain object directly         drone1.brain.better.push({             ladid : drone2.ladid,             brainid : drone2.brainid         });     } } 

now of course, number of iterations increases exponentially when dataset increases. 3000 brains in total, there 9 million iterations, magic adds more 10 seconds of execution time.

what optimizations (hugely) beneficial in scenario this, apart using multiple threads?

since judge() purely math, make difference if convert every single step of iteration callback style? (in imagination, create huge overhead of anonymous functions , memory usage.)


Comments

Popular posts from this blog

javascript - how to protect a flash video from refresh? -

visual studio 2010 - Connect to informix database windows form application -

android - Associate same looper with different threads -