Is it possible to execute function every x seconds in python, when it is performing pool.map? -

i running pool.map on big data array , want print report in console every minute. possible? understand, python synchronous language, can't nodejs.

perhaps can done threading.. or how?

finished = 0  def make_job():    sleep(1)    global finished    finished += 1  # want call function every minute def display_status():    print 'finished: ' + finished  def main():     data = [...]     pool = threadpool(45)     results = pool.map(make_job, data)     pool.close()     pool.join() 

you can use permanent threaded timer, question: python threading.timer - repeat function every 'n' seconds

from threading import timer,event   class perpetualtimer(object):     # give cycle time (t) , callback (hfunction)     def __init__(self,t,hfunction):       self.t=t       self.stop = event()       self.hfunction = hfunction       self.thread = timer(self.t,self.handle_function)     def handle_function(self):       self.hfunction()       self.thread = timer(self.t,self.handle_function)       if not self.stop.is_set():           self.thread.start()     def start(self):       self.stop.clear()       self.thread.start()     def cancel(self):       self.stop.set()       self.thread.cancel() 

basically wrapper timer object creates new timer object every time desired function called. don't expect millisecond accuracy (or close) this, purposes should ideal.

using example become:

finished = 0  def make_job():    sleep(1)    global finished    finished += 1  def display_status():    print 'finished: ' + finished  def main():     data = [...]     pool = threadpool(45)      # set monitor make run function every minute     monitor = perpetualtimer(60,display_status)     monitor.start()     results = pool.map(make_job, data)     pool.close()     pool.join()     monitor.cancel() 

edit:

a cleaner solution may (thanks comments below):

from threading import event,thread   class repeattimer(thread):     def __init__(self, t, callback, event):         thread.__init__(self)         self.stop = event         self.wait_time = t         self.callback = callback         self.daemon = true      def run(self):         while not self.stop.wait(self.wait_time):             self.callback() 

then in code:

def main():     data = [...]     pool = threadpool(45)     stop_flag = event()     repeattimer(60,display_status,stop_flag).start()     results = pool.map(make_job, data)     pool.close()     pool.join()     stop_flag.set()