Python Multiprocessing

The following code will enable you to implement multiprocessing in any Python project that contains a for loop. Take your for loop that has a long-running function in it and replace “for i in iterable” with your for loop conditions. Change “function_name” to the name of the function that will be executed. Change the list of function arguments to what you need.

import multiprocessing
number_cores = multiprocessing.cpu_count()
pool = multiprocessing.Pool(processes=number_cores)
r = [pool.apply_async(function_name, args=(function_arg1, function_arg2)) for i in iterable]
output = [p.get() for p in r]
pool.terminate()

The above code would be the multiprocessing implementation of the following.

output = []
for i in iterable:
    output += function_name(function_arg1, function_arg2)