eelioss eelioss - 4 months ago 48x
Python Question

Mock in multithreading context for testing in Python

In a Django project, I would like to write a test for a function that is used in a multiprocessing context (Processing.create_all_files). If I were using a single thread, I would do 'mock' in order to check the parameters used for calling a given function (FileCreator.create in my case).

However, once the function FileCreator.create is called by a multiprocessing.Pool, mock does not work anymore with it.

How should I do my test for create_all_files? Thanks.

def test_create_all_files(self):
file_paths = [ (...) ] # Initialize file_paths
processing = Processing()
with mock.patch('FileCreator.create', return_value=True) as create:
with mock.patch('os.path.isfile', return_value=False):
calls = create.call_args_list

for file_path in file_paths:
self.assertTrue(((file_path),) in calls)

def unwrap_self_create_one_file(arg):
return Processing.process_one_file(*arg)

class Processing:

def create_one_file(self, file_path):
if os.path.isfile(file_path):
FileCreator.create(file_path) # CREATE FILE

def create_all_files(file_paths):
(...) # define args_lst considering file_paths
ncpus = 4
pool = multiprocessing.Pool(ncpus), args_lst, chunksize=1)


Your issue have nothing to do with multithreading but it is more related on Where to patch.

In your test you use FileCreator.create(file_path) to create your file object so I guess you have something like from mymodule import FileCreator in

What you should do in these case is patch the FileCreator reference in program by:

with mock.patch('program.FileCreator.create', return_value=True) as create: