Pedro Batista Pedro Batista -4 years ago 178
Python Question

In Python, do I need to protect data transfer between multi-threaded processes?

For example, if a process is generating an image, and other parallel process is accessing this image through a get method, my intuition tells me that it may be dangerous to access that image while it is being written.

In C++ I have to use mutexes to make sure the image isn't accessed while it is being written, otherwise I'm experiencing random segfaults. but since python has some data protection mechanisms that I don't fully know, I'm not sure if I need to do this.


Class Camera(object):

def __init__(self):
self._capture = camera_handler() #camera_handler is a object that loads the driver and lets you control the camera.
self.new_image = None
self._is_to_run = False

def start(self):
self._is_to_run = True
self._thread = thread_object(

def run(self):
self.new_image = self._capture.update()

cam = Camera()

while True:
image = cam.new_image
result = do_some_process_image(image)

Is this safe?

Answer Source

First of al, the threading module uses threads, not different processes!

The crucial difference between threads and processes is that the former share an address space (memory), while the latter don't.

The "standard" python implementation (CPython) uses a Global Interpreter Lock to ensure that only one thread at a time can be executing Python bytecode. So for data that can be updated with one one bytecode instruction (like store_fast) you might not need mutexes. When a thread that is modifying such a variable is interrupted, either the store has been done or it hasn't.

But in general you definitely need to protect data structures from reading and modification by multiple threads. If a thread is interrupted while it is in the proces of modifying say a large dictionary and execution is passed to another thread that tries to read from the dictionary, it might find the data in an inconsistant state.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download