RatDon RatDon - 2 months ago 14
Python Question

Python multithreading doesn't use common memory when the parent object is created inside a process

I'm trying to implement a thread which will run and do a task in the background. Here's my code which gives expected output.

Code 1:

from time import sleep
from threading import Thread

class myTest( ):
def myThread( self ):
while True:
sleep( 1 )
print self.myList.keys( )
if 'exit' in self.myList.keys( ):
break
return

def __init__( self ):
self.myList = { }
self.thread = Thread( target = self.myThread, args = ( ) )
self.thread.start( )
return

def myFun( self ):
i = 0
while True:
sleep( 0.5 )
self.myList[ i ] = i
i += 1
if i > 5 :
self.myList[ 'exit' ] = ''
break
return

x = myTest( )
x.myFun( )


Output 1:

[0]
[0, 1, 2]
[0, 1, 2, 3, 4]
[0, 1, 2, 3, 4, 5, 'exit']


The moment I create a multi-process environment and create this object in the new sub-process, the thread is unable to access the common memory and the dictionary
myList
remains blank in the thread. Here is the code.

Code 2:

from time import sleep
from threading import Thread
from multiprocessing import Process

class myTest( ):
def myThread( self ):
while True:
sleep( 1 )
print "myThread", self.myList.keys( )
if 'exit' in self.myList.keys( ):
break
return

def __init__( self ):
self.myList = { }
self.thread = Thread( target = self.myThread, args = ( ) )
self.thread.start( )
self.i = 0
return

def myFun( self ):
self.myList[ self.i ] = self.i
self.i += 1
if self.i > 5 :
self.myList[ 'exit' ] = ''
return 'exit'
print 'myFun', self.myList.keys( )
return

class myProcess( Process ):
def __init__( self ):
super( myProcess, self ).__init__( )
self.myObject = myTest( )
return

def run(self):
while True:
sleep( 0.5 )
x = self.myObject.myFun( )
if x == 'exit':
break
return

x = myProcess( )
x.start( )


Output 2:

myFun [0]
myThread []
myFun [0, 1]
myFun [0, 1, 2]
myThread []
myFun [0, 1, 2, 3]
myFun [0, 1, 2, 3, 4]
myThread []
myThread []
myThread []
... ... ...
... ... ...


In the 1st code, the object is created inside a Python process (Though main process). In the 2nd code, the object is created inside a sub-process of Python where everything takes place which should have behaved just like the first one.


  1. Can anybody explain why in the 2nd code, the thread is unable to use the shared memory of the parent object?

  2. I want the
    output 1
    in
    code 2
    i.e. force the thread to use the shared common memory of the parent object. What modification do I need to make?


Answer

When you spawn the second process, is created about a copy of the first one, so the threads spawned in the 2nd process can only access that process stack.

In code2 The object of myTest is created in the constructor (__init__), so it won't share the same memory as of the spawned process which is run. Just by changing the position of the object creation from constructor to run, the object created becomes part of the spawned process memory.

Comments