BVengerov BVengerov - 1 month ago 10
PHP Question

Workaround for PHP taking up too much memory when using Reflection class in a loop

I have a loop like this

foreach ($classes as $class)
{
$reflectionClass = new \ReflectionClass($class);
... /*code that doesn't matter - commenting it out leaves the
memory consumption all the same */
}


Which gives the Fatal error: Allowed memory size of 134217728 bytes exhausted result in case of a big number of classes to loop through and relatively small allowed memory size in php.ini (for me that's around 4000 classes and 128 MB memory setting).

Memory usage right before the loop start is around 1.6 MB.

As you might guess, leaving
unset($reflectionClass)
at the end of the loop body doesn't help at all.

Upon some googling my guess is that PHP doesn't free the memory taken by the object in case it has some internal references to other objects.
Now, these posts (one, two, three) made me try using the garbage collector expicitly:

gc_enable();
foreach ($classes as $class)
{
$reflectionClass = new \ReflectionClass($class);
...
unset($reflectionClass);
gc_collect_cycles();
}


Which still leads to the same result.

The solutions I see:
1) Increase allowed memory setting - which is ugly and sad.
2) Separate classes into portions and get the needed result from each portion separately via forking or executing some other PHP script - but this sounds like a tough way to go.

Is there a simple workaround for the memory leak? Am I missing something here?

Update

Thanks to Paul Crovella for pointing out that it is keeping loaded class definitions that takes up memory, and it is in fact not a leak.

So it can be solved either by execution in the child process, or by another script. Added my own solution as an answer.

Answer

Implemented a workaround via delegating processing of portions of classes to child processes, which take all the loaded class definitions down to Hades with them upon exiting.

Looks like this:

    foreach (array_chunk($classes, 300) as $classesPortion)
    {
        if (socket_create_pair(AF_UNIX, SOCK_STREAM, 0, $socketArray) === false)
            throw new \Exception('Could not create socket');

        $pid = pcntl_fork();
        if ($pid === -1) //Forking failed
        {
            throw new \Exception('Could not fork process');
        }
        elseif ($pid === 0) //Is child process
        {
            socket_close($socketArray[1]);
            foreach ($classesPortion as $class)
            {
                $data = ...; //generating needed data
            }
            $dataString = serialize($data);
            if (!socket_write($socketArray[0], $dataString, strlen($dataString)))
            {
                throw new \Exception('Failed to write to socket');
            }
            socket_close($socketArray[0]);
            exit(0);
        }
        else //Is parent process
        {
            socket_close($socketArray[0]);
            pcntl_waitpid($pid, $childProcessStatus);
            if ($childProcessStatus !== 0)
            {
                throw new \Exception('Child process exited abnormally');
            }
            else
            {
                $result = socket_read($socketArray[1], 1000000, PHP_BINARY_READ);
                ... // deal with result
            }
            socket_close($socketArray[1]);
        }
    }