spiderface spiderface - 22 days ago 9
Python Question

Python deepcopy with custom __getattr__ and __setattr__

I have implemented a class that can distinguish between a fixed set of instance attributes (let's call them meta-attributes) and an arbitrary set of other instance attributes.

It has custom

__getattr__
and
__setattr__
:

class MyClass(object):

def __init__(self, meta1, meta2, **other_attr):
super(MyClass, self).__setattr__('meta1', meta1)
super(MyClass, self).__setattr__('meta2', meta2)
super(MyClass, self).__setattr__('params', {})

self.params = {key: other_attr[key] for key in other_attr}


# this is called when default lookup finds nothing
def __getattr__(self, key):
print('__getattr__({})'.format(key))
try:
return self.params[key]
except KeyError:
raise AttributeError(key)


# this is called always
def __setattr__(self, key, value):
print('__setattr__({}, {})'.format(key, value))
if key in self.__dict__:
super(MyClass, self).__setattr__(key, value)
else:
self.params[key] = value


This works fine, all meta-attributes go directly into the
__dict__
of an instance, while all the other attributes go into the
params
dictionary:

obj1 = MyClass(meta1 = 'foo', meta2 = 'bar', x=1, y=2, z=3)
obj1.w = 4
print(obj1.__dict__)


Output:

__setattr__(params, {'y': 2, 'x': 1, 'z': 3})
__setattr__(w, 4)
{'meta1': 'foo', 'meta2': 'bar', 'params': {'y': 2, 'x': 1, 'z': 3, 'w': 4}}


Except when I try to
deepcopy
my object, it does something strange:

import copy
obj1 = MyClass(meta1='foo', meta2='bar', x=1, y=2, z=3)
obj2 = copy.deepcopy(obj1)


Output:

__setattr__(params, {'y': 2, 'x': 1, 'z': 3})
__getattr__(__deepcopy__)
__getattr__(__getnewargs__)
__getattr__(__getstate__)
__getattr__(__setstate__)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
__getattr__(params)
...
and then it calls __getattr__ about a hundred more times


In the end it does create a copy, but why does it make so many calls to
__getattr__
?

Answer

This is causing a recursive lookup (remember obj2 is not initialised via __init__)

return self.params[key]

You should instead do this

return super().__getattribute__('params')[key]
Comments