zrneely zrneely - 1 year ago 96
Python Question

Can't pickle object error using SqlAlchemy

I'm trying to store a mutable object in a postgresql database using SqlAlchemy. The class of the object is roughly:

class Data(Mutable, object):
def coerce(cls, key, value):
return value

def __init__(self, foo, bar):
self._foo = foo # foo is an array of short strings
self._bar = bar # bar is a single short string

def append_to_foo(self, baz):

# Various other methods for mutating/accessing foo and bar, which call self.changed()
# when they finish

The column definition is:

data = Column(Data.as_mutable(PickleType))

Whenever I try to add a row to the table including this column, I get the following error:

sqlalchemy.exc.StatementError: (builtins.AttributeError) Can't pickle local object 'WeakKeyDictionary.__init__.<locals>.remove' [SQL: "INSERT INTO table (id, user_id, data) VALUES (nextval('data_id_seq'), %(user_id)s, %(data)s) RETURNING data.id"] [parameters: [{'data': <mypkg.foo.Data object at 0x7f79b3b52c88>, 'user_id': 36}]]

I've made sure that the
class itself is pickleable using the Python prompt; I can
instances with no issues. Google produces no results, and I couldn't find any relevant bug reports.

I'm using SqlAlchemy 1.0.13 and Python 3.5.

Answer Source

From reading "Supporting Pickling" it would seem you must provide at least the __getstate__ method for custom mutable types:

The developer responsibility here is only to provide a __getstate__ method that excludes the _parents() collection from the pickle stream:

This is because the mutable extension places a weakref.WeakKeyDictionary upon the value object, which cannot be pickled. A minimal __getstate__ implementation is provided in the documentation:

class Data(Mutable, object):


    def __getstate__(self):
        d = self.__dict__.copy()
        d.pop('_parents', None)
        return d

Depending on how your type is implemented you may have to provide __setstate__ as well.