max max - 1 month ago 11
Python Question

Exposing `defaultdict` as a regular `dict`

I am using

defaultdict(set)
to populate an internal mapping in a very large data structure. After it's populated, the whole structure (including the mapping) is exposed to the client code. At that point, I don't want anyone modifying the mapping.

And nobody does, intentionally. But sometimes, client code may by accident refer to an element that doesn't exist. At that point, a normal dictionary would have raised
KeyError
, but since the mapping is
defaultdict
, it simply creates a new element (an empty set) at that key. This is quite hard to catch, since everything happens silently. But I need to ensure this doesn't happen (the semantics actually doesn't break, but the mapping grows to a huge size).

What should I do? I can see these choices:


  1. Find all the instances in current and future client code where a dictionary lookup is performed on the mapping, and convert it to
    mapping.get(k, {})
    instead. This is just terrible.

  2. "Freeze"
    defaultdict
    after the data structure is fully initialized, by converting it to
    dict
    . (I know it's not really frozen, but I trust client code to not actually write
    mapping[k] = v
    .) Inelegant, and a large performance hit.

  3. Wrap
    defaultdict
    into a
    dict
    interface. What's an elegant way to do that? I'm afraid the performance hit may be huge though (this lookup is heavily used in tight loops).

  4. Subclass
    defaultdict
    and add a method that "shuts down" all the
    defaultdict
    features, leaving it to behave as if it's a regular
    dict
    . It's a variant of 3 above, but I'm not sure if it's any faster. And I don't know if it's doable without relying on the implementation details.

  5. Use regular
    dict
    in the data structure, rewriting all the code there to first check if the element is in the dictionary and adding it if it's not. Not good.


Answer

defaultdict docs say for default_factory:

If the default_factory attribute is None, this raises a KeyError exception with the key as argument.

What if you just set your defaultdict's default_factory to None? E.g.,

>>> d = defaultdict(int)
>>> d['a'] += 1
>>> d
defaultdict(<type 'int'>, {'a': 1})
>>> d.default_factory = None
>>> d['b'] += 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'
>>> 

Not sure if this is the best approach, but seems to work.