Marty Marty - 7 months ago 189
Python Question

How do I handle exceptions with Django objects.bulk_create()

I am using

bulk_create
to create objects of a model. Whenever an exception is raised, how do I handle it?

aList = [
Student(name="Jason"),
Student(name="Mark"),
Student(name="Tom"),
Student(name="Jason"),
Student(name="Tom"),
]

Student.objects.bulk_create(aList)


as the model has field
name
that is
unique=True
, I have to process the inputs as the exception is raised. How do I handle exceptions one by one.

This don't work as intended,

try:
# bulk_create
except:
# handlers


as the exception is raised, the
bulk_create
process is terminated.

PS. I am looking forward for the
bulk_create
. no loops with
create
or
update_or_create
or
get_or_create

Answer

Well, that's impossible. That's how bulk_create is defined:

def bulk_create(self, objs, batch_size=None):
        for parent in self.model._meta.get_parent_list():
            if parent._meta.concrete_model is not self.model._meta.concrete_model:
                raise ValueError("Can't bulk create a multi-table inherited model")
        if not objs:
            return objs
        self._for_write = True
        connection = connections[self.db]
        fields = self.model._meta.concrete_fields
        objs = list(objs)
        self._populate_pk_values(objs)
        with transaction.atomic(using=self.db, savepoint=False):
            if (connection.features.can_combine_inserts_with_and_without_auto_increment_pk
                    and self.model._meta.has_auto_field):
                self._batched_insert(objs, fields, batch_size)
            else:
                objs_with_pk, objs_without_pk = partition(lambda o: o.pk is None, objs)
                if objs_with_pk:
                    self._batched_insert(objs_with_pk, fields, batch_size)
                if objs_without_pk:
                    fields = [f for f in fields if not isinstance(f, AutoField)]
                    self._batched_insert(objs_without_pk, fields, batch_size)

        return objs

And the _batched_insert:

def _batched_insert(self, objs, fields, batch_size):
    """
    A little helper method for bulk_insert to insert the bulk one batch
    at a time. Inserts recursively a batch from the front of the bulk and
    then _batched_insert() the remaining objects again.
    """
    if not objs:
        return
    ops = connections[self.db].ops
    batch_size = (batch_size or max(ops.bulk_batch_size(fields, objs), 1))
    for batch in [objs[i:i + batch_size]
                  for i in range(0, len(objs), batch_size)]:
        self.model._base_manager._insert(batch, fields=fields,
                                         using=self.db)

So, as you can see bulk_create is basically a for loop inside an transaction.atomic.

One more thing. It is also impossible to save only some entries inside the transaction block. It either is executed fully or not executed at all.

Comments