Iron Attorney Iron Attorney - 9 days ago 5
Python Question

Python 3: Removing list item with for loop, is this the right way?

I am still learning the basics of python, and I have just spent a while reading about how to remove an item from a list in python from within a for loop. Everything I've read suggests complex ways of doing this, and they say that you cannot remove an item from a list while you're iterating over it. However... this seems to work:

class Object():
def __init__(self):
self.y = 0

object_list = [Object(), Object(), Object()]

for thing in object_list:
thing.y += 1
if thing.y > 10:
object_list.remove(thing)


Why is this working when others say it isn't and write complicated workarounds? Is it because you aren't allowed to do it in Python 2 but can in Python 3?

And is this the right way to do this? Will it work as I want it or will it be prone to bugs? Would it be advisable to iterate over the list in reverse order if I plan to remove items?

Sorry if this has been answered before, but it's hard to know which resources refer to what as they all just say "python" in the tag (at least, the ones I've been reading, maybe that's because all the ones I have read are python 2?)

Thanks!

EDIT:

Sorry, there were a couple of copy and paste errors... I've fixed them...

EDIT:

I've been watching another one of Raymond Hettinger's videos... He mentions a way of removing items from a dictionary while iterating over it by using dict.keys(). Something like:

d = {'text': 'moreText', 'other': 'otherText', 'blah': 'moreBlah'}

for k in d.keys():
if k.startswith('o'):
del d[k]


Apparently using the keys makes it safe to remove the item while iterating. Is there an equivalent for lists? If there was I could iterate backwards over the list and remove items safely

Answer

Here are some examples

def example1(lst):
    for item in lst:
        if item < 4:
            lst.remove(item) 
    return lst

def example2(lst):
    for item in lst[:]:
        if item < 4:
            lst.remove(item)       
    return lst

def example3(lst):
    i = 0
    while i < len(lst):
        if lst[i] < 4:
            lst.pop(i)
        else:
            i += 1
    return lst

def example4(lst):
    return [item for item in lst if not item < 4]

def example5(lst):
    for item in reversed(lst):
        if item < 4:
            lst.remove(item)
    return lst

def example6(lst):
    for i, item in reversed(list(enumerate(lst))):
        if item < 4:
            lst.pop(i)
    return lst

def example7(lst):
    size = len(lst) - 1
    for i, item in enumerate(reversed(lst)):
        if item < 4:
            lst.pop(size - i)
    return lst

def example8(lst):
    return list(filter(lambda item: not item < 4, lst))

import itertools
def example9(lst):
    return list(itertools.filterfalse(lambda item: item < 4, lst))

# Output
>>> lst = [1, 1, 2, 3, 2, 3, 4, 5, 6, 6]
>>> example1(lst[:])
[1, 3, 3, 4, 5, 6, 6]
>>> example2(lst[:])
[4, 5, 6, 6]
>>> example3(lst[:])
[4, 5, 6, 6]
>>> example4(lst[:])
[4, 5, 6, 6]
>>> example5(lst[:])
[4, 5, 6, 6]
>>> example6(lst[:])
[4, 5, 6, 6]
>>> example7(lst[:])
[4, 5, 6, 6]
>>> example8(lst[:])
[4, 5, 6, 6]
>>> example9(lst[:])
[4, 5, 6, 6]

Example 1 This example involves iterating through the list and removing values from it. The issue with this is that you are modifying the list as you go through it so your list changes during iteration and so some elements get skipped over.

Example 2 Here we are iterating over a shallow copy of the list instead of the list itself. The issue with this is if you have a large list it could be expensive to do so.

Example 3 The following is an example using pop instead of remove, the issue with remove is that it removes the first instance of the value it finds from the list. This will typically be of no issue unless you have objects which are equal. (See example 10)

Example 4 Instead of modifying the list here instead we create a new list using list comprehension allowing only specified values.

Example 5 This is an example of iterating through the list in reverse, the difference.

Example 6 Similar example using pop instead.

Example 7 Better example using pop

Example 8 Example using the built-in filter method to remove the specified values.

Example 9 Similar example using the filerfalse method from itertools

class Example(object):
    ID = 0
    def __init__(self, x):
        self._x = x
        self._id = str(Example.ID)
        Example.ID += 1

    def __eq__(self, other):
        return self._x == other._x

    def __repr__(self):
        return 'Example({})'.format(self._id)

def example10():
    lst = [Example(5), Example(5)]
    print(lst)
    lst.remove(lst[1])
    return lst

#Output
>>> example10()
[Example(0), Example(1)]
[Example(1)]

Example 10 Here we create two Example objects with the same values and by the equality method they are equal. The ID variable is there to help us differentiate between the two. Now we have specified that we want to remove the 2nd object from the list, however because both are equal the first item is actually removed instead.

Timings These are pretty rough times and can vary slightly depending on your device. Although these identify which one is faster, this was tested with a list of 10,000 items so if you don't have anything close to that then any choice is fine really.

import timeit
import random

# Code from above is here

def test(func_name):
    global test_lst
    test_lst = lst[:]
    return timeit.timeit("{}(test_lst)".format(func_name),
                         setup="from __main__ import {}, test_lst".format(func_name), number = 1)

if __name__ == '__main__':
    NUM_TRIALS = 1000
    lst = list(range(10000))
    random.shuffle(lst) # Don't have to but makes it a bit interesting
    test_list = lst[:]

    for func in ('example2', 'example3', 'example4', 'example5',
                 'example6', 'example7', 'example8', 'example9'):
        trials = []
        for _ in range(NUM_TRIALS):
            trials.append(test(func))
        print(func, sum(trials) / len(trials) * 10000)

#Output
example2 8.487979147454494
example3 20.407155912623292
example4 5.4595031069025035
example5 7.945100572479213
example6 14.43537688078149
example7 9.088818018676008
example8 14.898256300967116
example9 13.865010859443247