Derek Halden Derek Halden - 3 years ago 89
Python Question

for loop save to array but skip saving elements

Basically, I want a fancy oneliner that doesn't read all of the files I'm looking at into memory, but still processes them all, and saves a nice sample of them.

The oneliner I would like to do is:

def foo(findex):
return [bar(line) for line in findex] # but skip every nth term

But I would like to be able to not save every nth line in that. i.e., I still want it to run (for byte position purposes), but I don't want to save the image, because I don't have enough memory for that.

So, if the output of bar(line) is
I would like it to still run on
but I would like the return value to be
or something of the sort.

Answer Source

use enumerate to get the index, and a filter using modulo to take every other line:

return [bar(line) for i,line in enumerate(findex) if i%2]

Generalize that with i%n so everytime that the index is divisible by n then i%n==0 and bar(line) isn't issued into the listcomp.

Edit: if you're sure that findex is subscriptable (list, tuple) you could use range with step which would avoid to iterate on all occurrences (if n is big the enumerate solution is a CPU waster). So like posted in a now deleted answer: [bar(findex[i]) for i in range (0, len(findex), 2) ]

But enumerate works for every iterable (file handle, generator ...).

EDIT: if you want to still compute all the values (because you need the side effect generated by bar), the above doesn't work because the filter prevents that, so you have to do that in 2 passes, for instance using map to apply your function to all items of findex and pick only the results you're interested in (but it guarantees that all of the lines are processed):

l = [x for i,x in enumerate(map(bar,findex)) if i%n]
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download