Edward Edward - 4 months ago 6x
Python Question

What is the Python way of chaining maps and filters?

I'm currently learning Python (coming from other languages like JavaScript and Ruby). I am very used to chaining a bunch of transformations / filters, but I'm pretty sure that's not the right way to do it in Python:

takes a lambda before the enumerable, so writing a long / multi-line function looks really weird and chaining them means putting them in reverse order which isn't readable.

What would be the "Python way" of writing the maps and filters in this JavaScript function?

let is_in_stock = function() /* ... */
let as_item = function() /* ... */

let low_weight_items = shop.inventory
.filter(item => item.weight < 1000)
.map(item => {
if (item.type == "cake") {
let catalog_item = retrieve_catalog_item(item.id);

return {
id: item.id,
weight: item.weight,
barcode: catalog_item.barcode
} else {
return default_transformer(item);

I understand that I might use a list comprehension for the first map and the next two filters, but I am not sure how to do the last map and how to put everything together.

Thank you!


One good way to do this is to combine multiple filters/maps into a single generator comprehension. In cases where this can't be done, define an intermediate variable for the intermediate map/filter you need, instead of trying to force the maps into a single chain. For instance:

def is_in_stock(x):
   # ...
def as_item(x):
   # ...
def transform(item):
    if item.type == "cake":
        catalog_item = retrieve_catalog_item(item.id)
        return {
            "id": item.id,
            "weight": item.weight,
            "barcode": catalog_item.barcode
        return default_transformer(item)

items = (as_item(item) for item in shop.inventory)
low_weight_items = (transform(item) for item in items if is_in_stock(item) and item.weight < 1000)

Note that the actual application of the maps and filters is all done in the last two lines. The earlier part is just defining the functions that encode the maps and filters.

The second generator comprehension does the last two filters and the map all together. Using generator comprehensions means that each original item in inventory will be mapped/filtered lazily. It won't pre-process the entire list, so it is likely to perform better if the list is large.

Note that there is no Python equivalent to defining long functions inline as in your JavaScript example. You can't specify that complex filter (the one with item.type == "cake") inline. Instead, as shown in my example, you must define it as a separate function, just as you did with is_in_stock and as_item.

(The reason the first map was split is that later filters can't act on the mapped data until after it's mapped. It could be combined into one, but that would require manually redoing the as_item map inside the comprehension:

low_weight_items = (transform(as_item(item)) for item in items if is_in_stock(as_item(item)) and as_item(item).weight < 1000)

It's clearer to just separate out that map.)