ihightower ihightower - 1 month ago 11
Python Question

pandas dataframe filter to return True for ALL rows. how?

Hi I have a filter 'm' set that is flexible enough to change by me. Sometimes, I want to filter by Car or x_acft_body , or any of the various other fields, etc. Sometime I want to have all of the rows returned by commenting and uncommenting the required lines. But without changing the subsequent code, after the filter 'm' line.

How can I have a filter that will return true for ALL rows, when I don't want the filters applied? For e.g. something like

1==1
but i know this doesn't work.

I don't want to set
dfdata.somefield.notnull()
etc. as I will not be too sure if this field will be always not null or not. also I DO NOT want to change subsequent code to be like
dfdata.groupby.
i.e. without the
[m]


# set filter if needed
m = ( 1==1 #& return true at all times
# (dfdata.Car == 'PG') #&
# (dfdata.x_acft_body == 'N')# &
# (dfdata.Car.isin(['PG', 'VJ', 'VZ']))
)


dft1 = dfdata[m].groupby(['FLD1']).agg({'FLD2': 'count'})

Answer Source

You can create bool constant and change final mask by it:

#True for return all rows
m = (dfdata.Car == 'PG') | True

And:

#False for apply filter
m = (dfdata.Car == 'PG') | False

First solutions:

m = [True] * len(df.index)

m = np.repeat(True, len(df.index))