Piyush Piyush - 2 months ago 6
Python Question

Efficient way to read non-empty cells in a column in CSV file

I have a large python file (>500,000 rows), and would like to read non-empty cells in a column in the dataframe (panda). Right now, I am doing this:

df = pd.read_csv(filename)

myiter = []

for xiter, x in enumerate(df['Column_name']):

if (np.isnan(x) == False):
myiter.append(xiter)


Is there a more efficient way to do the same?

Answer

are they tagged as NaN in your df?

if yes then do

df.dropna()