srytoomanyquestions srytoomanyquestions - 7 months ago 17
Python Question

==1 still returning NaN values pandas dataframe

This is my first question and i'm relatively new to all this so sorry in advance if it is stupid, but i've spent quite a bit of time searching and trying dif solutions.

I have a df of track data and their weekly rankings. For weeks where tracks went unranked, the data is 'NaN'.

pv = df.pivot_table(df, index=['track'])
pv2 = pv.fillna(100)
pv2[pv2 == 1]


My question is, why is it still returning NaN in pv2 after the NaN is filled to '100'? I haven't bee able how to get rid of the NaN and for instance, return all tracks that were 1st or 10th (for all weeks). By the nature of the data, some columns will have 1's and NaN, other will have NaN but no 1's. Most columns are a combo of NaN and range between 1 < 100.

I've tried to create another pv table (with dif indexes, making it so only the weeks are the columns, and only int64 as the dtype), changing dtypes, to no avail. I think i may be looking at it the wrong way, as i'm quite new to all this...

Answer

If you apply mask to dataframe, you get NaN where values are False:

Sample:

import pandas as pd
import io

temp=u"""a,b,c,d,e,f,g,h
a,1,227,2.33,15.7,158,149,158
b,2,1,231,1.3,15.9,145,1
c,0,228,23.3,1.57,1,145,153
d,7,227,2.31,15.6,159,149,1"""
#after testing replace io.StringIO(temp) to filename
df = pd.read_csv(io.StringIO(temp))
print df
   a  b    c       d      e      f    g    h
0  a  1  227    2.33  15.70  158.0  149  158
1  b  2    1  231.00   1.30   15.9  145    1
2  c  0  228   23.30   1.57    1.0  145  153
3  d  7  227    2.31  15.60  159.0  149    1

print df == 1
       a      b      c      d      e      f      g      h
0  False   True  False  False  False  False  False  False
1  False  False   True  False  False  False  False   True
2  False  False  False  False  False   True  False  False
3  False  False  False  False  False  False  False   True

print df[df == 1]
     a    b    c   d   e    f   g    h
0  NaN  1.0  NaN NaN NaN  NaN NaN  NaN
1  NaN  NaN  1.0 NaN NaN  NaN NaN  1.0
2  NaN  NaN  NaN NaN NaN  1.0 NaN  NaN
3  NaN  NaN  NaN NaN NaN  NaN NaN  1.0
Comments