Michael Perdue Michael Perdue - 1 month ago 6
Python Question

Check if certain value is contained in pandas column python

I am trying to check if a certain value is contained in a python column. I'm using

df.date.isin(['07311954'])
, which I do not doubt to be a good tool. The problem is that I have over 350K rows and the output won't show
all of them so that I can see if the value is actually contained. Put simply, I just want to know (Y/N) whether or not a specific value is contained in a column. My code follows:

import numpy as np
import pandas as pd
import glob


df = (pd.read_csv('/home/jayaramdas/anaconda3/Thesis/FEC_data/itpas2_data/itpas214.txt',\
sep='|', header=None, low_memory=False, names=['1', '2', '3', '4', '5', '6', '7', \
'8', '9', '10', '11', '12', '13', 'date', '15', '16', '17', '18', '19', '20', \
'21', '22']))

df.date.isin(['07311954'])

Answer

I think you need str.contains, if you need rows where values of column date contains string 07311954:

print df[df['date'].astype(str).str.contains('07311954')]

Or if type of date column is string:

print df[df['date'].str.contains('07311954')]

If you want check last 4 digits for string 1954 in column date:

print df[df['date'].astype(str).str[-4:].str.contains('1954')]

Sample:

print df['date']
0    8152007
1    9262007
2    7311954
3    2252011
4    2012011
5    2012011
6    2222011
7    2282011
Name: date, dtype: int64

print df['date'].astype(str).str[-4:].str.contains('1954')
0    False
1    False
2     True
3    False
4    False
5    False
6    False
7    False
Name: date, dtype: bool

print df[df['date'].astype(str).str[-4:].str.contains('1954')]
     cmte_id trans_typ entity_typ state  employer  occupation     date  \
2  C00119040       24K        CCM    MD       NaN         NaN  7311954   

   amount     fec_id    cand_id  
2    1000  C00140715  H2MD05155  
Comments