Mike Jones Mike Jones - 1 month ago 16
Python Question

Finding and plotting data in a csv

My current code looks for this type of data:

AHR2,231,123,5,12,51
GPS,12,312,512,35,12
AHR2,13,125,125123,152,12
CMD,123,123,5,123,51,12
PRAM,1231,CM,12
PRAM,12345,DM,14
AND SO ON


Here is how I am doing it right now:

import csv
import matplotlib.pyplot as plt
AHR2 = []
CM = []
with open('data2.csv', 'r') as csvfile:
content = csv.reader(csvfile, delimiter=',')
for row in content:
if 'AHR2' == row[0]:
AHR2.append([row[0]] + list(map(float, row[1:])))
AHR2 = list(zip(*AHR2))
ax.plot(AHR2[1], AHR2[5], label='AHR2 Alt', color = 'red')


This is working fine, except when I am trying to parse the PRAM data part. As you can see there is more than 1 PRAM with different values (CM,DM) I am only interested in the ones that have CM. I have tried this but it gave me an index error.

if 'CM' == row[2]:
CM.append([row[0]] + list(map(float, row[1:])))


Is there a way for me to look between the rows and pull the entire row like I am doing above?

Answer

I would really suggest using pandas for this type of work, it will save you a lot of headaches. Your data is a bit messy since it doesn't have a constant number of columns per row, one way to deal with this is discussed here.

Something like this should work

import pandas as pd
MAX_COLS_PER_ROW = 7
my_cols = range(0, MAX_COLS_PER_ROW)
df = pd.read_csv('test.csv', names=my_cols)
df

Out[1]:
      0      1    2       3      4     5     6
0  AHR2    231  123       5   12.0  51.0   NaN
1   GPS     12  312     512   35.0  12.0   NaN
2  AHR2     13  125  125123  152.0  12.0   NaN
3   CMD    123  123       5  123.0  51.0  12.0
4  PRAM   1231   CM      12    NaN   NaN   NaN
5  PRAM  12345   DM      14    NaN   NaN   NaN

Then filtering can easily be done with something like

(df.loc[:,0] == "PRAM") & (df.loc[:,2] == "CM")
Out[2]:
0    False
1    False
2    False
3    False
4     True
5    False
dtype: bool

df.loc[(df.loc[:,0] == "PRAM") & (df.loc[:,2] == "CM"), :]
Out[3]:
      0     1   2   3   4   5   6
4  PRAM  1231  CM  12 NaN NaN NaN

I'll leave the plotting to you since the question seems more to do with filtering the data and I am slightly confused on the exact end goal.

Comments