GeekPro101 GeekPro101 - 2 months ago 8x
Python Question

Python counts duplicates as uniques in csv file

I've written a script that takes the html table from the executed offenders in Texas (can't post link due to restrictions but can be found in code for and converts it into a csv file. Another script then counts up the races of each person. However, I've been having an issue where it counts all but one of both white and hispanic, then counts it separately. This:

[('White', 237), ('Black', 196), ('Hispanic', 100), ('Other', 2), ('White ', 1), ('Hispanic ', 1)]
is the result.

This is the script that downloads the csv file (

import csv
from bs4 import BeautifulSoup
from urllib.request import urlopen

soup = BeautifulSoup(urlopen(''), "html.parser")
table = soup.find('table')
headers = [header.text for header in table.find_all('th')]
rows = []
for row in table.find_all('tr'):
rows.append([val.text for val in row.find_all('td')])

with open('new.csv', 'w', encoding="utf8", newline='') as f:
writer = csv.writer(f)
writer.writerows(row for row in rows if row)

This is the script the takes the races (

import csv
import collections

race = collections.Counter()

with open('new.csv') as input_file:
for row in csv.reader(input_file, delimiter=','):
race[row[8]] += 1

racecom = race.most_common()

print ('Number of white people executed: %s' % grades['White'])
print ('Number of black people executed: %s' % grades['Black'])
print ('Number of Hispanic people executed: %s' % grades['Hispanic'])
print ('Number of Other people executed: %s' % grades['Other'])
print (racecom)

However when I use a csv file generated by the problem disappears, so I am fairly sure it's that has the fault.

The generated file can be downloaded at as actual.csv and the auto downloaded one can be found at as new.csv.

Thanks in advance.


Whitespaces are important. You have to strip them away, if the keys should be the same:

with open('new.csv') as input_file:
    race = collections.Counter(row[8].strip()
        for row in csv.reader(input_file, delimiter=','))