Suhairi Suhaimin Suhairi Suhaimin - 19 days ago 5
Python Question

How to replace specific punctuation with new name?

My data sample is:

comment sarc_majority
0 [?, ?] sarc
1 [0] non-sarc
2 [!, !, !] sarc
3 [0] non-sarc
4 [?] sarc


I want to replace the punctuation with a new name. Such as ? = punct1, ! = punct2, ' = punct3. I tried using read from csv file.

replace_df = pd.read_csv('./final/eng-mly-punct.csv', sep=',', quoting=csv.QUOTE_NONE,
names=["punct", "replacer"])
replace_df.head()

punct replacer
0 ? punct1
1 ! punct2
2 ' punct3


Then I stucked at replacing:

for punct, replacer in replace_df.itertuples(index=False,name=None):
df.comment = df.comment.str.replace(r'\b{0}\b'.format(punct),replacer)


The error is: error: nothing to repeat

What have gone wrong? Or is there any possible way to do this?
The desired output should be just like:

comment sarc_majority
0 [punct1, punct1] sarc
1 [0] non-sarc
2 [punct2, punct2, punct2] sarc
3 [0] non-sarc
4 [punct1] sarc


Thanks in advance. Cheers.

Answer

You can use replace by dict d - but need escape ? to \?:

d = {'\?':'punct1','!':'punct2',"'":'punct3'}
df.comment = df.comment.replace(d, regex=True)
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc

Also you can create d from replace_df:

df = pd.DataFrame({'comment': {0: '[?, ?]', 1: '[0]', 2: '[!, !, !]', 3: '[0]', 4: '[?]'}, 'sarc_majority': {0: 'sarc', 1: 'non-sarc', 2: 'sarc', 3: 'non-sarc', 4: 'sarc'}})
print (df)
     comment sarc_majority
0     [?, ?]          sarc
1        [0]      non-sarc
2  [!, !, !]          sarc
3        [0]      non-sarc
4        [?]          sarc

replace_df = pd.DataFrame({'replacer': {0: 'punct1', 1: 'punct2', 2: 'punct3'}, 'punct': {0: '?', 1: '!', 2: "'"}})
print (replace_df)
  punct replacer
0     ?   punct1
1     !   punct2
2     '   punct3
replace_df.punct = '\\' + replace_df.punct
d = replace_df.set_index('punct')['replacer'].to_dict()
print (d)
{'\\!': 'punct2', "\\'": 'punct3', '\\?': 'punct1'}

df.comment = df.comment.replace(d, regex=True)
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc

EDIT by comment:

df = pd.DataFrame({'comment':[['?', '?'],[0], ['!', '!', '!'], [0], ['?']], 'sarc_majority': [ 'sarc','non-sarc', 'sarc', 'non-sarc','sarc']})
print (df)
     comment sarc_majority
0     [?, ?]          sarc
1        [0]      non-sarc
2  [!, !, !]          sarc
3        [0]      non-sarc
4        [?]          sarc

print (type(df.ix[0,'comment']))
<class 'list'>

replace_df = pd.DataFrame({'replacer': {0: 'punct1', 1: 'punct2', 2: 'punct3'}, 'punct': {0: '?', 1: '!', 2: "'"}})
#print (replace_df)

replace_df.punct = '\\' + replace_df.punct.apply(lambda x: x.format())
d = replace_df.set_index('punct')['replacer'].to_dict()
print (d)
{'\\!': 'punct2', "\\'": 'punct3', '\\?': 'punct1'}

df.comment = df.comment.apply(lambda x: pd.Series(x).astype(str).replace(d, regex=True).tolist())
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc
Comments