MoJi MoJi - 1 month ago 7
Python Question

Fastest way to substitute a set of characters in a string

I'm working with a string of bytes (which can be anywhere between 10kb and 3MB) and I need to filter out approximately 16 bytes (replacing them with other bytes)

At the moment I have a function a bit like this..

BYTE_REPLACE = {
52: 7, # first number is the byte I want to replace
53: 12, # while the second number is the byte I want to replace it WITH
}
def filter(st):
for b in BYTE_REPLACE:
st = st.replace(chr(b),chr(BYTE_REPLACE[b]))
return st


(Byte list paraphrased for the sake of this question)

Using map resulted in an execution time of ~.33s, while this results in a 10x faster time of ~.03s (Both performed on a HUGE string, larger than 1.5MB compressed).

While any performance gains would be considerably negligible, is there a better way of doing this?

(I am aware that it would be much more optimal to store the filtered string. This isn't an option, though. I'm fooling with a Minecraft Classic server's level format and have to filter out bytes that certain clients don't support)

Answer

Use str.translate:

Python 3.x

def subs(st):
    return st.translate(BYTE_REPLACE)

Example usage:

>>> subs('4567')
'\x07\x0c67'

Python 2.x

str.translate (Python 2)

import string
k, v = zip(*BYTE_REPLACE.iteritems())
k, v = ''.join(map(chr, k)), ''.join(map(chr, v))
tbl = string.maketrans(k, v)
def subs(st):
    return st.translate(tbl)