vy32 vy32 - 1 year ago 120
Python Question

how do I .decode('string-escape') in Python3?

I have some escaped strings that need to be unescaped. I'd like to do this in Python.

For example, in python2.7 I can do this:

>>> "\123omething special".decode('string-escape')
'Something special'

How do I do it in Python3? This doesn't work:

>>> b"\123omething special".decode('string-escape')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
LookupError: unknown encoding: string-escape

My goal is to be abel to take a string like this:


And turn it into:


After I do the conversion, I'll probe to see if the string I have is encoded in UTF-8 or UTF-16.

Answer Source

You'll have to use unicode_escape instead:

>>> b"\\123omething special".decode('unicode_escape')

If you start with a str object instead (equivalent to the python 2.7 unicode) you'll need to encode to bytes first, then decode with unicode_escape.

If you need bytes as end result, you'll have to encode again to a suitable encoding (.encode('latin1') for example, if you need to preserve literal byte values; the first 255 unicode code points map 1-on-1).

Your example is actually UTF-16 data with escapes. Decode from unicode_escape, back to latin1 to preserve the bytes, then from utf-16-le (UTF 16 little endian without BOM):

>>> value = b's\\000u\\000p\\000p\\000o\\000r\\000t\\000@\\000p\\000s\\000i\\000l\\000o\\000c\\000.\\000c\\000o\\000m\\000'
>>> value.decode('unicode_escape').encode('latin1')  # convert to bytes
>>> _.decode('utf-16-le') # decode from UTF-16-LE