Peter Smit Peter Smit - 8 months ago 55
JSON Question

Python 3, let json object accept bytes or let urlopen output strings

With Python3 I am requesting from some url a json document.

response = urllib.request.urlopen(request)

object is a file like object with read, readline functions.

Normally a json object can be created with a file (opened in textmode)

obj = json.load(fp)

What I would like to do is:

obj = json.load(response)

this however does not work as urlopen returns a file object in binary mode.

A work around is of course:

str_response = response.readall().decode('utf-8')
obj = json.loads(str_response)

but this feels bad...

Is there a better way that I can transform a byte file object to a string file object? Or am I missing any parameters for either
to give an encoding?

This would look to me as a common use case so I'm confident I'm missing some usefull function.


HTTP sends bytes. If the resource in question is text, the character encoding is normally specified, either by the Content-Type HTTP header or by another mechanism (an RFC, HTML meta http-equiv,...).

urllib should know how to encode the bytes to a string, but it's too naïve—it's a horribly underpowered and un-Pythonic library.

Dive Into Python 3 provides an overview about the situation.

Your "work-around" is fine—although it feels wrong, it's the correct way to do it.