Apero Apero - 10 months ago 66
Python Question

Python oauth2client async

I am fighting with tornado and the official python oauth2client, gcloud... modules.

These modules accept an alternate http client passed with http=, as long as it has a method called

which can be called by any of these libraries, whenever an http request must be sent to google and/or to renew the access tokens using the refresh tokens.

I have created a simple class which has a
self.client = AsyncHttpClient()

Then in its
method, returns

My goal is to be able to
any of these libraries calls, so that tornado will execute them in asynchronously.

The thing is that they are highly dependant on what the default client - set to httplib2.Http() returns: (response, content)

I am really stuck and cannot find a clean way of making this async

If anyone already found a way, please help.

Thank you in advance

Answer Source

These libraries do not support asynchronous. The porting process is not always easy.


Depending on what you want to do maybe Tornado's GoogleOAuth2Mixin or tornado-alf will be enough.


Since I am not aware of any Tornado/asyncio implementation of gcloud-python, so you could:

  • you may write it yourself. Again it's not simple transport change of Connection.http or request, all the stuff around must be able to use/yield future/coroutines.

  • wrap it in ThreadPoolExecutor (as @Apero mentioned). This is high level API, so any nested api calls within that yield will be executed in same thread (not using the pool). It could work well.

  • external app (with ProcessPoolExecutor or Popen).

When I had similar problem with AWS couple years ago, I've ended up with executing, asynchronously, CLI (Tornado + subprocess.Popen + some cli (awscli, or boto based)) and simple cases (like S3, basic EC2 operations) with plain AsyncHTTPClient.