oneloop oneloop - 7 months ago 22
Python Question

Python serving html async

I have a server serving HTML with nginx, uwsgi and flask. It's a simple webapp. When you refresh a page the webapp makes some API calls elsewhere and then treats the results returned from the APIs and generate the HTML which is then returned.

Now, most of the time is spend waiting for the APIs to respond. Because of this, I'm using gevent to make the API calls and monkey patching etc etc to make it all concurrent. If for every page refresh you need to make 3 API calls, you don't need to wait for the first call to return to initiate the second call. You initiate all three of them, wait for their responses, and then generate the HTML to be returned.

However, a separate question is whether or not the page refreshes from beginning to end are themselves concurrent. To test this I wrote a small concurrent script to make refreshes to my server:

from gevent import monkey
monkey.patch_all()

import requests, datetime

import gevent

def call_func(n):
before = datetime.datetime.now()
print 'initiating call '+str(n),before
requests.get('http://examplemyserver.com')
after = datetime.datetime.now()
print 'finishing call '+str(n),after, after-before

gevent_list = [ gevent.spawn(call_func, n=n) for n in range(10) ]

gevent.joinall(gevent_list)


Here are the results:

initiating call 0 2016-05-04 23:03:49.437540
initiating call 1 2016-05-04 23:03:49.446304
initiating call 2 2016-05-04 23:03:49.447911
initiating call 3 2016-05-04 23:03:49.450232
initiating call 4 2016-05-04 23:03:49.451774
initiating call 5 2016-05-04 23:03:49.453331
initiating call 6 2016-05-04 23:03:49.454759
initiating call 7 2016-05-04 23:03:49.456301
initiating call 8 2016-05-04 23:03:49.457663
initiating call 9 2016-05-04 23:03:49.458981
finishing call 0 2016-05-04 23:03:51.270078 0:00:01.832538
finishing call 6 2016-05-04 23:03:52.169842 0:00:02.715083
finishing call 3 2016-05-04 23:03:52.907892 0:00:03.457660
finishing call 1 2016-05-04 23:03:53.498008 0:00:04.051704
finishing call 8 2016-05-04 23:03:54.150188 0:00:04.692525
finishing call 4 2016-05-04 23:03:55.032089 0:00:05.580315
finishing call 7 2016-05-04 23:03:55.994570 0:00:06.538269
finishing call 2 2016-05-04 23:03:56.659146 0:00:07.211235
finishing call 9 2016-05-04 23:03:57.149156 0:00:07.690175
finishing call 5 2016-05-04 23:03:58.002210 0:00:08.548879


So you see, ten page refreshes are initiated virtually at the same time, however they are returned sequentially with a difference of about 1 second. This suggests that while internally the external API calls are done concurrently, until an HTTP response is given, no new HTTP responses are being accepted.

So, given that most of the webapp time is spent waiting for API responses, how can I make the whole process concurrent? Is it something that I need to change with nginx? With uwsgi? With flask? I have no idea about the architecture of these things.

Thanks.

EDIT 1:
Following Rob's idea, I replaced the server to be called with
'http://httpbin.org/delay/1'
. This is the result I obtain:

initiating call 0 2016-05-04 23:36:18.697813
initiating call 1 2016-05-04 23:36:18.706510
initiating call 2 2016-05-04 23:36:18.708645
initiating call 3 2016-05-04 23:36:18.711055
initiating call 4 2016-05-04 23:36:18.712679
initiating call 5 2016-05-04 23:36:18.714051
initiating call 6 2016-05-04 23:36:18.715471
initiating call 7 2016-05-04 23:36:18.717015
initiating call 8 2016-05-04 23:36:18.718412
initiating call 9 2016-05-04 23:36:18.720193
finishing call 0 2016-05-04 23:36:20.599483 0:00:01.901670
finishing call 2 2016-05-04 23:36:20.600829 0:00:01.892184
finishing call 8 2016-05-04 23:36:20.611292 0:00:01.892880
finishing call 5 2016-05-04 23:36:20.618842 0:00:01.904791
finishing call 7 2016-05-04 23:36:20.620065 0:00:01.903050
finishing call 6 2016-05-04 23:36:20.621344 0:00:01.905873
finishing call 1 2016-05-04 23:36:20.622407 0:00:01.915897
finishing call 4 2016-05-04 23:36:20.623392 0:00:01.910713
finishing call 9 2016-05-04 23:36:20.624236 0:00:01.904043
finishing call 3 2016-05-04 23:36:20.625075 0:00:01.914020


This is what you'd expect from a server that can take HTTP calls concurrently. So this shows that the above code that I used for the test is correct, and that the problem is indeed on the server. It's not taking HTTP calls concurrently.

EDIT2: I was playing around and checked that the simplest possible flask app has the same behavior as my server, so I'm giving that as my "minimal example"

Consider the following server:

import time

from flask import Flask, jsonify
app = Flask(__name__)

@app.route("/")
def hello():
time.sleep(2)
return ''

if __name__ == "__main__":
app.run(host='0.0.0.0', port=80, debug=True)


Now connect to it by taking the previous code and pointing it to
http://localhost
. The result is the following:

initiating call 0 2016-05-04 23:49:12.629250
initiating call 1 2016-05-04 23:49:12.638554
initiating call 2 2016-05-04 23:49:12.643844
initiating call 3 2016-05-04 23:49:12.645630
initiating call 4 2016-05-04 23:49:12.647866
initiating call 5 2016-05-04 23:49:12.649332
initiating call 6 2016-05-04 23:49:12.650853
initiating call 7 2016-05-04 23:49:12.652448
initiating call 8 2016-05-04 23:49:12.653865
initiating call 9 2016-05-04 23:49:12.655348
finishing call 0 2016-05-04 23:49:14.671281 0:00:02.042031
finishing call 1 2016-05-04 23:49:16.673649 0:00:04.035095
finishing call 2 2016-05-04 23:49:18.676576 0:00:06.032732
finishing call 3 2016-05-04 23:49:20.679746 0:00:08.034116
finishing call 4 2016-05-04 23:49:22.681615 0:00:10.033749
finishing call 5 2016-05-04 23:49:24.683817 0:00:12.034485
finishing call 6 2016-05-04 23:49:26.686309 0:00:14.035456
finishing call 7 2016-05-04 23:49:28.688079 0:00:16.035631
finishing call 8 2016-05-04 23:49:30.691382 0:00:18.037517
finishing call 9 2016-05-04 23:49:32.696471 0:00:20.041123


So basically a simple flask webapp does not take HTTP calls concurrently. How can I change this?

Answer

Short answer: use --processes=N and/or --threads=M when you start uwsgi.

Long answer: The stack you've described involved three components: nginx in the role of a reverse-proxy web server; uwsgi in the role of an application server; and flask as an application framework. The data flow occurs in that order, with the response flowing in the oppose order.

nginx is inherently multi-tasking and flask is inherently single-tasking. uwsgi can be configured to be either. By default, uwsgi is single-tasking.

Here is the setup I used to investigate this. My OS is Ubuntu 14.04.

/etc/nginx/sites-available/myapp:

server {
        listen 8100;
        location / {
                include uwsgi_params;
                uwsgi_pass unix:/tmp/uwsgi.sock;
        }
}

uwsgi command line:

One of:
  uwsgi -s /tmp/uwsgi.sock -w app:app
  uwsgi -s /tmp/uwsgi.sock -w app:app --processes=4
  uwsgi -s /tmp/uwsgi.sock -w app:app --threads=4

Followed by:
  chmod 777 /tmp/uwsgi.sock

app.py:

import time
from flask import Flask, render_template_string
app = Flask(__name__)

@app.route('/')
@app.route('/<name>')
def hello(name='World'):
    time.sleep(1)
    return render_template_string('''
        <html><body><p>
        Hello, <b>{{ name }}</b>!
        </p></body></html>''', name=name)

if __name__=="__main__":
    app.run(debug=True)

Client: I used your gevent/requests client from above, substituting the following url: http://localhost:8100/World.

Typical result:

initiating call 0 2016-05-04 21:08:16.473978
initiating call 1 2016-05-04 21:08:16.485589
initiating call 2 2016-05-04 21:08:16.487258
initiating call 3 2016-05-04 21:08:16.490140
initiating call 4 2016-05-04 21:08:16.493158
initiating call 5 2016-05-04 21:08:16.494417
initiating call 6 2016-05-04 21:08:16.495620
initiating call 7 2016-05-04 21:08:16.497180
initiating call 8 2016-05-04 21:08:16.498413
initiating call 9 2016-05-04 21:08:16.499528
finishing call 3 2016-05-04 21:08:17.512949 0:00:01.022809
finishing call 6 2016-05-04 21:08:17.515944 0:00:01.020324
finishing call 7 2016-05-04 21:08:17.517655 0:00:01.020475
finishing call 4 2016-05-04 21:08:17.519555 0:00:01.026397
finishing call 2 2016-05-04 21:08:18.520249 0:00:02.032991
finishing call 0 2016-05-04 21:08:18.522902 0:00:02.048924
finishing call 5 2016-05-04 21:08:18.524902 0:00:02.030485
finishing call 1 2016-05-04 21:08:18.526682 0:00:02.041093
finishing call 9 2016-05-04 21:08:19.526909 0:00:03.027381
finishing call 8 2016-05-04 21:08:19.528926 0:00:03.030513

References: