Lucas Ou Lucas Ou - 1 year ago 136
Python Question

django serving robots.txt efficiently

Here is my current method of serving robots.txt

url(r'^robots\.txt/$', TemplateView.as_view(template_name='robots.txt',

I don't think that this is the best way. I think it would be better if it were just a pure
static resource and served statically. But the way my django app is structured is that the static root and all subsequent static files are located in

Any thoughts? I'm amateur at django but


looks a lot more resource consuming than just a static call to my static directory which is served on nginx.

Answer Source

Yes, robots.txt should not be served by Django if the file is static. Try something like this in your Nginx config file:

location  /robots.txt {
    alias  /path/to/static/robots.txt;

See here for more info:

Same thing applies to the favicon.ico file if You have one.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download