Lucas Ou Lucas Ou - 1 month ago 5x
Python Question

django serving robots.txt efficiently

Here is my current method of serving robots.txt

url(r'^robots\.txt/$', TemplateView.as_view(template_name='robots.txt',

I don't think that this is the best way. I think it would be better if it were just a pure
static resource and served statically. But the way my django app is structured is that the static root and all subsequent static files are located in

Any thoughts? I'm amateur at django but


looks a lot more resource consuming than just a static call to my static directory which is served on nginx.


Yes, robots.txt should not be served by Django if the file is static. Try something like this in your Nginx config file:

location  /robots.txt {
    alias  /path/to/static/robots.txt;

See here for more info:

Same thing applies to the favicon.ico file if You have one.