coriolinus coriolinus - 5 days ago 6
Python Question

Django doesn't see environment variables when deployed to Elastic Beanstalk

I'm trying to set up a Django/DRF application on Elastic Beanstalk, and for whatever reason, Django just can't see the desired environment variables. When I log in, I can see them just fine, by using

$ eb ssh
$ cat /opt/python/current/env


I can also see them, except relatively sensitive ones involving RDS, simply using
$eb printenv
.

All that appears to be set up and working properly. However, Django likes to read the environment immediately on starting up, and it appears that the environment variables just aren't set yet. I've experimented with simply inserting
print(os.environ)
in
settings.py
, and when I do that, I discover a whole bunch of environment variables which I don't need (i.e.
'SUPERVISOR_GROUP_NAME': 'httpd'
), and none of the ones I've set myself, like
DJ_SECRET_KEY
.

I've since changed the code to report the absence of specific environment variables when it loads the settings, and from a recent run, it generated this:

[Wed Nov 23 15:56:38.164153 2016] [:error] [pid 15708] DJ_SECRET_KEY not in environment; falling back to hardcoded value.
[Wed Nov 23 15:56:38.189717 2016] [:error] [pid 15708] RDS_DB_NAME not in environment; falling back to sqlite
[Wed Nov 23 15:56:38.189751 2016] [:error] [pid 15708] AWS_STORAGE_BUCKET_NAME not in environment; falling back to local static storage.


Again, those variables are set in the settings, and they show up with any other reporting tool EB gives me. They just aren't set in time for Django to read them when it launches and reads
settings.py
.

This looks pretty close to this issue, but it's not really the same: I know how to see / load the environment variables into the shell when ssh-ing into the eb instance; they're just not showing up when I need them to for the actual project.

This is almost exactly the issue I'm having, but the accepted-correct answer makes no sense to me, and the top-voted answer doesn't apply; those files are already in git.

How should I configure things so that Django can see the environment variables?

Answer

Given that the EB stores all these environment variables in a canonical location as a bash script, I ended up simply having bash execute the script, and updating the environment from the parsed results.

I created the file get_eb_env.py in parallel to my settings.py. Its main contents:

import os
import subprocess

ENV_PATH = '/opt/python/current/env'

def patch_environment(path=ENV_PATH):
    "Patch the current environment, os.environ, with the contents of the specified environment file."
    # mostly pulled from a very useful snippet: http://stackoverflow.com/a/3505826/504550
    command = ['bash', '-c', 'source {path} && env'.format(path=path)]

    proc = subprocess.Popen(command, stdout=subprocess.PIPE, universal_newlines=True)
    proc_stdout, _ = proc.communicate(timeout=5)

    # proc_stdout is just a big string, not a file-like object
    # we can't iterate directly over its lines.
    for line in proc_stdout.splitlines():
        (key, _, value) = line.partition("=")
        os.environ[key] = value

Then, I just import and call patch_environment() near the head of my settings.py.

Comments