Using VirtualEnv with Obscured Django Settings

Welcome back in today’s event loop on {{ PLP }}, we’re about to start the advanced discussions on developing a Django powered sites and how we can cover the most classified information from the Django settings.py with a better sense of security practice among developers.

While, previously we’ve learned about The Automatic Admin Interface – Django Admin Site with the actual Django Administration for better illustration and observe how exactly it works, but, for now, I can only access it to manage the back-end system due to some sensitive information.

In this event loop, we have to discuss two significant advanced topics that would change completely the way we handle the Django projects in a much-matured way as I’ve assumed and confident that you can follow and understand easier as you’ve gained a foothold from our various discussions.

Introduction

The importance of obscuring some of the classified information from the dev/dev/settings.py which are the SECRET_KEY and the DATABASES section of it. Thus, these are the most highly classified piece of information that we wanted to shift out to a different file located at the secured directory from the web server.

However, the sense of isolating the Python environment is the most ideal way when working with Python, you know why? it’s because, right now, all of our previous installations is a system-wide, meaning, we’re not using a virtual environment to store the installed software, third-party libraries, etc.

Thus, above all, we’re working on various Django projects with different library versions and specifications to run from the same web server and practically, without the virtual environment to isolate Python, it’s hard to maintain various Django projects in one web server by itself.

So, prepare your self once again and stay focus because this would be exciting and fun learning with {{ PLP }}.

Getting Started

In the meantime, we can’t isolate Python environments without using the VirtualEnv tool which helps us to have it’s own installation directories that are contained within a virtual environment, meaning, once we start installing Python, Django, and the rest of its libraries from one VirtualEnv to the other VirtualEnv environments that can’t be shared its own installed libraries nor has the access to the system-wide libraries.

Though, the sensitive information from the settings.py must be obscured from the unwanted users to relish, dilute, and possible misuse it. But, certainly, we must do something to preserve it.

Step 1: Obscured the Django SECRET_KEY

Now, open up the dev/dev/settings.py and look for the SECRET_KEY, and then COPY the long string value of the SECRET_KEY and PASTE it to a new text file called the django_secret_key.txt and upload it to your secured directory from your web server. But, of course, you can have your own naming conventions alike.

1
/root/dev/config

Now, in my case, I created a new folder directory called config, from my web server using the FileZilla as usual.

However, we need to modify the SECRET_KEY settings to read the django_secret_key.txt file’s content to load it directly from the settings.py

1
2
3
4
5
# SECURITY WARNING: keep the secret key used in production secret!
path = '/root/dev/config/django_secret_key.txt'
f = open(path, "r")
SECRET_KEY = f.read()
f.close()

While, it’s very simple and straightforward approach and easy to maintain the SECRET_KEY value that’s now stored from the text file located at the web server. In this way, we avoid the prying eyes to easily grab the Django secret key or even accidentally shared the settings.py from a public view and that’s the worst thing we can imagine for now.

Step 2: Obscured the Django DATABASES

Moreover, the DATABASES section from the Django settings.py must change as well, this is the most classified information that we must protect, otherwise, the production database would be easily diluted by the unauthorized access.

Meanwhile, the original DATABASES setting would look like this.

1
2
3
4
5
6
7
8
9
10
DATABASES = {
  'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'db_name',
        'USER': 'db_user_name',
        'PASSWORD': 'db_password_here',
        'HOST': 'localhost',
        'PORT': '3306',
    },
}

Hence, the naive database config is absolutely an open book shall I say, it’s not good at all especially if that’s a production database settings.

Likewise, we have to address this issue and shift out the highly classified database information to a much-secured place from our web server. Now, create a new file called the default.cnf and this file contents would look like the below code snippets.

1
2
3
4
5
6
[client]
database = db_name
user = db_user_name
password = db_password_here
host = localhost
port = 3306

Now, the DATABASES section of the Django settings must change the way the code snippets below must follow.

1
2
3
4
5
6
7
8
DATABASES = {
  'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/root/dev/config/default.cnf',
        },
    },
}

It actually reads the /root/dev/config/default.cnf from the web server directory and loads it accordingly. Thus, it’s more secure and fewer worries at our side.

Another example if you’ve multiple databases set up with obscured database settings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/root/dev/config/default.cnf',
        },
    },
    'db_1': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/root/dev/config/db_1.cnf',
        },
    },
    'db_2': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': '/root/dev/config/db_2.cnf',
        },
    },
}

Although, you can use different database engines here such as the PostgreSQL, Oracle, etc. You can read more about the Django Databases connections as well, but for now, we’re focusing on MySQL database as our primary database throughout this guide.

Step 3: Install VirtualEnv

First of all, to start isolating the Python environments, we need to install the VirtualEnv from the system-wide so we can always use this tool every time we need to create a new virtual environment for other Django projects outside of the isolated Python environment.

1
pip3 install virtualenv

Now, head over to your web console and then execute the command above to install the VirtualEnv. Next, move out to the dev directory and go back to the main root directory by giving a command below.

1
cd

Afterward, we’re about to create a new virtual environment from the main root directory or it’s up to your choice, but I personally recommend to go back to the main root and create a new VirtualEnv there.

1
virtualenv env_dev

The env_dev is the new folder directory where the default VirtualEnv folders and structures are all there after the successful execution of the command.

Step 4: Activate the VirtualEnv

Hence, we need to activate the VirtualEnv in order for us to use the isolated Python environment. Now, navigate to the env_dev which is the newly created VirtualEnv so we can start installing back the Django package and the rest of the libraries.

1
(env_dev) root@localhost:~# source env_dev/bin/activate

In case you have noticed something, there is an (env_dev) after activated, meaning, always activate the specific VirtualEnv before using it, otherwise, if you install something, it will not be installed from that VirtualEnv, but goes to the system-wide.

In contrast, you can deactivate the VirtualEnv which is currently running by executing the command below.

1
deactivate

And the (env_dev) will be gone and back to normal which is the [email protected]:~# only.

In the meantime, we need to activate again the env_dev isolated Python environment so we can install everything again to make it fully functional for our Django project.

Step 5: Install the Latest Python and the requirements.txt

Now, once the env_dev VirtualEnv has been activated, execute the following commands below to install the libraries listed from the requirements.txt and the Python to the latest versions as possible.

But, of course, it’s up to your own version and requirements for your Django projects, that’s why, we have installed the VirtualEnv for the incompatibilities sake.

1
2
sudo apt-get update && apt-get upgrade
sudo apt-get install python3.7

And then once the UBUNTU system has been upgraded, next would be the latest Python which is currently at Python3.7 as of this writing. So anyway, you can now install all the library dependencies from the dev/requirements.txt.

Make it sure, that you activate first the specific VirtualEnv, in this case, I need to activate the env_dev virtual environment.

1
pip3 install -r dev/requirements.txt

Surely, this command will automatically install everything from the requirements.txt file, awesome right?

Step 6: Remove the Previous Django Installed at the System-wide level (optional)

Since, this is an optional step to do, but you might think it’s not necessary anymore to have the Django installed at the system-wide level, you can probably remove it using the pip3 and then uninstall the Django package. In case, you’re still on an activate VirtualEnv, just deactivate first before you uninstall the Django, because, we don’t want the Django installed from the VirtualEnv to be removed instead.

1
pip3 freeze

First of all, after deactivating the VirtualEnv, must go back to the main root directory by issuing a cd command. Next, we need to check all the installed libraries, by giving the pip3 freeze command, seeing that, if the Django is present from the list, therefore, we can uninstall it.

1
pip3 uninstall Django

Afterward, you can execute the command pip3 freeze once again to check if Django package is still there or not. Sure thing, it will be removed.

Step 7: Test the new VirtualEnv

For the most part, we need to test out the brand new VirtualEnv that we have created earlier, in my case, I remove the Django at the system-wide level to illustrate that we’re now running with the VirtualEnv to isolate the Python environment. Now, activate the env_dev virtual environment first.

1
source env_dev/bin/activate

And then navivate to the dev Django project folder to run the daphne web service once again.

1
(env_dev) root@localhost:~# cd dev

Afterward, execute the command to run the daphne web service.

1
(env_dev) root@localhost:~/dev#  daphne -b 172.104.190.249 -p 8000 dev.asgi:application

Finally, we’re now using the VirtualEnv tool to isolate every Python and Django projects we have, you can create multiple VirtualEnv directories from your web server with different versions of Python and Django and it’s dependencies in a single web server machine.

Indeed, Python environment is very flexible and easy to maintain and can save you a lot of overheads when it comes to managing your own web server and it’s compatibility with the help of the VirtualEnv tool that’s made for Python ecosystem that really works in isolating the Python environment.

You can download the source code from our GitHub repositories at https://github.com/pinoylearnpython/dev and stay tuned for any updates.

In the next event loop on {{ PLP }}.

Congratulations!, we are now running with an isolated Python environment using the VirtualEnv tool and we don’t bother much from the number one overheads in each developer and system administrators mind which is the Python’s library compatibility issues, and of course we can save a lot of resources that we can utilize especially using the single web server machine itself.

Thus, obscuring the highly classified information from the Django settings.py is a highly recommended practice and a responsibility in every developer to protect the most sensitive information with most especially the Django SECRET_KEY and the DATABASES configurations.

For those who’re not able to successfully launch your VirtualEnv and obscuring your Django settings.py tutorial or you need more clarifications, don’t worry, leave a comment below and I’m happy to help you to succeed.

See you in the next event loop on Django User Authentication System with reCAPTCHA v3, JQuery and AJAX Method.

That’s all, have fun learning with {{ PLP }}.

To help Filipino students to learn Python programming language with Django to enhance their capabilities in developing robust web-based applications with practical and direct to the point tutorials, step-by-step with actual information that I provided for you. Leave a comment below or email me at [email protected], thank you!