Properly configuring a local development environment remains a steep challenge despite all the other advances in modern programming. There are simply too many variables: different computers, operating systems, versions of Django, virtual environment options, and so on. When you add in the challenge of working in a team environment where everyone needs to have the same set up the problem only magnifies.

In recent years a solution has emerged: Docker. Although only a few years old, Docker has quickly become the default choice for many developers working on production-level projects.

With Docker it’s finally possible to faithfully and dependably reproduce a production environment locally, everything from the proper Python version to installing Django and running additional services like a production-level database. This means it no longer matter if you are on a Mac, Windows, or Linux computer. Everything is running within Docker itself.

Docker also makes collaboration in teams exponentially easier. Gone are the days of sharing long, out-of-date README files for adding a new developer to a group project. Instead with Docker you simply share two files–a Dockerfile and docker-compose.yml file–and the developer can have confidence that their local development environment is exactly the same as the rest of the team.

Docker is not a perfect technology. It is still relatively new, complex under-the-hood, and under active development. But the promise that it aspires to–a consistent and shareable developer environment, that can be run either locally on any computer or deployed to any server–makes it a solid choice.

In this chapter we’ll learn a little bit more about Docker itself and “Dockerize” our first Django project.

What is Docker?

Docker is a way to isolate an entire operating system via Linux containers which are a type of virtualization. Virtualization has its roots at the beginning of computer science when large, expensive mainframe computers were the norm. How could multiple programmers use the same single machine? The answer was virtualization and specifically virtual machines which are complete copies of a computer system from the operating system on up.

If you rent space on a cloud provider like Amazon Web Services (AWS) they are typically not providing you with a dedicated piece of hardware. Instead you are sharing one physical server with other clients. But because each client has their virtual machine running on the server, it appears to the client as if they have their own server.

This technology is what makes it possible to quickly add or remove servers from a cloud provider. It’s largely software behind the scenes, not actual hardware being changed.

What’s the downside to a virtual machine? Size and speed. A typical guest operating system can easily take up 700MB of size. So if one physical server supports three virtual machines, that’s at least 2.1GB of disk space taken up along with separate needs for CPU and memory resources.

Enter Docker. The key idea is that most computers rely on the same Linux operating system, so what if we virtualized from the Linux layer up instead? Wouldn’t that provide a lightweight, faster way to duplicate much of the same functionality? The answer is yes. And in recent years Linux containers have become widely popular. For most applications–especially web applications–a virtual machine provides far more resources than are needed and a container is more than sufficient.

This, fundamentally, is what Docker is: a way to implement Linux containers!

An analogy we can use here is that of homes and apartments. Virtual Machines are like homes: stand-alone buildings with their own infrastructure including plumbing and heating, as well as a kitchen, bathrooms, bedrooms, and so on. Docker containers are like apartments: they share common infrastructure like plumbing and heating, but come in various sizes that match the exact needs of an owner.

Containers vs. Virtual Environments

As a Python programmer you should already familiar with the concept of virtual environments, which are a way to isolate Python packages. Thanks to virtual environments, one computer can run multiple projects locally. For example, Project A might use Python 3.4 and Django 1.11 among other dependencies; whereas Project B uses Python 3.7 and Django 2.2. By configuring a dedicated virtual environment for each project we can manage these different software packages while not polluting our global environment.

Confusingly there are multiple popular tools right now to implement virtual environments: everything from virtualenv to venv to Pipenv, but fundamentally they all do the same thing.

The important distinction between virtual environments and Docker is that virtual environments can only isolate Python packages. They cannot isolate non-Python software like a PostgreSQL or MySQL database. And they still rely on a global, system-level installation of Python (in other words, on your computer). The virtual environment points to an existing Python installation; it does not contain Python itself.

Linux containers go a step further and isolate the entire operating system, not just the Python parts. In other words, we will install Python itself within Docker as well as install and run a production-level database.

Docker itself is a complex topic and we won’t dive that deep into it in this book, however understanding its background and key components is important. If you’d like to learn more about it, I recommend the Dive into Docker video course.

Install Docker

Ok, enough theory. Let’s start using Docker and Django together. The first step is to sign up for a free account on Docker Hub and then install the Docker desktop app on your local machine:

This download might take some time to download as it is a big file! Feel free to stretch your legs at this point.

Once Docker is done installing we can confirm the correct version is running by typing the command docker --version on the command line. It should be at least version 18.

$ docker --version
Docker version 19.03.2, build 6a30dfc

Docker is often used with an additional tool, Docker Compose, to help automate commands. Docker Compose is included with Mac and Windows downloads but if you are on Linux you will need to add it manually. You can do this by running the command sudo pip install docker-compose after your Docker installation is complete.

To confirm Docker Compose is correctly installed run the command docker-compose --version.

$ docker-compose --version
docker-compose version 1.24.1, build 4667896b

Docker Hello, World

Docker ships with its own “Hello, World” image that is a helpful first step to run. On the command line type docker run hello-world. This will download an official Docker image and then run it within a container. We’ll discuss both images and containers in a moment.

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:b8ba256769a0ac28dd126d584e0a2011cd2877f3f76e093a7ae560f2a5301c00
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

The command docker info lets us inspect Docker. It will contain a lot of output but focus on the top lines which show we now have 1 container which is stopped and 1 image.

$ docker info
Client:
 Debug Mode: false

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 1
...

This means Docker is successfully installed and running.

Django Hello, World

Now we will create a Django “Hello, World” project that runs locally on our computer and then move it entirely within Docker so you can see how all the pieces fit together.

The first step is to choose a location for our code. This can be anywhere on your computer, but if you are on a Mac, an easy-to-find location is the Desktop. From the command line navigate to the Desktop and create a code directory for all the code examples in this book.

$ cd ~/Desktop
$ mkdir code && cd code

Then create a hello directory for this example and install Django using Pipenv which creates both a Pipfile and a Pipfile.lock file. Activate the virtual environment with the shell command.

$ mkdir hello && cd hello
$ pipenv install django==2.2.5
$ pipenv shell
(hello) $

A> If you need help installing Pipenv or Python 3 you can find more details here.

Now we can use the startproject command to create a new Django project called hello_project. Adding a period, ., at the end of the command is an optional step but one many Django developers do. Without the period Django adds an additional directory to the project; with the period it does not.

Finally use the migrate command to initialize the database and start the local web server with the runserver command.

(hello) $ django-admin startproject hello_project .
(hello) $ python manage.py migrate
(hello) $ python manage.py runserver

Assuming everything worked correctly you should now be able to navigate to see the Django Welcome page at http://127.0.0.1:8000/ in your web browser.

Django welcome page

Pages App

Now we will make a simple homepage by creating a dedicated pages app for it. Stop the local server by typing Control+c and then use the startapp command appending our desired pages name.

(hello) $ python manage.py startapp pages

Django automatically installs a new pages directory and several files for us. But even though the app has been created our hello_project won’t recognize it until we add it to the INSTALLED_APPS config within the hello_project/settings.py file.

Django loads apps from top to bottom so generally speaking it’s a good practice to add new apps below built-in apps they might rely on such as admin, auth, and all the rest.

Note that while it is possible to simply type the name of the app, pages, you are better off typing the full pages.apps.PagesConfig which opens up more possibilities in configuring apps.

# hello_project/settings.py
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'pages.apps.PagesConfig', # new
]

Now we can set the URL route for the pages app. Since we want our message to appear on the homepage we’ll use the empty string ''. Don’t forget to add the include import on the second line as well.

# hello_project/urls.py
from django.contrib import admin
from django.urls import path, include # new

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('pages.urls')), # new
]

Rather than set up a template at this point we can just hardcode a message in our view layer at pages/views.py which will output the string “Hello, World!”.

# pages/views.py
from django.http import HttpResponse


def home_page_view(request):
    return HttpResponse('Hello, World!')

What’s next? Our last step is to create a urls.py file within the pages app and link it to home_page_view. If you are on an Mac or Linux computer the touch command can be used from the command line to create new files. On Windows create the new file with your text editor.

(hello) $ touch pages/urls.py

Within your text editor import path on the top line, add the home_page_view, and then set its route to again be the empty string of ''. Note that we also provide an optional name, home, for this route which is a best practice.

# pages/urls.py
from django.urls import path

from .views import home_page_view


urlpatterns = [
    path('', home_page_view, name='home')
]

The full flow of our Django homepage is as follows:

  • when a user goes to the homepage they will first be routed to hello_project/urls.py
  • then routed to pages/urls.py
  • and finally directed to the home_page_view which returns the string “Hello, World!”

Our work is done for a basic homepage. Start up the local server again.

(hello) $ python manage.py runserver

If you refresh the web browser at http://127.0.0.1:8000/ it will now output our desired message.

Hello World

Now it’s time to switch to Docker. Stop the local server again with Control+c and exit our virtual environment since we no longer need it by typing exit.

(hello) $ exit
$

How do we know the virtual environment is no longer active? There will no longer be parentheses around the directory name on the command line prompt. Any normal Django commands you try to run at this point will fail. For example, try python manage.py runserver to see what happens.

$ python manage.py runserver
File "./manage.py", line 14
  ) from exc
       ^
SyntaxError: invalid syntax

This means we’re fully out of the virtual environment and ready for Docker.

Images, Containers, and the Docker Host

A Docker image is a snapshot in time of what a project contains. It is represented by a Dockerfile and is literally a list of instructions that must be built. A Docker container is a running instance of an image. To continue our apartment analogy from earlier, the image is the blueprint or set of plans for the apartment; the container is the actual, fully-built building.

The third core concept is the “Docker host” which is the underlying OS. It’s possible to have multiple containers running within a single Docker host. When we refer to code or processes running within Docker, that means they are running in the Docker host.

Let’s create our first Dockerfile to see all of this theory in action.

$ touch Dockerfile

Within the Dockerfile add the following code which we’ll walk through line-by-line below.

# Dockerfile

# Pull base image
FROM python:3.7

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /code

# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system

# Copy project
COPY . /code/

Dockerfiles are read from top-to-bottom when an image is created. The first instruction must be the FROM command which lets us import a base image to use for our image, in this case Python 3.7.

Then we use the ENV command to set two environment variables:

  • PYTHONUNBUFFERED ensures our console output looks familiar and is not buffered by Docker, which we don’t want
  • PYTHONDONTWRITEBYTECODE means Python wont try to write .pyc files which we also do not desire

Next we use WORKDIR to set a default work directory path called code which is where we will store our code. If we didn’t do this then each time we wanted to execute commands within our container we’d have to type in a long path. Instead Docker will just assume we mean to execute all commands from this directory.

For our dependencies we are using Pipenv so we copy over both the Pipfile and Pipfile.lock files into a /code/ directory in Docker.

It’s worth taking a moment to explain why Pipenv creates a Pipfile.lock, too. The concept of lock files is not unique to Python or Pipenv; in fact it is already present in package managers for most modern programming languages: Gemfile.lock in Ruby, yarn.lock in JavaScript, composer.lock in PHP, and so on. Pipenv was the first popular project to incorporate them into Python packaging.

The benefit of a lock file is that this leads to a deterministic build: no matter how many times you install the software packages, you’ll have the same result. Without a lock file that “locks down” the dependencies and their order, this is not necessarily the case. Which means that two team members who install the same list of software packages might have slightly different build installations.

When we’re working with Docker where there is code both locally on our computer and also within Docker, the potential for Pipfile.lock conflicts arises when updating software packages. We’ll explore this properly in the next chapter.

Moving along we use the RUN command to first install Pipenv and then pipenv install to install the software packages listed in our Pipfile.lock, currently just Django. It’s important to add the --system flag as well since by default Pipenv will look for a virtual environment in which to install any package, but since we’re within Docker now, technically there isn’t any virtual environment. In a way, the Docker container is our virtual environment and more. So we must use the --system flag to ensure our packages are available throughout all of Docker for us.

As the final step we copy over the rest of our local code into the /code/ directory within Docker. Why do we copy local code over twice, first the Pipfile and Pipfile.lock and then the rest? The reason is that images are created based on instructions top-down so we want things that change often–like our local code–to be last. That way we only have to regenerate that part of the image when a change happens, not reinstall everything each time there is a change. And since the software packages contained in our Pipfile and Pipfile.lock change infrequently, it makes sense to copy them over and install them earlier.

Our image instructions are now done so let’s build the image using the command docker build . The period, ., indicates the current directory is where to execute the command. There will be a lot of output here; I’ve only included the first two lines and the last three.

$ docker build .
Sending build context to Docker daemon  154.1kB
Step 1/7 : FROM python:3.7
...
Step 7/7 : COPY . /code/
 ---> a48b2acb1fcc
Successfully built a48b2acb1fcc

Moving on we now need to create a docker-compose.yml file to control how to run the container that will be built based upon our Dockerfile image.

$ touch docker-compose.yml

It will contain the following code.

# docker-compose.yml
version: '3.7'

services:
  web:
    build: .
    command: python /code/manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - 8000:8000

On the top line we specify the most recent version of Docker Compose which is currently 3.7. Don’t be confused by the fact that Python is also on version 3.7 at the moment; there is no overlap between the two! It’s just a coincidence.

Then we specify which services (or containers) we want to have running within our Docker host. It’s possible to have multiple services running, but for now we just have one for web. We specify how to build the container by saying, Look in the current directory . for the Dockerfile. Then within the container run the command to start up the local server.

The volumes mount automatically syncs the Docker filesystem with our local computer’s filesystem. This means that we don’t have to rebuild the image each time we change a single file!

Lastly we specify the ports to expose within Docker which will be 8000, which is the Django default.

If this is your first time using Docker, it is highly likely you are confused right now. But don’t worry. We’ll create multiple Docker images and containers over the course of this book and with practice the flow will start to make more sense. You’ll see we use very similar Dockerfile and docker-compose.yml files in each of our projects.

The final step is to run our Docker container using the command docker-compose up. This command will result in another long stream of output code on the command line.

$ docker-compose up
Creating network "hello_default" with the default driver
Building web
Step 1/7 : FROM python:3.7
...
Creating hello_web_1 ... done
Attaching to hello_web_1
web_1  | Performing system checks...
web_1  |
web_1  | System check identified no issues (0 silenced).
web_1  | September 20, 2019 - 17:21:57
web_1  | Django version 2.2.5, using settings 'hello_project.settings'
web_1  | Starting development server at http://0.0.0.0:8000/
web_1  | Quit the server with CONTROL-C.

To confirm it actually worked, go back to http://127.0.0.1:8000/ in your web browser. Refresh the page and the “Hello, World” page should still appear.

Django is now running purely within a Docker container. We are not working within a virtual environment locally. We did not execute the runserver command. All of our code now exists and our Django server is running within a self-contained Docker container. Success!

Stop the container with Control+c (press the “Control” and “c” button at the same time) and additionally type docker-compose down. Docker containers take up a lot of memory so it’s a good idea to stop them in this way when you’re done using them. Containers are meant to be stateless which is why we use volumes to copy our code over locally where it can be saved.

$ docker-compose down
Removing hello_web_1 ... done
Removing network hello_default

Git

Git is the version control system of choice these days and we’ll use it in this book. First, add a new Git file with git init, then check the status of changes, add updates, and include a commit message.

$ git init
$ git status
$ git add .
$ git commit -m 'ch1'

You can compare your code for this chapter with the official repository available on Github.

Conclusion

Docker is a self-contained environment that includes everything we need for local development: web services, databases, and more if we want. The general pattern will always be the same when using it with Django:

  • create a virtual environment locally and install Django
  • create a new project
  • exit the virtual environment
  • write a Dockerfile and then build the initial image
  • write a docker-compose.yml file and run the container with docker-compose up

We’ll build several more Django projects with Docker so this flow makes more sense, but that’s really all there is to it. In the next chapter we’ll create a new Django project using Docker and add PostgreSQL in a separate container as our database.