Learn Web Development with Python (4): Deploy Website with Docker, Gunicorn and Nginx

Introduction

In the previous three articles of this series, we used the Flask framework of Python to develop a simple weather forecast website. So far, our website is still limited to access on our own computer. In this article, we will use Docker, Gunicorn and Nginx to deploy this website, so that our website can be accessed on the public network.

Prerequisites

Already know how to develop a simple web application using the Flask framework (see “Learn Web Development with Python (1): Use Flask Framework”)
Already installed PostgreSQL database and know how to use SQLAlchemy to operate the database (see “Learn Web Development with Python (2): Use PostgreSQL and SQLAlchemy”)
Already installed Docker and docker-compose
Already have a domain name

Project Structure (Development Version)

Up to the previous article “Learn Web Development with Python (3): Use Input and API”, our project structure is as follows:

1
2
3
4
5
6


learn_flask
├── app.py
├── static
│   └── style.css
└── templates
    └── weather.html

In this article, we will use Docker and Gunicorn to deploy this website, so we need to add some files. For convenience, we will adjust the project structure to the following structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


learn_flask
├── docker-compose.yml
├── .env
├── .env.db
└── learn_flask
    ├── app
    │   ├── config.py
    │   ├── __init__.py
    │   ├── static
    │   │   └── style.css
    │   └── templates
    │       └── weather.html
    ├── Dockerfile
    ├── manage.py
    └── requirements.txt

Next we will create these files step by step and introduce the functions of these files.

Add Dependencies

Before developing this web application, we have defined a Python virtual environment, which contains the dependencies we need. We can use the following command to export the dependencies in the current environment to the requirements.txt file:

1

pip freeze > requirements.txt

For some reason, there are some unnecessary dependencies in it, such as click, itsdangerous, Jinja2, MarkupSafe, Werkzeug, we can manually delete these dependencies. Finally, our requirements.txt file is as follows:

1
2
3
4
5


Flask==3.0.0
Flask-SQLAlchemy==3.1.1
Jinja2==3.1.2
gunicorn==20.1.0
psycopg2-binary==2.9.9

Package the Application as a Python Package

We rename the original app.py file to __init__.py and move it to the learn_flask/app directory.

Then we can create a config.py file in the learn_flask/app directory to store our database configuration information:

1
2
3
4
5
6
7


import os

basedir = os.path.abspath(os.path.dirname(__file__))

class Config(object):
    SQLALCHEMY_DATABASE_URI = os.getenv("DATABASE_URL", "sqlite://")
    SQLALCHEMY_TRACK_MODIFICATIONS = False

This way we can delete the database configuration information in __init__.py:

1
2


app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://test:test_password@localhost:5432/weather_db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

We create a manage.py file outside the app directory to manage our application. We can add the following code to manage.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


from flask.cli import FlaskGroup
from sqlalchemy.exc import ProgrammingError
from app import app

cli = FlaskGroup(app)

@cli.command("create_db")
def create_db():
    from app import db
    from app import Weather
    with app.app_context():
        try:
            db.create_all()
            db.session.commit()
        except ProgrammingError:
            pass

@cli.command("drop_db")
def drop_db():
    from app import db
    from app import Weather
    with app.app_context():
        try:
            db.drop_all()
            db.session.commit()
        except ProgrammingError:
            pass

if __name__ == "__main__":
    cli()

Here we use FlaskGroup to manage our application, and the create_db and drop_db commands are used to create and delete the database. Here we use ProgrammingError to determine whether the database exists. If the database does not exist, the database will not be deleted.

Then we can delete the code for creating and deleting the database in __init__.py:

1
2
3
4
5
6
7
8


if __name__ == '__main__':
    with app.app_context():
        try:
            db.create_all()
            db.session.commit()
        except ProgrammingError:
            pass
    app.run()

Deploy the Application with Docker

Create Dockerfile

We create a Dockerfile file under the learn_flask directory, which is used to build our application. We can add the following code to Dockerfile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


# pull official base image
FROM python:3.11.4-slim-buster as builder

# set work directory
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app

# Install any needed packages specified in requirements.txt
COPY ./requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Run app.py when the container launches
CMD ["gunicorn", "-b", "0.0.0.0:5001", "app:app"]

The function of each command is marked in the comments.

Create docker-compose.yml

We create a docker-compose.yml file under the learn_flask directory, which is used to manage our application. We can add the following code to docker-compose.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


version: "3.8"

services:
  learn_flask:
    build: ./learn_flask
    command: python manage.py run -h 0.0.0.0
    volumes:
      - ./learn_flask:/app/
    ports:
      - 5001:5000
    env_file:
      - ./.env
    depends_on:
      - db
  db:
    image: postgres:13
    volumes:
      - ./postgres_data_prod:/var/lib/postgresql/data/
    env_file:
      - ./.env.db

volumes:
  postgres_data:

In this docker-compose.yml file, we define two services, one is the application learn_flask, and the other is the database db.

Create .env File

In the above docker-compose.yml file, we used the .env file and .env.db file, we can create these two files under the learn_flask directory to store our environment variables. We can add the following environment variables to the .env file:

1
2
3
4
5
6


FLASK_APP=app/__init__.py
FLASK_DEBUG=1
DATABASE_URL=postgresql://your_postgre_user_name:your_postgres_password@db:5432/weather_db
SQL_HOST=db
SQL_PORT=5432
DATABASE=postgres

Add database environment variables to the .env.db file:

1
2
3


POSTGRES_USER=your_postgre_user_name
POSTGRES_PASSWORD=your_postgres_password
POSTGRES_DB=weather_db

Deploy the Application (Development Version)

Build the Image

When building the image for the first time, our application has not yet created the database, so we need to create the database first. We need to change the command in docker-compose.yml to the following code:

1
2
3
4


command: >
    sh -c "python manage.py drop_db &&
    python manage.py create_db && 
    python manage.py run -h    

Then we can use the following command to build the image:

1

docker-compose up -d --build

If everything goes well, you can access our website in the browser by entering http://localhost:5001.

Deploy the Application

After the first successful build, we can temporarily stop the application, and then change the command in docker-compose.yml back to the original command:

1

command: python manage.py run -h 0.0.0.0

Then we can use the following command to deploy the application:

1

docker-compose up -d

Deploy the Application (Production Version)

After completing the deployment according to the above steps, our program is still running in development mode. We can see the following prompt information in the docker log:

1

WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

This prompts us that the above deployment is not enough for the production environment, so we need to make some adjustments to make our application run in the production environment.

docker-compose

First, we create a new docker-compose.yml file for the production environment deployment. We can create a docker-compose.prod.yml file under the learn_flask directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


version: "3.8"

services:
  learn_flask:
    build:
      context: ./learn_flask
      dockerfile: Dockerfile.prod
    command: gunicorn --bind 0.0.0.0:5000 manage:app
    ports:
      - 5001:5000
    env_file:
      - ./.env.prod
    depends_on:
      - db
  db:
    image: postgres:13
    volumes:
      - postgres_data_prod:/var/lib/postgresql/data/
    env_file:
      - ./.env.prod.db

volumes:
  postgres_data_prod:

Note that we no longer specify volumes for the application, because we no longer need to mount the code of the application to the container, but package the application into the image.

Dockerfile

In the docker-compose.prod.yml file above, we used a new Dockerfile.prod file, we can create a Dockerfile.prod file under the learn_flask/learn_flask directory to build our application image:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69


###########
# BUILDER #
###########

# pull official base image
FROM python:3.11.3-slim-buster as builder

# set work directory
WORKDIR /usr/src/app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install system dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc

# lint
RUN pip install --upgrade pip
# RUN pip install flake8==6.0.0
COPY . /usr/src/app/
# RUN flake8 --ignore=E501,F401 .

# install python dependencies
COPY ./requirements.txt .
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/app/wheels -r requirements.txt


#########
# FINAL #
#########

# pull official base image
FROM python:3.11.3-slim-buster

# create directory for the app user
RUN mkdir -p /home/app

# create the app user
RUN addgroup --system app && adduser --system --group app

# create the appropriate directories
ENV HOME=/home/app
ENV APP_HOME=/home/app/learn_flask
RUN mkdir $APP_HOME
WORKDIR $APP_HOME

# install dependencies
RUN apt-get update && apt-get install -y --no-install-recommends netcat
COPY --from=builder /usr/src/app/wheels /wheels
COPY --from=builder /usr/src/app/requirements.txt .
RUN pip install --upgrade pip
RUN pip install --no-cache /wheels/*

# copy entrypoint-prod.sh
COPY ./entrypoint.prod.sh $APP_HOME

# copy project
COPY . $APP_HOME

# chown all the files to the app user
RUN chown -R app:app $APP_HOME

# change to the app user
USER app

# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/learn_flask/entrypoint.prod.sh"]

entrypoint

In the Dockerfile.prod file above, we used an entrypoint.prod.sh file, we can create an entrypoint.prod.sh file under the learn_flask/learn_flask directory to run our application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

if [ "$FLASK_DEBUG" = "1" ]
then
    echo "Creating the database tables..."
    python manage.py create_db
    echo "Tables created"
fi

exec "$@"

Note: After creating the entrypoint.prod.sh file, we need to change its permissions to executable, otherwise an error will be reported when running docker later:

1

chmod +x entrypoint.prod.sh

Environment Variables

Similar to deploying in the development environment, we need to create .env.prod and .env.prod.db files to store the environment variables for the production environment. We can add the following environment variables to the .env.prod file:

1
2
3
4
5
6


FLASK_APP=app/__init__.py
FLASK_DEBUG=0
DATABASE_URL=postgresql://your_postgre_user_name:your_postgres_password@db:5432/weather_db_prod
SQL_HOST=db
SQL_PORT=5432
DATABASE=postgres

Add database environment variables to the .env.prod.db file:

1
2
3


POSTGRES_USER=your_postgre_user_name
POSTGRES_PASSWORD=your_postgres_password
POSTGRES_DB=weather_db_prod

Build the Image

We can use the following command to build the image:

1

sudo docker-compose -f docker-compose.prod.yml up -d --build

At this time, we only built the image, and the database is still empty, so we need to create the database in the container. We can use the following command to create the database in the container:

1

sudo docker-compose -f docker-compose.prod.yml exec learn_flask python manage.py create_db

Then we can enter http://localhost:5001 in the browser to see our website.

Project Structure (Production Version)

Finally, our directory structure for the production environment is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


learn_flask
├── docker-compose.prod.yml
├── env.prod
├── env.prod.db
└── learn_flask
    ├── app
    │   ├── config.py
    │   ├── __init__.py
    │   ├── static
    │   │   └── style.css
    │   └── templates
    │       └── weather.html
    ├── Dockerfile.prod
    ├── entrypoint.prod.sh
    ├── manage.py
    └── requirements.txt

Use Nginx Reverse Proxy

After completing the above deployment, we can still only access our website locally. If we want to access it on the public network, we need to use Nginx reverse proxy. Since Nginx has been installed on my server, I only need to add a configuration file on the server. We can create a learn_flask.conf file under the /etc/nginx/sites-available directory on the server to store our configuration information. We can add the following code to the learn_flask.conf file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


upstream learn_flask.jinli.io {  
    server 127.0.0.1:5001; 
}

server {
    listen 80;
    server_name  learn_flask.jinli.io;
    return 301 https://learn_flask.jinli.io$request_uri;
}

server {
    listen 443 ssl;
    server_name  learn_flask.jinli.io;

    ssl_certificate /media/lijin/learn_flask/cert/cert.pem;
    ssl_certificate_key /media/lijin/learn_flask/cert/key.pem;

    location / {
        proxy_redirect off;
        proxy_pass http://learn_flask.jinli.io;

        proxy_set_header  Host                $http_host;
        proxy_set_header  X-Real-IP           $remote_addr;
        proxy_set_header  X-Forwarded-Ssl     on;
        proxy_set_header  X-Forwarded-For     $proxy_add_x_forwarded_for;
        proxy_set_header  X-Forwarded-Proto   $scheme;
        proxy_set_header  X-Frame-Options     SAMEORIGIN;

	    client_max_body_size 100m;
    }
}

Then we can use the following command to activate this configuration file:

1

sudo service nginx reload