If you want to save time testing your Python code in many settings before publishing/deploying your packages automatically, creating CI/CD pipelines is a beautiful way to go. It’s also an excellent approach to catch bugs early and guarantee that your development process is consistent and repeatable.
Create an account and link your Github or Bitbucket account to start a Travis CI build. After that, you’ll be able to choose which projects Travis CI should work on. Every time you push to the master branch or open a pull request, a build is initiated. Travis CI will additionally provide the build results in the pull request.
We decided to deploy the Django application to Heroku. During this process, we discovered that Travis CI has a CLI that allows quick configuration of the .travis.yml for continuous delivery after successful builds. This CLI is open source and available via this Github repo. Within your .travis.yml, deployments can grow reasonably intricate, but a successful build, in our opinion, should distribute the program to a single location. In the future, we may deploy master to a staging environment before deploying a production branch to the live environment.
The intention is to create a CI/CD pipeline that would accomplish the following:
- At each merging request, the code is automatically tested.
- Calculating and presenting the master branch’s test coverage.
- If a build on the staging branch passes the tests, the python package/wheel is automatically deployed to PyPi.
This article intends to test a Django & Postgres web project using Travis continuous integration and deployment to Heroku as a fundamental approach to continuous integration and deployment. To accomplish this, we utilized Github, Travis-CI, which are all free to use for open-source projects. We also utilized Heroku for the hosting.
Components utilized in summary include:
- Github: For code hosting and version control.
- Travis CI: For building and deploying applications to Heroku.
- Heroku: For hosting the Django application, use Heroku.
Prerequisite:
We hope you have Github, Travis, and Heroku accounts set up and configured in this article. A few examples are linking Travis to Github, setting up a Postgres server in Heroku, and configuring OS environment variables in Travis and Heroku websites.
The processes in this pipeline are common to all Django apps. Hence we center on setting up the CI/CD pipeline rather than the Django application. First, however, we’ve included some comments on the application to provide some background.
Which Postgres copy should I use at each stage of the workflow?
The local Postgres database server is used during development. When testing in Travis, we’ll use Travis’s Postgres, and when deploying, we’ll need to use Heroku’s Postgres.
makemigrations
Before deploying to Heroku or, in our case, before pushing to Github, consistently execute python manage.py makemigrations.
The deploy portion in the.travis.yml file will perform the python manage.py migration for the Postgres server addon from Heroku.
settings.py
The database credentials must be made available to Travis and Heroku as OS environment variables under the database section. They can be placed on their web pages.
To the settings.py file, add and/or update the following:
import os
import environ
import django_heroku
import dj_database_url
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware', # new
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ['DATABASE_HOST'],
'PORT': os.environ['DATABASE_PORT'],
}
}
or
Settings specific to MySql
If you opt to use MySQL, this is the right configuration. We’re storing our data in a MySQL database, a popular use-case for Django applications. Subsequently, install mysqlclient as an additional project dependency. The latter is a connector that allows the application to communicate with the database.
import os
if 'RDS_HOSTNAME' in os.environ:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
}
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
django_heroku.settings(locals())
Requirements.txt
Dependencies are included in the requirements.txt. This file informs Travis of the app’s package requirements.
psycopg2 psycopg2-binary dj-database-url==0.5.0 gunicorn whitenoise django-heroku pytz sqlparse
Django Temperate Conversion application
It is a Django application that converts temperatures from degrees Celcius to Fahrenheit. It is the application we will be deploying to Heroku using TravisCI. Below are the key parts of the application you should have if you want to follow along.
# temperature/urls.py
from django.urls import path
from .views import convert
app_name = 'temperature'
urlpatterns = [
path('temperature/',convert, name='temperature-conversion'),
]
# temperature/views.py
from django.shortcuts import render
from django.http import JsonResponse
def convert(request):
temp_in_celcius =float(request.GET.get('temperature_in_celcius'))
temp_in_fahrenheit= float((temp_in_celcius * 9/5) + 32)
data = {
'temp_in_fahrenheit': temp_in_fahrenheit,
}
return JsonResponse(data)
# temperature/tests.py
from django.test import TestCase
from django.test import TestCase, Client
from django.urls import reverse
class TestTemperatureConversion(TestCase):
"""
This class contains tests that convert temperature from Celcius
to Fahrenheit.
"""
def setUp(self):
"""
It runs before any execution in each test case.
"""
self.client = Client()
self.url = reverse("temperature:temperature-conversion")
def test_celcius_to_fahrenheit_conversion(self):
"""
Tests conversion of celcius to Fahrenheit.
"""
celsius_1 = 25.8
data = {
"temperature_in_celcius": celsius_1,
}
response = self.client.get(self.url, data)
self.assertContains(response, 78.44)
# temperature_converter/urls.py
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('', include('temperature.urls')),
]
# temperature_converter/settings.py
import os
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir.'
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.0/howto/deployment/checklist/
# SECURITY WARNING: Always ensure that the secret key is private, especially on production!
SECRET_KEY = 'django-insecure-w(j%%%%yy$msdk1)yx-e_mq67klolpe@lo@9qi*rwkh&bygae90='
# SECURITY WARNING: While on production, turn Debug to False!
DEBUG = True
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware', # new
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'temperature_converter.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'temperature_converter.wsgi.application'
# Database
# https://docs.djangoproject.com/en/4.0/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ['DATABASE_NAME'],
'USER': os.environ['DATABASE_USER'],
'PASSWORD': os.environ['DATABASE_PASSWORD'],
'HOST': os.environ['DATABASE_HOST'],
'PORT': os.environ['DATABASE_PORT'],
}
}
# Password validation
# https://docs.djangoproject.com/en/4.0/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/4.0/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.0/howto/static-files/
STATIC_URL = 'static/'
# Default primary key field type
# https://docs.djangoproject.com/en/4.0/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
Using TravisCI to deploy
Getting started using GitHub with Travis CI
First, head to Travis-ci.com and Sign up for an account using your GitHub account. Accept Travis CI’s Authorization. The last-mentioned will redirect you to GitHub.
Please read on if you have any questions about the Travis CI GitHub Authorized OAuth App access rights notice. By clicking on your profile image located on the top right section of the Dashboard in Travis CI to choose the repositories, you wish to integrate with Travis CI.
To do so, click on the Settings and the green Activate button.
You may also activate all your repositories by clicking the Activate all repositories using the GitHub Apps button on the getting started page. To inform Travis CI what to do, add a.travis.yml file to your repository. The following example specifies a Python project developed using Python 3.6 and the most recent Django versions.
Add the .travis.yml file to git, commit, and push to trigger a Travis CI build. Travis only executes on commits that have a.travis.yml file attached to them. By accessing Travis CI and selecting your repository, you can check the build status page to see if your build passes or fails based on the return status of the build command.
Access privileges to Travis CI’s GitHub OAuth App
When you initially sign in to travis-ci.com using GitHub, you’ll see a notice from GitHub that says:
” Travis CI by Travis-pro is requesting access to your [account name].“
and it will say in the repositories section:
“This app will read and write data from both public and private repositories.“
Travis CI does not automatically access all of your repositories. Travis CI OAuth application will appear in your GitHub Authorized OAuth Apps list once you accept the access rights. Still, you must specifically configure which repositories Travis CI has access to within your travis-ci.com account. When you activate Travis CI for your repositories, you’ll be prompted to configure it. You can choose between using the ‘All repositories’ option or the ‘Only select repositories’ option during the activation procedure, as shown above.
The Travis CI GitHub Application is installed in the Installed GitHub Apps area once the Travis CI activation is complete.
OAuth permissions are used in Travis CI in the following way:
- The Travis CI system syncs some metadata with GitHub. This info is essential for the service to work correctly. Users, orgs, memberships, repos, permissions, and (optionally) branches are synced to GitHub via the Checks API. This form of sync occurs either daily or at the user’s request.
- Travis CI’s mechanism clones a repository from which the build is triggered to the built environment to conduct builds. The built environment is a standalone virtual machine or LXD container terminated once the build is complete. Cloning occurs only after a build request has been made, and thus only for the repositories that have been expressly enabled in the GitHub settings.
- Travis CI’s system collects and processes the .travis.yml config file from the repository and the branch mentioned explicitly in the build request, prompted by GitHub, to set up a build environment and prepare the build.
- Travis CI’s system reports Travis CI’s build results.
Configuration of Travis CI
Let’s start by creating a travis.yml build file based on the documentation’s sample configuration file.
.travis.yml
This file includes Travis’ instructions and is required when Travis begins running. The Travis website can generate $HEROKU_API_KEY from the Heroku website and store it as an OS environment variable. Travis’s copy of Postgres is used for the test.
language: python
python:
- 3.6
services:
- postgresql
install:
- pip install -r requirements.txt
script:
- python manage.py test
deploy:
provider: heroku
api_key: $HEROKU_API_KEY
app: webapp-dpth
run: python manage.py migrate
on: main
The key reminders when dealing with the .travis.yml file include: A test, such as pytest, is required; otherwise, Travis CI would reject the build. Travis CI needs the access key and secret key to deploy the application on Heroku. As per best practices, these keys are specified as environment variables on Travis CI instead of hard-coded in the.travis.yml file.
On Travis CI, enable the build for the specific repository.
Procfile
It is a Heroku file. It instructs Heroku to use Gunicorn as the production server when deploying the web app. It’s worth noting that temperate_conversion is the name of the Django project.
web: gunicorn temperature_conversion.wsgi
Running the application for the first time
There is a need to adjust a few settings upon the initial run. By ssh’ing into the instance, you are manually specifying these parameters. These may quickly be written and included in TravisCI deployment as a post-install script.
source /opt/python/run/venv/bin/activate source /opt/python/current/env cd /opt/python/current/app python manage.py migrate
It builds the tables supplied in Django’s “INSTALLED_APPS” configuration. Create a user to manage the application’s admin interface.
python manage.py createsuperuser
Put the pipeline to test
We’re now ready to put our pipeline through its paces from beginning to end.
- Modify your codebase.
- Changes should be committed and pushed to GitHub.
- Travis CI detects the change in the code and initiates a build.
- The new version of the application is deployed on Heroku using Travis CI.
The Deployed web application
Push to Github with the above files in place, and Travis will begin testing. Deployment begins after all tests have passed. If no errors occur, the web application will execute on:
https://tempconvertertest.herokuapp.com/
Security on the internet
Please note that web security has not been thoroughly considered in this basic approach described above. Therefore, do not use this workflow listed above for production. The SECRET_KEY in settings.py, for example, isn’t addressed at all, and web security is far beyond the scope of this article.
Conclusion
Travis CI is a system for continuously integrating and deploying on a hosted online system. On travis-ci.com, you can now test and deploy open source and private projects!
You’ll be able to use the GitHub Apps Integration right away, and you’ll have access to all of your repositories in one spot.
