How to use Django import-export library

Interacting and mastering Python code has never been more flexible, thanks to the existence of the reputable Django framework. This Python framework makes it easy for a developer to define project files and work with them from different directory levels.

Django-import-export is a library that adds more flavor to how Django handles data. It supports all the data formats under tablib and JSON, Xls, YAML, and CSV. Moreover, it also extends its functionality to the Django Admin side, which caters to your back-end data management. Therefore, you can flexibly apply this library’s import-export functionality to the already stored data within your project.

Installing and Configuring Django-Import-Export

The first step is to get your Django affairs in order. You need to ensure you are using the right Python version and that it is properly installed on your system. You also need to have a virtual environment installed on your system too. It greatly aids in the isolated executions of your different project files not to run into unnecessary runtime conflicts.

Installing Python 3

Ubuntu 16.04 and earlier versions come preinstalled with Python. Therefore, there is no need to relive the basics of installing Python on your operating system. Since we only want to deal with Python 3, we need to update and upgrade the installed apps and services on our operating system so that the latest Python version gets bundled up inside it.

tuts@codeunderscored:~$  sudo apt update
tuts@codeunderscored:~$  sudo -y upgrade

The second command will upgrade your Ubuntu apps and services to their latest version. The -y flag implies that you agree with the upgrade decision. The upgrade process should take a moment, depending on the number of apps and services you installed on your Linux system. Once done, you will need to check that you have a Python 3 upgraded version on your system.

tuts@codeunderscored:~$  python3 -V 

You should be able to get an output similar to the following:

python 3.8.5

Your Python3 will also need pip to install and manage its packages.

tuts@codeunderscored:~$  sudo apt install -y python3-pip 

To use pip you will need to adhere to the following command syntax.

tuts@codeunderscored:~$  pip3 install package_name

We are going to revisit this command structure soon.

Our Python programming environment needs some boost in its confidence, and for that reason, we also need to install other essential packages and development tools.

tuts@codeunderscored:~$  sudo apt install build-essential libffi-dev libssl-dev python-dev

Installing Virtual Environment

Now that we have the confidence of Python3 in our system, the next step is to set up an environment that enables us to run or execute different Python projects without them interfering with each other. The significance of a virtual environment is that different Python projects will require different Python packages. Therefore, instead of mixing these Python packages and only using a few, you get to only work with the packages that you need.

tuts@codeunderscored:~$  sudo apt install -y python3-venv

This successful installation gives you the privilege of creating and working with different project environments. Therefore, you have the option of defining various project environments and storing them in a single directory. For instance, you could do the following:

tuts@codeunderscored:~$  mkdir my_environments
tuts@codeunderscored:~$  cd my_environments
tuts@codeunderscored:~/my_environments$  python3 -m venv venv_one
tuts@codeunderscored:~/my_environments$  python3 -m venv venv_two

We have created a folder named my_environments, navigated into that folder, and created two virtual environments called venv_one and venv_two. To use either of these environments, mock the following command syntax:

tuts@codeunderscored:~$  source venv_name/bin/activate

The successful use of the above command should lead to an output like the one below.

(venv_name) tuts@codeunderscored:~/my_environments$

From here, you can navigate to the directory of your project files and use the virtual environment you activated to install and run packages.

(venv_name) tuts@codeunderscored:~/my_projects$ 

Exiting the virtual environment does not need you to navigate back to the my_environments folder. Regardless of your current path on the terminal, you only need to run the following command:

(venv_name) tuts@codeunderscored:~/some_random_folder$ deactivate 

The successful execution of this command should return you terminal back to its normal display.

tuts@codeunderscored:~/some_random_folder$ 

Installing and Setting Up Django

Now that we have Python3 on our system and know how to configure the virtual environment, it’s time we got back to the objective of our article. But before we do, we need to have Django installed. To work with Django, we will need a separate virtual environment on a defined project folder. Create the project folder you want, then create and activate the virtual environment you will use.

(venv_name) tuts@codeunderscored:~/my_project_folder$

You now need to install Django.

(venv_name) tuts@codeunderscored:~/my_project_folder$ pip3 install django >= 2.0 

The next step is to initialize a Django project.

(venv_name) tuts@codeunderscored:~/my_project_folder$ django-admin startproject importexport .

This command creates the needed Django project directory structure with the needed Python files to successfully run the project.

Finally, you will need an app or two that will interact with the project file settings. The app folder lets you create and configure the needed files that will bring your Django project to life.

(venv_name) tuts@codeunderscored:~/my_project_folder$ python3 manage.py startapp app_name

Please think of the initiated Django project as a car engine and the initiated Django app as its gas pedal. This logic is clear enough.

Back to Django-Import-Export

Now that you know how to set up your operating system for Django programming, the following steps should not give you any headaches. The pip tool you installed earlier will help get the Django-import-export library set up on your system.

(venv_name) tuts@codeunderscored:~/my_project_folder$ pip3 install django-import-export

A folder was created with a Django project setup when you ran this command;django-admin startproject importexport, earlier. Please browse through the importexport directory and update its settings.py file.

INSTALLED_APPS = (
	…
	'your_app_name',
	'import_export',
)  

At the bottom of this settings.py file, you can optionally add the following line as an additional configuration.

IMPORT_EXPORT_USE_TRANSACTIONS = True

The default value for this setting is usually False. A transaction refers to a logical task that a database management system performs. To be safe, we have allowed database transactions to be valid when we import data.

Resources

If this is not your first rodeo with Django, then you are familiar with how Django deals with admin classes and model forms. Now, if we bring the Django-import-export library into the picture, it will apply a similar classes concept but by using Resources.

The authors of the Django-import-export library have some interesting suggestions on the library’s implementation. They imply that the Django app’s admin.py file should host the resources-related code. However, if you want to implement this library’s functionality outside the Django Admin, another approach would be to create a resources.py file and save it inside your Django app’s folder alongside the likes of admin.py and views.py. For a demo on how Django-import-export works, we will create and work with a Person model. In your models.py file, populate it with the following content.

from django.db import models
class Person (models.model):
	name = models.CharField(max_length=30)
	email = models.EmailField(blank=True)
	birthday = models.DateField()
	location = models.CharField(max_length=100, blank=True)

In the resources.py file you created, populate it with the following content.

from import_export import resources
from .models import Person
 
class PersonResource (resources.ModelResource):
	class meta:
		model = Person

The Django-import-export documentation gives a broader perspective of how to deal with other configurations applicable to the Meta class like exclude and fields.

Exporting Data

Before we start exporting data, we need to populate the Person model with some information. The Django Admin interface will help us accomplish this objective. Before we can access the Django Admin, we need to adhere to a few guidelines.

(venv_name) tuts@codeunderscored:~/my_project_folder$ python3 manage.py makemigrations
(venv_name) tuts@codeunderscored:~/my_project_folder$ python3 manage.py migrate

Run the above commands sequentially. These commands configure the Django Admin and creates a database instance with the models we defined in the admin.py file. The makemigration command segment tracks the model changes that need implementation, and the migrate command segment implements those changes. Before we proceed, for my case, I named my app persons and the model person. Therefore, the app name you are working with should be listed under INSTALLED_APPS in the settings.py file the same way we did with import_export for the installed django-import-export library. We now need to give ourselves some user privileges.

(venv_name) tuts@codeunderscored:~/my_project_folder$ python3 manage.py createsuperuser

You will be prompted for a username, email, and password. Time to launch the Django server.

(venv_name) tuts@codeunderscored:~/my_project_folder$ python3 manage.py runserver

I everything goes as planned, your server instance should be accessible through the URL http://127.0.0.1:8000. However, since we want to access the Django Admin, you can modify the URL to http://127.0.0.1:8000/admin/. Try and access this URL on your browser and log in with your superuser credentials.

Our Person model is not yet visible in the Django Admin. To make it visible, we need to make some modifications to the admin.py file.

from import_export.admin import ImportExportModelAdmin
from django.contrib import admin
from .models import Person

@admin.register(Person)
class PersonAdmin(ImportExportModelAdmin):
	pass        

We registered the Person model in the Django Admin and implemented the Import-Export library function to be visible and effective. If you refresh your browser, you will be able to see the Person model listed.

Click on it, and another interface will appear with Import and Export tabs on the right side of your display window. They are tied to the Django-import-export library and will be useful in importing and exporting generated data.

You can now use the Django Admin interface to populate the Person model with as much data as possible. For my case, I made six entries, which I think are sufficient enough to meet the objective of this article.

After you are done populating your database, you should see a list of all the person object () entries you made. Click on the Export tab and choose the format in which you want your data exported. The listed data format on your admin side should be CSV, Xls, xlsx, CSV, ods, JSON, YAML, and HTML.

Depending on your data format selection, your output should resemble the following screenshots.

Exporting Data in Yaml Format

Exporting Data in CSV Format

Exporting Data in JSON Format

Exporting Data through Django Views

The code needed to export data in the format of your choice is straightforward. However, make sure you have some data stored on the database for exporting. If you are on the same directory structure as the resources.py file, then the code implementation should be easy as follows:

from .resources import PersonResource
person_resource = PersonResource()
dataset = person_resource.export()
dataset.csv  # you can change the extension(csv) to the format you want e.g dataset.json, dataset.yaml 

Filtering Exported Data

If you do not want to export all the data at once, you can filter your export results. For example, we can filter and export data by location, email, name, and even birth_date for the Person model we created. The code for implementing this functionality will look like the following:

from .resources import PersonResource
from .models import Person
person_resource = PersonResource()
queryset = person.object.filter(location='Russia') 
dataset =  person_resource.export(queryset)
dataset.csv #you can also change the extension(csv) to the format you want e.g dataset.yaml or dataset.json

Exporting Data Through HttpResponse

Supposing, we want to export data stored on the Person model through a web interface in JSON format. To achieve this objective, we would need to create a view function that caters to the data export.

Your views.py files should be like the following:

from django.http import HttpResponse
from .resources import PersonResource
def export (request):
	person_resource = PersonResource()
	dataset = person_resource.export()
	response = HttpResponse(dataset.json, content_type='application/json')
	response['Content-Disposition'] = 'attachment; filename="persons.json"'
	return response

Your project urls.py files should also look like the following:

from django.contrib import admin
from django.urls import path
from persons import views
from django.conf import settings
urlpatterns = [
	path('admin/', admin.site.urls),
	path('data/json/export/', views.export),
]

Now navigate to your browser and access the URL 127.0.0.1:8000/data/json/export/

You will get a prompt to export data from your database in json format.

Importing Data

Importing data is also much easier when using the Django Admin. You only need to make sure Django supports the extension of the file you are importing. We listed the supported extensions a while back. Therefore, you should use that list as a reference manual. Moreover, the contents of the file you want to import should also adhere to the extension format it represents. Therefore, you should avoid the misrepresentation of the file data content and the file data extension. When you use Django Admin to import data, you will get a preview display to review the data you intend to import before confirming your import.

It is an efficient way of dealing with already existing data. Confirming the Import will populate your database with new data.

Importing Data Through Django Forms

You can edit the CSV file we exported earlier through Django Admin and use a form to import data. The applicability of this case is when you have numerous users who need to upload various data formats on a system. It is because you can’t give everyone admin access and privileges to the system. Using Django Forms is, therefore, another flexible alternative of seamlessly uploading data.

Final Note

The Django-import-export library is a significant tool for dealing with various data formats. It provides an easy way to export your database records in useful formats. It saves time and, at the same time, adding a layer of efficiency to your data-related projects. It is an opportunity to make your Django projects more scalable and dynamic.