Including external dependencies in your project

Sometimes, it is better to include external dependencies directly within your project. This ensures that whenever a developer upgrades third-party modules, all of the other developers will receive the upgraded version in the next update from the version control system (Git, Subversion, or others).

Also, it is better to have external dependencies included in your project when the libraries are taken from unofficial sources, that is, somewhere other than the Python Package Index (PyPI) or different version control systems.

Getting ready

Start with a virtual environment with a Django project in it.

How to do it...

Execute the following steps one by one for a virtual environment project:

1. If you haven't done so already, create an externals directory under your Django project django-myproject directory. Then, create the libs and apps directories under it. The libs directory is for the Python modules that are required by your project, for example, Boto, Requests, Twython, and Whoosh. The apps directory is for third-party Django apps, for example, Django CMS, Django Haystack, and django-storages.

We highly recommend that you create README.md files in the libs and apps directories, where you mention what each module is for, what the used version or revision is, and where it is taken from.

2. The directory structure should look something similar to the following:

externals/
├── apps/
│   ├── cms/
│   ├── haystack/
│   ├── storages/
│   └── README.md
└── libs/
    ├── boto/
    ├── requests/
    ├── twython/
    └── README.md

3. The next step is to put the external libraries and apps under the Python path so that they are recognized as if they were installed. This can be done by adding the following code in the settings:

# settings.py
import os, sys
BASE_DIR = os.path.dirname(os.path.dirname(
    os.path.abspath(__file__)))

EXTERNAL_BASE = os.path.join(BASE_DIR, "externals")
EXTERNAL_LIBS_PATH = os.path.join(EXTERNAL_BASE, "libs")
EXTERNAL_APPS_PATH = os.path.join(EXTERNAL_BASE, "apps")
sys.path = ["", EXTERNAL_LIBS_PATH, EXTERNAL_APPS_PATH] + sys.path

How it works...

A module is meant to be under the Python path if you can run Python and import that module. One of the ways to put a module under the Python path is to modify the sys.path variable before importing a module that is in an unusual location. The value of sys.path, as specified by the settings.py file, is a list of directories starting with an empty string for the current directory, followed by the directories in the project, and finally the globally shared directories of the Python installation. You can see the value of sys.path in the Python shell, as follows:

(myproject)$ ./manage.py shell
>>> import sys
>>> sys.path

The same could be done for a Docker project, assuming the container name were django_myproject_app_1, as follows:

myproject_docker/$ docker exec -it django_myproject_app_1 \
> python3 manage.py shell
>>> import sys
>>> sys.path

When trying to import a module, Python searches for the module in this list and returns the first result that is found.

Therefore, we first define the BASE_DIR variable, which is the absolute path to one level higher than the settings.py file. Then, we define the EXTERNAL_LIBS_PATH and EXTERNAL_APPS_PATH variables, which are relative to BASE_DIR. Lastly, we modify the sys.path property, adding new paths to the beginning of the list. Note that we also add an empty string as the first path to search, which means that the current directory of any module should always be checked first before checking other Python paths.

This way of including external libraries doesn't work cross-platform with the Python packages that have C language bindings, for example, lxml. For such dependencies, we would recommend using the pip requirements that were introduced in the Handling project dependencies with pip recipe.

There's more...

With a Docker project, there is significantly more control of the libraries and apps that are installed within the container:

• For Python libraries needed for the project, we can use version specifications in the requirements.txt file to require a version known to be compatible. Furthermore, it was demonstrated in the Handling project dependencies with pip recipe that we can differentiate these requirements by environment, as well as being so precise as to require an exact repository version using the -e flag.
• All Django applications are stored under the apps directory. Here would reside not only the code for ones specifically under development, but also any external apps that are not made available globally via the requirements.txt dependency list.

See also

• The Creating a virtual environment project file structure recipe
• The Creating a Docker project file structure recipe
• The Handling project dependencies with pip recipe
• The Defining relative paths in the settings recipe
• The Using the Django shell recipe in Chapter 11, Bells and Whistles