Running Django Tests Without Migrations

python_and_mouse

Django is a strong framework for building reliable websites. However, it is showing signs of age. Particularly, it is failing where tools such as Spring Boot excel.

This article examines how to overcome yet another issue encountered in Django, running tests without migrating databases.

Use Case

There are many times when a database migration is inappropriate for testing:

  • we have separated test, development, and production databases with custom features requiring testing
  • there are fixtures pre-loaded into a database
  • we have legwork to complete through scripts alongside database setup
  • dropping and re-creating a database eliminates data that is absolutely necessary to another function in our test environment
  • nothing we do can be included in a TestRunner, at least not reliably

Sadly, Django developers, among other issues they refuse to fix, were not willing to create a workaround until recently. The workaround does not work in the latest version.

Django Unit Test Framework

The Django unit test framework comes with many features typically reserved for tools such as ghost.py or selenium.

For instance, Django provides:

from django.test import TestCase
from django.test import Client
from django.test import RequestFactory

A RequestFactory is used to perform specific tests while the Client acts as a browser would.

While there is an option to use pytest, the changing nature of Django manages to continually break this tool. Importing models from different applications using multiple databases is a broken process at the time of publication.

Unittest Format

Django provides an extensive tutorial for setting up unit tests. The setup is straightforward:

from django.test import TestCase
from django.test import Client
from django.test import RequestFactory


class AdminTestCases(TestCase):
    multi_db = True

    def setUp(self):
        self.client = Client()
        self.rf = RequestFactory()

    def test_create_superuser(self):
        pass

    def test_remove_superuser(self):
        pass

    def test_fail_create_superuser(self):
        pass

    def test_fail_remove_superuser(self):
        pass

    def tearDown(self):
        pass

This skeleton prepares  a client and request factory, generates tests, and provides a tear down function. Setup and tear down appear to run before and after each test.

Multiple Databases

For security and other purposes, it is often necessary to separate databases. This offers a degree of protection for sensitive information.

Unfortunately, this also presents an issue with the tests in which a single database is used. To avoid this, each test case must include the following:

multi_db = True

This line informs the the test runner to search for different databases.

Ignoring Migrated Databases

Now, for the meat and potatoes. We want to ignore database migration altogether at times. It is still wise to have a script and drop any unused data.

Django allows for the generation of custom TestRunner classes. As of Django > 2.0. the following runner avoids migrating a database:

from django.test.runner import DiscoverRunner


class NoMigrationTestRunner(DiscoverRunner):
  """ A test runner to test without database creation """

  def setup_databases(self, **kwargs):
    """ Override the database creation defined in parent class """
    pass

  def teardown_databases(self, old_config, **kwargs):
    """ Override the database teardown defined in parent class """
    pass

The NoMigrationTestRunner extends the DiscoverRunner and directly overwrites the setup_databases and teardown_databases methods. These methods are used to setup certain connections, create database tables, and perform cleanup.

In this example, the setup and teardown methods are left blank to avoid creating new tables as well as to allow for the smooth usage of multiple databases.

Django is informed of the new test runner through a variable in the relevant settings configuration:

TEST_RUNNER = 'path_to.NoMigrationTestRunner'

Execution

Tests are executed using manage.py:

python manage.py test
python<version> manage.py path.to.tests.TestModule --settings=<settings_file>

The first example is generic and collects tests found through our DiscoverRunner. The second command runs tests using a specific python version and a path to the relevant module which is imported using this string. A settings file was also provided in the second command.

Conclusion

Django has extensive documentation. However, as the framework changes, certain aspects will break. This article covered how to setup tests for multiple databases, not migrate databases in test, and how to execute our tests.

Automating Django Database Re-Creation on PostgreSQL

border_wall

photo: Oren Ziv/Activestills.org

The Django database migration system is a mess. This mess is made more difficult when using PostgreSQL.

Schemas go unrecognized without a work around, indices can break when using the work around, and processes are very difficult to automate with the existing code base.

The authors of Django, or someone such as myself who occasionally finds the time to dip in and unclutter a mess or two, will eventually resolve this. However, the migration system today is a train wreck.

My current solution to migrating a Postgres based Django application is presented in this article.

Benefits of Getting Down and Dirty

The benefits of taking the time to master this process for your own projects are not limited to the ability to automate Django. They include the ability to easily manage migrations and even automate builds.

Imagine having a DDL that is defined in your application and changed with one line of code without ever touching your database.

Django is a powerful and concurrent web framework with an enormous amount of add-ons and features. Mastering this task can open your code base to amazing amounts of abstraction and potential.

Well Defined Problems

While Django is powerful, integration with Postgres is horrible. A schema detection problem can cause a multitude of problems as can issues related to using multiple databases. When combined, these problems sap hours of valuable time.

The most major issues are:

  • lack of migration support for multiple databases
  • lack of schema support (an issue recognized over 13 years ago)
  • some indices (PostGIS polygon ids here) break when using the schema workaround

Luckily, the solutions for these problems are simple and require hacking only a few lines of your configuration.

Solving the Multiple Database Issue

This issue is more annoying than a major problem. Simply obtain a list of your applications and use your database names as defined by your database router in migration.

If there are migrations you want to reset, use the following article instead of the migrate command to delete your migrations and make sure to drop tables from your database without deleting any required schemas.

Using python[version] manage.py migrate [app] zero  did not actually drop and recreate my tables for some reason.

For thoroughness, discover your applications using:

python[verion] manage.py showmigrations

With your applications defined, make the migrations and run the second line of code in an appropriate order for each application:

python[version] manage.py makemigrations [--settings=my_settings.py]
python[version] manage.py migrate [app] --database= [--settings=my_settings.py]
....

As promised, the solution is simple. The –database switch matches the database name in your configuration and must be present as Django only recognizes your default configuration if it is not. This can complete migrations without actually performing them.

Solving the Schema Problem without Breaking PostGIS Indices

Django offers terrific Postgres support, including support for PostGIS. However, Postgres schemas are not supported. Tickets were opened over 13 years ago but were merged and forgotten.

To ensure that schemas are read properly, setup a database router and add a database configuration utilizing the following template:

"default": {
        "ENGINE": "django.contrib.gis.db.backends.postgis",
        "NAME": "test",
        "USER": "test",
        "PASSWORD": "test",
        "HOST": "127.0.0.1",
        "PORT": "5432",
        "OPTIONS": {
            "options": "-csearch_path=my_schema,public"
        }
 }

An set of options append to your dsn including a schema and a backup schema, used if the first schema is not present, are added via the configuration. Note the lack of spacing.

Now that settings.py is configured and a database router is established, add the following Meta class to each Model:

    class Meta():
        db_table=u'schema\".\"table'

Notice the awkward value for db_table. This is due to the way that tables are specified in Django. It is possible to leave managed as True, allowing Django to perform migrations, as long as the database is cleaned up a bit.

If there are any indices breaking on migration at this point, simply drop the table definition and use whichever schema this table ends up in. There is no apparent work around for this.

Now Your Migration Can Be Run In a Script, Even in a CI Tool

After quite a bit of fiddling, I found that it is possible to script and thus automate database builds. This is incredibly useful for testing.

My script, written to recreate a test environment, included a few direct SQL statements as well as calls to manage.py:

echo "DROP SCHEMA IF EXISTS auth CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP SCHEMA IF EXISTS filestorage CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP SCHEMA IF EXISTS licensing CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP SCHEMA IF EXISTS simplred CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP SCHEMA IF EXISTS fileauth CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP SCHEMA IF EXISTS licenseauth CASCADE" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "DROP OWNED BY simplrdev" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS auth" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS filestorage" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS licensing" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS simplred" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS licenseauth" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
echo "CREATE SCHEMA IF NOT EXISTS fileauth" | python3 ./app_repo/simplred/manage.py dbshell --settings=test_settings
find ./app_repo -path "*/migrations/*.py" -not -name "__init__.py" -delete
find ./app_repo -path "*/migrations/*.pyc"  -delete
python3 ./app_repo/simplred/manage.py makemigrations --settings=test_settings
python3 ./app_repo/simplred/manage.py migrate auth --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate admin --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate registration --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate sessions --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate oauth2_provider --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate auth_middleware --settings=test_settings --database=auth
python3 ./app_repo/simplred/manage.py migrate simplredapp --settings=test_settings --database=data
python3 ./app_repo/simplred/manage.py loaddata --settings=test_settings --database=data fixture

Assuming the appropriate security measures are followed, this script works well in a Bamboo job.  My script drops and recreates any necessary database components as well as clears migrations and then creates and performs migrations. Remember, this script recreates a test environment.

The running Django application, which is updated via a webhook, doesn’t actually break when I do this. I now have a completely automated test environment predicated on merely merging pull requests into my test branch and ignoring any migrations folders through .gitignore.

Conclusion

PostgreSQL is powerful, as is Django, but combining the two requires some finagling to achieve maximum efficiency. Here, we examined a few issues encountered when using Django and Postgres together and discussed how to solve them.

We discovered that it is possible to automate database migration without too much effort if you already know the hacks that make this process work.