Django and Oscar

Django Migrations

Published 2021-02-12. Last modified 2021-03-28.
Time to read: 2 minutes.

This page is part of the django collection.

Each Django app can have its own migrations. Each migration is stored in a separate file, under the migrations directory for that app. I had installed django-oscar in the $oscar directory, and the django-oscar apps are found within the $oscar/lib/python3.8/site-packages/oscar/apps/ subdirectory.

Here is how I listed all the migration files, without the uninteresting __init__.py files:

Shell
(aw) $cd $oscar/lib/python3.8/site-packages/oscar/apps/
(aw) $ls **/migrations/*.py | grep -v __init__ address/migrations/0001_initial.py address/migrations/0002_auto_20150927_1547.py address/migrations/0003_auto_20150927_1551.py address/migrations/0004_auto_20170226_1122.py address/migrations/0005_regenerate_user_address_hashes.py address/migrations/0006_auto_20181115_1953.py analytics/migrations/0001_initial.py analytics/migrations/0002_auto_20140827_1705.py analytics/migrations/0003_auto_20200801_0817.py basket/migrations/0001_initial.py basket/migrations/0002_auto_20140827_1705.py basket/migrations/0003_basket_vouchers.py basket/migrations/0004_auto_20141007_2032.py basket/migrations/0005_auto_20150604_1450.py basket/migrations/0006_auto_20160111_1108.py basket/migrations/0007_slugfield_noop.py basket/migrations/0008_auto_20181115_1953.py basket/migrations/0009_line_date_updated.py catalogue/migrations/0001_initial.py catalogue/migrations/0002_auto_20150217_1221.py catalogue/migrations/0003_data_migration_slugs.py catalogue/migrations/0004_auto_20150217_1710.py catalogue/migrations/0005_auto_20150604_1450.py catalogue/migrations/0006_auto_20150807_1725.py catalogue/migrations/0007_auto_20151207_1440.py catalogue/migrations/0008_auto_20160304_1652.py catalogue/migrations/0009_slugfield_noop.py catalogue/migrations/0010_auto_20170420_0439.py catalogue/migrations/0011_auto_20170422_1355.py catalogue/migrations/0012_auto_20170609_1902.py catalogue/migrations/0013_auto_20170821_1548.py catalogue/migrations/0014_auto_20181115_1953.py catalogue/migrations/0015_product_is_public.py catalogue/migrations/0016_auto_20190327_0757.py catalogue/migrations/0017_auto_20190816_0938.py catalogue/migrations/0018_auto_20191220_0920.py catalogue/migrations/0019_option_required.py catalogue/migrations/0020_auto_20200801_0817.py catalogue/migrations/0021_auto_20201005_0844.py communication/migrations/0001_initial.py communication/migrations/0002_reset_table_names.py communication/migrations/0003_remove_notification_category_make_code_uppercase.py communication/migrations/0004_auto_20200801_0817.py customer/migrations/0001_initial.py customer/migrations/0002_auto_20150807_1725.py customer/migrations/0003_update_email_length.py customer/migrations/0004_email_save.py customer/migrations/0005_auto_20181115_1953.py customer/migrations/0006_auto_20190430_1736.py customer/migrations/0007_auto_20200801_0817.py offer/migrations/0001_initial.py offer/migrations/0002_auto_20151210_1053.py offer/migrations/0003_auto_20161120_1707.py offer/migrations/0004_auto_20170415_1518.py offer/migrations/0005_auto_20170423_1217.py offer/migrations/0006_auto_20170504_0616.py offer/migrations/0007_conditionaloffer_exclusive.py offer/migrations/0008_auto_20181115_1953.py offer/migrations/0009_auto_20200801_0817.py offer/migrations/0010_conditionaloffer_combinations.py order/migrations/0001_initial.py order/migrations/0002_auto_20141007_2032.py order/migrations/0003_auto_20150113_1629.py order/migrations/0004_auto_20160111_1108.py order/migrations/0005_update_email_length.py order/migrations/0006_orderstatuschange.py order/migrations/0007_auto_20181115_1953.py order/migrations/0008_auto_20190301_1035.py order/migrations/0009_surcharge.py order/migrations/0010_auto_20200724_0909.py order/migrations/0011_auto_20200801_0817.py partner/migrations/0001_initial.py partner/migrations/0002_auto_20141007_2032.py partner/migrations/0003_auto_20150604_1450.py partner/migrations/0004_auto_20160107_1755.py partner/migrations/0005_auto_20181115_1953.py partner/migrations/0006_auto_20200724_0909.py payment/migrations/0001_initial.py payment/migrations/0002_auto_20141007_2032.py payment/migrations/0003_auto_20160323_1520.py payment/migrations/0004_auto_20181115_1953.py payment/migrations/0005_auto_20200801_0817.py shipping/migrations/0001_initial.py shipping/migrations/0002_auto_20150604_1450.py shipping/migrations/0003_auto_20181115_1953.py voucher/migrations/0001_initial.py voucher/migrations/0002_auto_20170418_2132.py voucher/migrations/0003_auto_20171212_0411.py voucher/migrations/0004_auto_20180228_0940.py voucher/migrations/0005_auto_20180402_1425.py voucher/migrations/0006_auto_20180413_0911.py voucher/migrations/0007_auto_20181115_1953.py voucher/migrations/0008_auto_20200801_0817.py wishlists/migrations/0001_initial.py wishlists/migrations/0002_auto_20160111_1108.py wishlists/migrations/0003_auto_20181115_1953.py

showmigrations Subcommand

The showmigrations subcommand of ./manage.py is a handy way to display the same migrations, and it also shows an X next to the migrations that have already been applied.

Here is the showmigrations subcommand help:

Shell
(aw) $./manage.py showmigrations -h
usage: manage.py showmigrations [-h] [--database DATABASE] [--list | --plan]
                                  [--version] [-v {0,1,2,3}]
                                  [--settings SETTINGS]
                                  [--pythonpath PYTHONPATH] [--traceback]
                                  [--no-color] [--force-color] [--skip-checks]
                                  [app_label [app_label ...]]
Shows all available migrations for the current project
positional arguments: app_label App labels of applications to limit the output to.
optional arguments: -h, --help show this help message and exit --database DATABASE Nominates a database to synchronize. Defaults to the "default" database. --list, -l Shows a list of all migrations and which are applied. With a verbosity level of 2 or above, the applied datetimes will be included. --plan, -p Shows all migrations in the order they will be applied. With a verbosity level of 2 or above all direct migration dependencies and reverse dependencies (run_before) will be included. --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.

Here is sample output for the showmigrations subcommand:

Shell
(aw) $./manage.py showmigrations
address
 [X] 0001_initial
 [X] 0002_auto_20150927_1547
 [X] 0003_auto_20150927_1551
 [X] 0004_auto_20170226_1122
 [X] 0005_regenerate_user_address_hashes
 [X] 0006_auto_20181115_1953
admin
 [X] 0001_initial
 [X] 0002_logentry_remove_auto_add
 [X] 0003_logentry_add_action_flag_choices
analytics
 [X] 0001_initial
 [X] 0002_auto_20140827_1705
 [X] 0003_auto_20200801_0817
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
 [X] 0008_alter_user_username_max_length
 [X] 0009_alter_user_last_name_max_length
 [X] 0010_alter_group_name_max_length
 [X] 0011_update_proxy_permissions
 [X] 0012_alter_user_first_name_max_length
basket
 [X] 0001_initial
 [X] 0002_auto_20140827_1705
 [X] 0003_basket_vouchers
 [X] 0004_auto_20141007_2032
 [X] 0005_auto_20150604_1450
 [X] 0006_auto_20160111_1108
 [X] 0007_slugfield_noop
 [X] 0008_auto_20181115_1953
 [X] 0009_line_date_updated
catalogue
 [X] 0001_initial
 [X] 0002_auto_20150217_1221
 [X] 0003_data_migration_slugs
 [X] 0004_auto_20150217_1710
 [X] 0005_auto_20150604_1450
 [X] 0006_auto_20150807_1725
 [X] 0007_auto_20151207_1440
 [X] 0008_auto_20160304_1652
 [X] 0009_slugfield_noop
 [X] 0010_auto_20170420_0439
 [X] 0011_auto_20170422_1355
 [X] 0012_auto_20170609_1902
 [X] 0013_auto_20170821_1548
 [X] 0014_auto_20181115_1953
 [X] 0015_product_is_public
 [X] 0016_auto_20190327_0757
 [X] 0017_auto_20190816_0938
 [X] 0018_auto_20191220_0920
 [X] 0019_option_required
 [X] 0020_auto_20200801_0817
 [X] 0021_auto_20201005_0844
communication
 [X] 0001_initial
 [X] 0002_reset_table_names
 [X] 0003_remove_notification_category_make_code_uppercase
 [X] 0004_auto_20200801_0817
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
customer
 [X] 0001_initial
 [X] 0002_auto_20150807_1725
 [X] 0003_update_email_length
 [X] 0004_email_save
 [X] 0005_auto_20181115_1953
 [X] 0006_auto_20190430_1736
 [X] 0007_auto_20200801_0817
flatpages
 [X] 0001_initial
offer
 [X] 0001_initial
 [X] 0002_auto_20151210_1053
 [X] 0003_auto_20161120_1707
 [X] 0004_auto_20170415_1518
 [X] 0005_auto_20170423_1217
 [X] 0006_auto_20170504_0616
 [X] 0007_conditionaloffer_exclusive
 [X] 0008_auto_20181115_1953
 [X] 0009_auto_20200801_0817
 [X] 0010_conditionaloffer_combinations
order
 [X] 0001_initial
 [X] 0002_auto_20141007_2032
 [X] 0003_auto_20150113_1629
 [X] 0004_auto_20160111_1108
 [X] 0005_update_email_length
 [X] 0006_orderstatuschange
 [X] 0007_auto_20181115_1953
 [X] 0008_auto_20190301_1035
 [X] 0009_surcharge (1 squashed migrations)
 [X] 0010_auto_20200724_0909
 [X] 0011_auto_20200801_0817
partner
 [X] 0001_initial
 [X] 0002_auto_20141007_2032
 [X] 0003_auto_20150604_1450
 [X] 0004_auto_20160107_1755
 [X] 0005_auto_20181115_1953
 [X] 0006_auto_20200724_0909
payment
 [X] 0001_initial
 [X] 0002_auto_20141007_2032
 [X] 0003_auto_20160323_1520
 [X] 0004_auto_20181115_1953
 [X] 0005_auto_20200801_0817
reviews
 [X] 0001_initial
 [X] 0002_update_email_length
 [X] 0003_auto_20160802_1358
 [X] 0004_auto_20170429_0941
sessions
 [X] 0001_initial
shipping
 [X] 0001_initial
 [X] 0002_auto_20150604_1450
 [X] 0003_auto_20181115_1953
sites
 [X] 0001_initial
 [X] 0002_alter_domain_unique
thumbnail
 [X] 0001_initial
voucher
 [X] 0001_initial
 [X] 0002_auto_20170418_2132
 [X] 0003_auto_20171212_0411
 [X] 0004_auto_20180228_0940
 [X] 0005_auto_20180402_1425
 [X] 0006_auto_20180413_0911
 [X] 0007_auto_20181115_1953
 [X] 0008_auto_20200801_0817
wishlists
 [X] 0001_initial
 [X] 0002_auto_20160111_1108
 [X] 0003_auto_20181115_1953 %}

Here is the first migration file, address/migrations/0001_initial.py:

address/migrations/0001_initial.py
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
 
from django.db import models, migrations
import oscar.models.fields
from django.conf import settings
 
 
class Migration(migrations.Migration):
 
    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
    ]
 
    operations = [
        migrations.CreateModel(
            name='Country',
            fields=[
                ('iso_3166_1_a2', models.CharField(primary_key=True, max_length=2, verbose_name='ISO 3166-1 alpha-2', serialize=False)),
                ('iso_3166_1_a3', models.CharField(max_length=3, verbose_name='ISO 3166-1 alpha-3', blank=True)),
                ('iso_3166_1_numeric', models.CharField(max_length=3, verbose_name='ISO 3166-1 numeric', blank=True)),
                ('printable_name', models.CharField(max_length=128, verbose_name='Country name')),
                ('name', models.CharField(max_length=128, verbose_name='Official name')),
                ('display_order', models.PositiveSmallIntegerField(default=0, verbose_name='Display order', db_index=True, help_text='Higher the number, higher the country in the list.')),
                ('is_shipping_country', models.BooleanField(default=False, db_index=True, verbose_name='Is shipping country')),
            ],
            options={
                'ordering': ('-display_order', 'printable_name'),
                'verbose_name_plural': 'Countries',
                'verbose_name': 'Country',
                'abstract': False,
            },
            bases=(models.Model,),
        ),
        migrations.CreateModel(
            name='UserAddress',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('title', models.CharField(verbose_name='Title', max_length=64, blank=True, choices=[('Mr', 'Mr'), ('Miss', 'Miss'), ('Mrs', 'Mrs'), ('Ms', 'Ms'), ('Dr', 'Dr')])),
                ('first_name', models.CharField(max_length=255, verbose_name='First name', blank=True)),
                ('last_name', models.CharField(max_length=255, verbose_name='Last name', blank=True)),
                ('line1', models.CharField(max_length=255, verbose_name='First line of address')),
                ('line2', models.CharField(max_length=255, verbose_name='Second line of address', blank=True)),
                ('line3', models.CharField(max_length=255, verbose_name='Third line of address', blank=True)),
                ('line4', models.CharField(max_length=255, verbose_name='City', blank=True)),
                ('state', models.CharField(max_length=255, verbose_name='State/County', blank=True)),
                ('postcode', oscar.models.fields.UppercaseCharField(max_length=64, verbose_name='Post/Zip-code', blank=True)),
                ('search_text', models.TextField(editable=False, verbose_name='Search text - used only for searching addresses')),
                ('phone_number', oscar.models.fields.PhoneNumberField(verbose_name='Phone number', help_text='In case we need to call you about your order', blank=True)),
                ('notes', models.TextField(verbose_name='Instructions', help_text='Tell us anything we should know when delivering your order.', blank=True)),
                ('is_default_for_shipping', models.BooleanField(default=False, verbose_name='Default shipping address?')),
                ('is_default_for_billing', models.BooleanField(default=False, verbose_name='Default billing address?')),
                ('num_orders', models.PositiveIntegerField(default=0, verbose_name='Number of Orders')),
                ('hash', models.CharField(max_length=255, editable=False, db_index=True, verbose_name='Address Hash')),
                ('date_created', models.DateTimeField(auto_now_add=True, verbose_name='Date Created')),
                ('country', models.ForeignKey(verbose_name='Country', to='address.Country', on_delete=models.CASCADE)),
                ('user', models.ForeignKey(verbose_name='User', related_name='addresses', to=settings.AUTH_USER_MODEL, on_delete=models.CASCADE)),
            ],
            options={
                'ordering': ['-num_orders'],
                'verbose_name_plural': 'User addresses',
                'verbose_name': 'User address',
                'abstract': False,
            },
            bases=(models.Model,),
        ),
        migrations.AlterUniqueTogether(
            name='useraddress',
            unique_together=set([('user', 'hash')]),
        ),
    ]

sqlmigrate Subcommand

The SQL generated from the above migration file can be viewed with the sqlmigrate subcommand of manage.py.

Here is the sqlmigrate subcommand help:

Shell
(aw) $./manage.py sqlmigrate -h
usage: manage.py sqlmigrate [-h] [--database DATABASE] [--backwards]
                              [--version] [-v {0,1,2,3}] [--settings SETTINGS]
                              [--pythonpath PYTHONPATH] [--traceback]
                              [--no-color] [--force-color] [--skip-checks]
                              app_label migration_name
Prints the SQL statements for the named migration.
positional arguments: app_label App label of the application containing the migration. migration_name Migration name to print the SQL for.
optional arguments: -h, --help show this help message and exit --database DATABASE Nominates a database to create SQL for. Defaults to the "default" database. --backwards Creates SQL to unapply the migration, rather than to apply it --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.

I tried using the --database option, so the generated SQL would refer to the proper database for my Frobshop web application, but after examining the generated SQL I discovered that it did not mention the database by name.

Shell
(aw) $./manage.py sqlmigrate address 0001_initial
BEGIN;
--
-- Create model Country
--
CREATE TABLE "address_country" ("iso_3166_1_a2" varchar(2) NOT NULL PRIMARY KEY, "iso_3166_1_a3" varchar(3) NOT NULL, "iso_3166_1_numeric" varchar(3) NOT NULL, "printable_name" varchar(128) NOT NULL, "name" varchar(128) NOT NULL, "display_order" smallint NOT NULL CHECK ("display_order" >= 0), "is_shipping_country" boolean NOT NULL);
--
-- Create model UserAddress
--
CREATE TABLE "address_useraddress" ("id" serial NOT NULL PRIMARY KEY, "title" varchar(64) NOT NULL, "first_name" varchar(255) NOT NULL, "last_name" varchar(255) NOT NULL, "line1" varchar(255) NOT NULL, "line2" varchar(255) NOT NULL, "line3" varchar(255) NOT NULL, "line4" varchar(255) NOT NULL, "state" varchar(255) NOT NULL, "postcode" varchar(64) NOT NULL, "search_text" text NOT NULL, "phone_number" varchar(128) NOT NULL, "notes" text NOT NULL, "is_default_for_shipping" boolean NOT NULL, "is_default_for_billing" boolean NOT NULL, "num_orders" integer NOT NULL CHECK ("num_orders" >= 0), "hash" varchar(255) NOT NULL, "date_created" timestamp with time zone NOT NULL, "country_id" varchar(2) NOT NULL, "user_id" integer NOT NULL);
--
-- Alter unique_together for useraddress (1 constraint(s))
--
ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_user_id_hash_9d1738c7_uniq" UNIQUE ("user_id", "hash");
CREATE INDEX "address_country_iso_3166_1_a2_f395eed0_like" ON "address_country" ("iso_3166_1_a2" varchar_pattern_ops);
CREATE INDEX "address_country_display_order_dc88cde8" ON "address_country" ("display_order");
CREATE INDEX "address_country_is_shipping_country_f7b6c461" ON "address_country" ("is_shipping_country");
ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_country_id_fa26a249_fk_address_c" FOREIGN KEY ("country_id") REFERENCES "address_country" ("iso_3166_1_a2") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_user_id_6edf0244_fk_auth_user_id" FOREIGN KEY ("user_id") REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "address_useraddress_hash_e0a6b290" ON "address_useraddress" ("hash");
CREATE INDEX "address_useraddress_hash_e0a6b290_like" ON "address_useraddress" ("hash" varchar_pattern_ops);
CREATE INDEX "address_useraddress_country_id_fa26a249" ON "address_useraddress" ("country_id");
CREATE INDEX "address_useraddress_country_id_fa26a249_like" ON "address_useraddress" ("country_id" varchar_pattern_ops);
CREATE INDEX "address_useraddress_user_id_6edf0244" ON "address_useraddress" ("user_id");
COMMIT; 

sqlcreate Subcommand

The sqlcreate subcommand generates the SQL to create the database.

Here is the sqlcreate subcommand help:

Shell
(aw) $./manage.py sqlcreate -h
usage: manage.py sqlcreate [-h] [-R ROUTER] [--database DATABASE] [-D]
                             [--version] [-v {0,1,2,3}] [--settings SETTINGS]
                             [--pythonpath PYTHONPATH] [--traceback]
                             [--no-color] [--force-color]
Generates the SQL to create your database for you, as specified in settings.py The envisioned use case is something like this: ./manage.py sqlcreate [--database=<databasename>] | mysql -u <db_administrator> -p ./manage.py sqlcreate [--database=<databasname>] | psql -U <db_administrator> -W
optional arguments: -h, --help show this help message and exit -R ROUTER, --router ROUTER Use this router-database other then defined in settings.py --database DATABASE Nominates a database to run command for. Defaults to the "default" database. -D, --drop If given, includes commands to drop any existing user and database. --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output.

The output of the sqlcreate subcommand can be piped into the dbshell subcommand, and then all migrations can be applied, like this:

Shell
(aw) $./manage.py sqlcreate | \
  ./manage.py dbshell && ./manage.py migrate

Resetting Migrations

I found myself resetting migrations several as I stumbled forward in my learning experience. Vitor Freitas wrote a helpful article that allowed me to write the following script for resetting migrations and setting up the database with my desired django-oscar shipping information:

reset
#!/bin/bash
# Reset migrations and set up the database with only US & Canada enabled for shipping
DB=ancient_warmth set -e
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete find . -path "*/migrations/*.pyc" -delete
dropdb -U postgres -h localhost $DB createdb -U postgres -h localhost $DB
./manage.py migrate --fake-initial # See https://stackoverflow.com/a/37371482/553865 ./manage.py makemigrations ./manage.py migrate
./manage.py oscar_populate_countries --no-shipping bin/psql -c "update address_country set is_shipping_country = 't' where iso_3166_1_a2 in ('CA', 'US');"
DJANGO_SUPERUSER_EMAIL=admin@domain.com DJANGO_SUPERUSER_PASSWORD=secret ./manage.py createsuperuser --noinput --username admin

This is my bin/psql script, used by the above:

bin/psql
#!/bin/bash
if [ "$1" == -t ]; then PREFIX="test_" shift fi PGPASSWORD=secret /usr/bin/psql \ -U postgres \ -h localhost \ -d ${PREFIX}ancient_warmth \ "$@"
* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.