Published 2021-02-12.
Last modified 2021-03-28.
Time to read: 2 minutes.
Each Django app can have its own migrations.
Each migration is stored in a separate file, under the migrations directory for that app.
I had installed django-oscar in the $oscar directory, and the django-oscar apps are found within the
$oscar/lib/python3.8/site-packages/oscar/apps/ subdirectory.
Here is how I listed all the migration files, without the uninteresting __init__.py files:
Shell
(aw) $cd $oscar/lib/python3.8/site-packages/oscar/apps/
(aw) $ls **/migrations/*.py | grep -v __init__ address/migrations/0001_initial.py address/migrations/0002_auto_20150927_1547.py address/migrations/0003_auto_20150927_1551.py address/migrations/0004_auto_20170226_1122.py address/migrations/0005_regenerate_user_address_hashes.py address/migrations/0006_auto_20181115_1953.py analytics/migrations/0001_initial.py analytics/migrations/0002_auto_20140827_1705.py analytics/migrations/0003_auto_20200801_0817.py basket/migrations/0001_initial.py basket/migrations/0002_auto_20140827_1705.py basket/migrations/0003_basket_vouchers.py basket/migrations/0004_auto_20141007_2032.py basket/migrations/0005_auto_20150604_1450.py basket/migrations/0006_auto_20160111_1108.py basket/migrations/0007_slugfield_noop.py basket/migrations/0008_auto_20181115_1953.py basket/migrations/0009_line_date_updated.py catalogue/migrations/0001_initial.py catalogue/migrations/0002_auto_20150217_1221.py catalogue/migrations/0003_data_migration_slugs.py catalogue/migrations/0004_auto_20150217_1710.py catalogue/migrations/0005_auto_20150604_1450.py catalogue/migrations/0006_auto_20150807_1725.py catalogue/migrations/0007_auto_20151207_1440.py catalogue/migrations/0008_auto_20160304_1652.py catalogue/migrations/0009_slugfield_noop.py catalogue/migrations/0010_auto_20170420_0439.py catalogue/migrations/0011_auto_20170422_1355.py catalogue/migrations/0012_auto_20170609_1902.py catalogue/migrations/0013_auto_20170821_1548.py catalogue/migrations/0014_auto_20181115_1953.py catalogue/migrations/0015_product_is_public.py catalogue/migrations/0016_auto_20190327_0757.py catalogue/migrations/0017_auto_20190816_0938.py catalogue/migrations/0018_auto_20191220_0920.py catalogue/migrations/0019_option_required.py catalogue/migrations/0020_auto_20200801_0817.py catalogue/migrations/0021_auto_20201005_0844.py communication/migrations/0001_initial.py communication/migrations/0002_reset_table_names.py communication/migrations/0003_remove_notification_category_make_code_uppercase.py communication/migrations/0004_auto_20200801_0817.py customer/migrations/0001_initial.py customer/migrations/0002_auto_20150807_1725.py customer/migrations/0003_update_email_length.py customer/migrations/0004_email_save.py customer/migrations/0005_auto_20181115_1953.py customer/migrations/0006_auto_20190430_1736.py customer/migrations/0007_auto_20200801_0817.py offer/migrations/0001_initial.py offer/migrations/0002_auto_20151210_1053.py offer/migrations/0003_auto_20161120_1707.py offer/migrations/0004_auto_20170415_1518.py offer/migrations/0005_auto_20170423_1217.py offer/migrations/0006_auto_20170504_0616.py offer/migrations/0007_conditionaloffer_exclusive.py offer/migrations/0008_auto_20181115_1953.py offer/migrations/0009_auto_20200801_0817.py offer/migrations/0010_conditionaloffer_combinations.py order/migrations/0001_initial.py order/migrations/0002_auto_20141007_2032.py order/migrations/0003_auto_20150113_1629.py order/migrations/0004_auto_20160111_1108.py order/migrations/0005_update_email_length.py order/migrations/0006_orderstatuschange.py order/migrations/0007_auto_20181115_1953.py order/migrations/0008_auto_20190301_1035.py order/migrations/0009_surcharge.py order/migrations/0010_auto_20200724_0909.py order/migrations/0011_auto_20200801_0817.py partner/migrations/0001_initial.py partner/migrations/0002_auto_20141007_2032.py partner/migrations/0003_auto_20150604_1450.py partner/migrations/0004_auto_20160107_1755.py partner/migrations/0005_auto_20181115_1953.py partner/migrations/0006_auto_20200724_0909.py payment/migrations/0001_initial.py payment/migrations/0002_auto_20141007_2032.py payment/migrations/0003_auto_20160323_1520.py payment/migrations/0004_auto_20181115_1953.py payment/migrations/0005_auto_20200801_0817.py shipping/migrations/0001_initial.py shipping/migrations/0002_auto_20150604_1450.py shipping/migrations/0003_auto_20181115_1953.py voucher/migrations/0001_initial.py voucher/migrations/0002_auto_20170418_2132.py voucher/migrations/0003_auto_20171212_0411.py voucher/migrations/0004_auto_20180228_0940.py voucher/migrations/0005_auto_20180402_1425.py voucher/migrations/0006_auto_20180413_0911.py voucher/migrations/0007_auto_20181115_1953.py voucher/migrations/0008_auto_20200801_0817.py wishlists/migrations/0001_initial.py wishlists/migrations/0002_auto_20160111_1108.py wishlists/migrations/0003_auto_20181115_1953.py
showmigrations Subcommand
The showmigrations subcommand of ./manage.py is a handy way to display the same migrations,
and it also shows an X next to the migrations that have already been applied.
Here is the showmigrations subcommand help:
Shell
(aw) $./manage.py showmigrations -h usage: manage.py showmigrations [-h] [--database DATABASE] [--list | --plan] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks] [app_label [app_label ...]] Shows all available migrations for the current project positional arguments: app_label App labels of applications to limit the output to. optional arguments: -h, --help show this help message and exit --database DATABASE Nominates a database to synchronize. Defaults to the "default" database. --list, -l Shows a list of all migrations and which are applied. With a verbosity level of 2 or above, the applied datetimes will be included. --plan, -p Shows all migrations in the order they will be applied. With a verbosity level of 2 or above all direct migration dependencies and reverse dependencies (run_before) will be included. --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.
Here is sample output for the showmigrations subcommand:
Shell
(aw) $./manage.py showmigrations
address
[X] 0001_initial
[X] 0002_auto_20150927_1547
[X] 0003_auto_20150927_1551
[X] 0004_auto_20170226_1122
[X] 0005_regenerate_user_address_hashes
[X] 0006_auto_20181115_1953
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add
[X] 0003_logentry_add_action_flag_choices
analytics
[X] 0001_initial
[X] 0002_auto_20140827_1705
[X] 0003_auto_20200801_0817
auth
[X] 0001_initial
[X] 0002_alter_permission_name_max_length
[X] 0003_alter_user_email_max_length
[X] 0004_alter_user_username_opts
[X] 0005_alter_user_last_login_null
[X] 0006_require_contenttypes_0002
[X] 0007_alter_validators_add_error_messages
[X] 0008_alter_user_username_max_length
[X] 0009_alter_user_last_name_max_length
[X] 0010_alter_group_name_max_length
[X] 0011_update_proxy_permissions
[X] 0012_alter_user_first_name_max_length
basket
[X] 0001_initial
[X] 0002_auto_20140827_1705
[X] 0003_basket_vouchers
[X] 0004_auto_20141007_2032
[X] 0005_auto_20150604_1450
[X] 0006_auto_20160111_1108
[X] 0007_slugfield_noop
[X] 0008_auto_20181115_1953
[X] 0009_line_date_updated
catalogue
[X] 0001_initial
[X] 0002_auto_20150217_1221
[X] 0003_data_migration_slugs
[X] 0004_auto_20150217_1710
[X] 0005_auto_20150604_1450
[X] 0006_auto_20150807_1725
[X] 0007_auto_20151207_1440
[X] 0008_auto_20160304_1652
[X] 0009_slugfield_noop
[X] 0010_auto_20170420_0439
[X] 0011_auto_20170422_1355
[X] 0012_auto_20170609_1902
[X] 0013_auto_20170821_1548
[X] 0014_auto_20181115_1953
[X] 0015_product_is_public
[X] 0016_auto_20190327_0757
[X] 0017_auto_20190816_0938
[X] 0018_auto_20191220_0920
[X] 0019_option_required
[X] 0020_auto_20200801_0817
[X] 0021_auto_20201005_0844
communication
[X] 0001_initial
[X] 0002_reset_table_names
[X] 0003_remove_notification_category_make_code_uppercase
[X] 0004_auto_20200801_0817
contenttypes
[X] 0001_initial
[X] 0002_remove_content_type_name
customer
[X] 0001_initial
[X] 0002_auto_20150807_1725
[X] 0003_update_email_length
[X] 0004_email_save
[X] 0005_auto_20181115_1953
[X] 0006_auto_20190430_1736
[X] 0007_auto_20200801_0817
flatpages
[X] 0001_initial
offer
[X] 0001_initial
[X] 0002_auto_20151210_1053
[X] 0003_auto_20161120_1707
[X] 0004_auto_20170415_1518
[X] 0005_auto_20170423_1217
[X] 0006_auto_20170504_0616
[X] 0007_conditionaloffer_exclusive
[X] 0008_auto_20181115_1953
[X] 0009_auto_20200801_0817
[X] 0010_conditionaloffer_combinations
order
[X] 0001_initial
[X] 0002_auto_20141007_2032
[X] 0003_auto_20150113_1629
[X] 0004_auto_20160111_1108
[X] 0005_update_email_length
[X] 0006_orderstatuschange
[X] 0007_auto_20181115_1953
[X] 0008_auto_20190301_1035
[X] 0009_surcharge (1 squashed migrations)
[X] 0010_auto_20200724_0909
[X] 0011_auto_20200801_0817
partner
[X] 0001_initial
[X] 0002_auto_20141007_2032
[X] 0003_auto_20150604_1450
[X] 0004_auto_20160107_1755
[X] 0005_auto_20181115_1953
[X] 0006_auto_20200724_0909
payment
[X] 0001_initial
[X] 0002_auto_20141007_2032
[X] 0003_auto_20160323_1520
[X] 0004_auto_20181115_1953
[X] 0005_auto_20200801_0817
reviews
[X] 0001_initial
[X] 0002_update_email_length
[X] 0003_auto_20160802_1358
[X] 0004_auto_20170429_0941
sessions
[X] 0001_initial
shipping
[X] 0001_initial
[X] 0002_auto_20150604_1450
[X] 0003_auto_20181115_1953
sites
[X] 0001_initial
[X] 0002_alter_domain_unique
thumbnail
[X] 0001_initial
voucher
[X] 0001_initial
[X] 0002_auto_20170418_2132
[X] 0003_auto_20171212_0411
[X] 0004_auto_20180228_0940
[X] 0005_auto_20180402_1425
[X] 0006_auto_20180413_0911
[X] 0007_auto_20181115_1953
[X] 0008_auto_20200801_0817
wishlists
[X] 0001_initial
[X] 0002_auto_20160111_1108
[X] 0003_auto_20181115_1953 %}
Here is the first migration file, address/migrations/0001_initial.py:
address/migrations/0001_initial.py
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
import oscar.models.fields
from django.conf import settings
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='Country',
fields=[
('iso_3166_1_a2', models.CharField(primary_key=True, max_length=2, verbose_name='ISO 3166-1 alpha-2', serialize=False)),
('iso_3166_1_a3', models.CharField(max_length=3, verbose_name='ISO 3166-1 alpha-3', blank=True)),
('iso_3166_1_numeric', models.CharField(max_length=3, verbose_name='ISO 3166-1 numeric', blank=True)),
('printable_name', models.CharField(max_length=128, verbose_name='Country name')),
('name', models.CharField(max_length=128, verbose_name='Official name')),
('display_order', models.PositiveSmallIntegerField(default=0, verbose_name='Display order', db_index=True, help_text='Higher the number, higher the country in the list.')),
('is_shipping_country', models.BooleanField(default=False, db_index=True, verbose_name='Is shipping country')),
],
options={
'ordering': ('-display_order', 'printable_name'),
'verbose_name_plural': 'Countries',
'verbose_name': 'Country',
'abstract': False,
},
bases=(models.Model,),
),
migrations.CreateModel(
name='UserAddress',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('title', models.CharField(verbose_name='Title', max_length=64, blank=True, choices=[('Mr', 'Mr'), ('Miss', 'Miss'), ('Mrs', 'Mrs'), ('Ms', 'Ms'), ('Dr', 'Dr')])),
('first_name', models.CharField(max_length=255, verbose_name='First name', blank=True)),
('last_name', models.CharField(max_length=255, verbose_name='Last name', blank=True)),
('line1', models.CharField(max_length=255, verbose_name='First line of address')),
('line2', models.CharField(max_length=255, verbose_name='Second line of address', blank=True)),
('line3', models.CharField(max_length=255, verbose_name='Third line of address', blank=True)),
('line4', models.CharField(max_length=255, verbose_name='City', blank=True)),
('state', models.CharField(max_length=255, verbose_name='State/County', blank=True)),
('postcode', oscar.models.fields.UppercaseCharField(max_length=64, verbose_name='Post/Zip-code', blank=True)),
('search_text', models.TextField(editable=False, verbose_name='Search text - used only for searching addresses')),
('phone_number', oscar.models.fields.PhoneNumberField(verbose_name='Phone number', help_text='In case we need to call you about your order', blank=True)),
('notes', models.TextField(verbose_name='Instructions', help_text='Tell us anything we should know when delivering your order.', blank=True)),
('is_default_for_shipping', models.BooleanField(default=False, verbose_name='Default shipping address?')),
('is_default_for_billing', models.BooleanField(default=False, verbose_name='Default billing address?')),
('num_orders', models.PositiveIntegerField(default=0, verbose_name='Number of Orders')),
('hash', models.CharField(max_length=255, editable=False, db_index=True, verbose_name='Address Hash')),
('date_created', models.DateTimeField(auto_now_add=True, verbose_name='Date Created')),
('country', models.ForeignKey(verbose_name='Country', to='address.Country', on_delete=models.CASCADE)),
('user', models.ForeignKey(verbose_name='User', related_name='addresses', to=settings.AUTH_USER_MODEL, on_delete=models.CASCADE)),
],
options={
'ordering': ['-num_orders'],
'verbose_name_plural': 'User addresses',
'verbose_name': 'User address',
'abstract': False,
},
bases=(models.Model,),
),
migrations.AlterUniqueTogether(
name='useraddress',
unique_together=set([('user', 'hash')]),
),
]
sqlmigrate Subcommand
The SQL generated from the above migration file can be viewed with the
sqlmigrate subcommand
of manage.py.
Here is the sqlmigrate subcommand help:
Shell
(aw) $./manage.py sqlmigrate -h usage: manage.py sqlmigrate [-h] [--database DATABASE] [--backwards] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks] app_label migration_name Prints the SQL statements for the named migration. positional arguments: app_label App label of the application containing the migration. migration_name Migration name to print the SQL for. optional arguments: -h, --help show this help message and exit --database DATABASE Nominates a database to create SQL for. Defaults to the "default" database. --backwards Creates SQL to unapply the migration, rather than to apply it --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.
I tried using the --database option, so the generated SQL would refer to the proper database for my Frobshop web application,
but after examining the generated SQL I discovered that it did not mention the database by name.
Shell
(aw) $./manage.py sqlmigrate address 0001_initial BEGIN; -- -- Create model Country -- CREATE TABLE "address_country" ("iso_3166_1_a2" varchar(2) NOT NULL PRIMARY KEY, "iso_3166_1_a3" varchar(3) NOT NULL, "iso_3166_1_numeric" varchar(3) NOT NULL, "printable_name" varchar(128) NOT NULL, "name" varchar(128) NOT NULL, "display_order" smallint NOT NULL CHECK ("display_order" >= 0), "is_shipping_country" boolean NOT NULL); -- -- Create model UserAddress -- CREATE TABLE "address_useraddress" ("id" serial NOT NULL PRIMARY KEY, "title" varchar(64) NOT NULL, "first_name" varchar(255) NOT NULL, "last_name" varchar(255) NOT NULL, "line1" varchar(255) NOT NULL, "line2" varchar(255) NOT NULL, "line3" varchar(255) NOT NULL, "line4" varchar(255) NOT NULL, "state" varchar(255) NOT NULL, "postcode" varchar(64) NOT NULL, "search_text" text NOT NULL, "phone_number" varchar(128) NOT NULL, "notes" text NOT NULL, "is_default_for_shipping" boolean NOT NULL, "is_default_for_billing" boolean NOT NULL, "num_orders" integer NOT NULL CHECK ("num_orders" >= 0), "hash" varchar(255) NOT NULL, "date_created" timestamp with time zone NOT NULL, "country_id" varchar(2) NOT NULL, "user_id" integer NOT NULL); -- -- Alter unique_together for useraddress (1 constraint(s)) -- ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_user_id_hash_9d1738c7_uniq" UNIQUE ("user_id", "hash"); CREATE INDEX "address_country_iso_3166_1_a2_f395eed0_like" ON "address_country" ("iso_3166_1_a2" varchar_pattern_ops); CREATE INDEX "address_country_display_order_dc88cde8" ON "address_country" ("display_order"); CREATE INDEX "address_country_is_shipping_country_f7b6c461" ON "address_country" ("is_shipping_country"); ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_country_id_fa26a249_fk_address_c" FOREIGN KEY ("country_id") REFERENCES "address_country" ("iso_3166_1_a2") DEFERRABLE INITIALLY DEFERRED; ALTER TABLE "address_useraddress" ADD CONSTRAINT "address_useraddress_user_id_6edf0244_fk_auth_user_id" FOREIGN KEY ("user_id") REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED; CREATE INDEX "address_useraddress_hash_e0a6b290" ON "address_useraddress" ("hash"); CREATE INDEX "address_useraddress_hash_e0a6b290_like" ON "address_useraddress" ("hash" varchar_pattern_ops); CREATE INDEX "address_useraddress_country_id_fa26a249" ON "address_useraddress" ("country_id"); CREATE INDEX "address_useraddress_country_id_fa26a249_like" ON "address_useraddress" ("country_id" varchar_pattern_ops); CREATE INDEX "address_useraddress_user_id_6edf0244" ON "address_useraddress" ("user_id"); COMMIT;
sqlcreate Subcommand
The sqlcreate subcommand generates the SQL to create the database.
Here is the sqlcreate subcommand help:
Shell
(aw) $./manage.py sqlcreate -h usage: manage.py sqlcreate [-h] [-R ROUTER] [--database DATABASE] [-D] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] Generates the SQL to create your database for you, as specified in settings.py The envisioned use case is something like this: ./manage.py sqlcreate [--database=<databasename>] | mysql -u <db_administrator> -p ./manage.py sqlcreate [--database=<databasname>] | psql -U <db_administrator> -W optional arguments: -h, --help show this help message and exit -R ROUTER, --router ROUTER Use this router-database other then defined in settings.py --database DATABASE Nominates a database to run command for. Defaults to the "default" database. -D, --drop If given, includes commands to drop any existing user and database. --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output.
The output of the sqlcreate subcommand can be piped into the dbshell subcommand,
and then all migrations can be applied, like this:
Shell
(aw) $./manage.py sqlcreate | \
./manage.py dbshell && ./manage.py migrate
Resetting Migrations
I found myself resetting migrations several as I stumbled forward in my learning experience.
Vitor Freitas wrote a helpful article
that allowed me to write the following script for resetting migrations and setting up the database with my desired
django-oscar shipping information:
reset
#!/bin/bash
# Reset migrations and set up the database with only US & Canada enabled for shipping
DB=ancient_warmth
set -e
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
dropdb -U postgres -h localhost $DB
createdb -U postgres -h localhost $DB
./manage.py migrate --fake-initial # See https://stackoverflow.com/a/37371482/553865
./manage.py makemigrations
./manage.py migrate
./manage.py oscar_populate_countries --no-shipping
bin/psql -c "update address_country set is_shipping_country = 't' where iso_3166_1_a2 in ('CA', 'US');"
DJANGO_SUPERUSER_EMAIL=admin@domain.com
DJANGO_SUPERUSER_PASSWORD=secret
./manage.py createsuperuser --noinput --username admin
This is my bin/psql script, used by the above:
bin/psql
#!/bin/bash
if [ "$1" == -t ]; then
PREFIX="test_"
shift
fi
PGPASSWORD=secret /usr/bin/psql \
-U postgres \
-h localhost \
-d ${PREFIX}ancient_warmth \
"$@"