Django Models, Automatic Form Generation, Data Backup & Restore

Published 2021-04-04.
Time to read: 3 minutes.

This page is part of the django collection.

Django has a super-useful feature: Automatic HTML forms created for model CRUD (Create, Read, Update, Delete). The MDN Django Tutorial has a page on this feature.

Django's automatic CRUD form generation feature is degraded by poor integration with WYSIWYG HTML editors

Every Django model automatically maps to a database table. Programmatically, new rows can be added to the table by creating new model instances. Those instances can be edited or deleted programmatically as well.

Automatic CRUD Form Generation

The wonderful thing is that every Django automatally creates an HTML forms-based editor for every model. These CRUID forms are easily to modify: their appearance can be styled and the content can be laid out easily. Just visit the admin/ section of a Django website, for example: http://localhost:8000/admin. You will see a listing of all the model classes. If you are working with django-oscar, those classes will also be listed, grouped by Django app name.

When I click on one of the apps I created, called Pricing, the browser showed all the model classes defined at http://localhost:8000/adminpricing/. Clicking on my Chemical model class listed all the instances already defined (there were none) at http://localhost:8000/adminpricing/chemical/. When I clicked on the + Add chemical button at the top right of the screen, I was presented with a form-based editor for all the model fields. This is the list page for my Ancient Warmth Core app's Chemical model:

I took advantage of Django's controls for laying out the information on this page. Most of the Chemical fields were listed, and I created @propertys when a foreign key pointed to something interesting.

Portion of core/model.py

@admin.register(models.Chemical)
class ChemicalAdmin(admin.ModelAdmin):
  list_display = ('name', 'full_name', 'formula', 'remaining_inventory',
      'retail_price_per_kg', 'recommended_max_percent', 'designs',
      'bathers', 'usage_rate')
  ordering = ('name',)
  search_fields = ('name', 'formula')

list_display: Specifies the model fields and properties to show from a Django model
ordering: Specifies the model fields to use as sort keys
search_fields: Specifies the model fields to search withing

WSYWIG HTML Editor Fail

I wrote a short piece on Django WYSIWYG HTML editors yesterday. I don't know the full story yet. Seems like one of Django's potential strengths is being significantly weakened by the disarray with respect to official interfacing of at least one current WSIWYG HTML editors.

I had previously installed Grapelli, so I expected to see the TinyMCE WYSIWYG HTML editor visible for the Text fields, but that did not happen. I looked around for documentation, but only found a confusing set of out-of-date or poorly written instructions that could not be followed. What a mess! Maybe that is why other WYSIWYG editors are talked about – however, I have not seen good installation documentation for those either yet.

Field Order

The order that a model's fields appear in the web page can be controlled by defining a companion class in the same models.py file. For a model class called MyModel, the companion class is called MyModelForm. An inner class called Meta defines the field order, like this:

Shell

class MyModelForm(ModelForm):
    class Meta:
        model = Chemical
        fields = [
            'field1',
            'field2',
            'field3',
        ]

This worked, and then the order flipped back to the order the fields were declared in models.py. Another mystery. Django is a mixture of wonderfulness and neglected bits, chattering to each other, rotting forgotten in a dank, dark corner.

Database Table Backup

Now that some data has been entered into a model's database tables, you probably want to back it up. Django stores the data for a model class called MyModel in an app called MyApp in a table called MyAppMyModel.

I wrote the following script to back up the test data I was working with.

backup

#!/bin/bash

./manage.py dumpdata auth.user > backup/auth_user.json

APP=myApp
for X in model_1_name model_2_name model_3_name ; do
  ./manage.py dumpdata $APP.$X > backup/$APP.$X.json
done

Dumpdata Subcommand Help

To back up the data for a model in JSON format, use the dumpdata subcommand of manage.py. This is the help message:

Shell

(aw) $ ./manage.py dumpdata --help
usage: manage.py dumpdata [-h] [--format FORMAT] [--indent INDENT] [--database DATABASE] [-e EXCLUDE] [--natural-foreign] [--natural-primary] [-a]
                          [--pks PRIMARY_KEYS] [-o OUTPUT] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback]
                          [--no-color] [--force-color] [--skip-checks]
                          [app_label[.ModelName] [app_label[.ModelName] ...]]

Output the contents of the database as a fixture of the given format (using each model's default manager unless --all is specified).

positional arguments:
  app_label[.ModelName]
                        Restricts dumped data to the specified app_label or app_label.ModelName.

optional arguments:
  -h, --help            show this help message and exit
  --format FORMAT       Specifies the output serialization format for fixtures.
  --indent INDENT       Specifies the indent level to use when pretty-printing output.
  --database DATABASE   Nominates a specific database to dump fixtures from. Defaults to the "default" database.
  -e EXCLUDE, --exclude EXCLUDE
                        An app_label or app_label.ModelName to exclude (use multiple --exclude to exclude multiple apps/models).
  --natural-foreign     Use natural foreign keys if they are available.
  --natural-primary     Use natural primary keys if they are available.
  -a, --all             Use Django's base manager to dump all models stored in the database, including those that would otherwise be filtered or
                        modified by a custom manager.
  --pks PRIMARY_KEYS    Only dump objects with given primary keys. Accepts a comma-separated list of keys. This option only works when you specify one
                        model.
  -o OUTPUT, --output OUTPUT
                        Specifies file to which the output is written.
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE
                        environment variable will be used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g. "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --force-color         Force colorization of the command output.
  --skip-checks         Skip system checks.

Using the Dumpdata Subcommand

The dumpdata subcommand needs the app and model field names to be specified in lower case.

For example, to extract data for all the models in the MyApp app in JSON format, type:

Shell

(aw) $ ./manage.py dumpdata myapp

Of course, you can direct the JSON to a file. Lets first make a backup/ directory:

Shell

(aw) $ mkdir backup

(aw) $ ./manage.py dumpdata myapp > backup/myapp.json

To extract data for just the MyModel model MyApp app in JSON format:

Shell

(aw) $ ./manage.py dumpdata myapp.mymodel

To direct the JSON for this one field to a file:

Shell

(aw) $ ./manage.py dumpdata myapp > backup/myapp.mymodel.json

Database Table Restore

Loaddata Subcommand Help

To restore the data from a backup for a model in JSON format, use the loaddata subcommand of manage.py. This is the help message:

Shell

(aw) $ ./manage.py loaddata --help
usage: manage.py loaddata [-h] [--database DATABASE] [--app APP_LABEL] [--ignorenonexistent] [-e EXCLUDE] [--format FORMAT] [--version] [-v {0,1,2,3}]
[--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks]
fixture [fixture ...]

Installs the named fixture(s) in the database.

positional arguments:
fixture               Fixture labels.

optional arguments:
-h, --help            show this help message and exit
--database DATABASE   Nominates a specific database to load fixtures into. Defaults to the "default" database.
--app APP_LABEL       Only look for fixtures in the specified app.
--ignorenonexistent, -i
Ignores entries in the serialized data for fields that do not currently exist on the model.
-e EXCLUDE, --exclude EXCLUDE
An app_label or app_label.ModelName to exclude. Can be used multiple times.
--format FORMAT       Format of serialized data when reading from stdin.
--version             show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output
--settings SETTINGS   The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE
environment variable will be used.
--pythonpath PYTHONPATH
A directory to add to the Python path, e.g. "/home/djangoprojects/myproject".
--traceback           Raise on CommandError exceptions
--no-color            Don't colorize the command output.
--force-color         Force colorization of the command output.
--skip-checks         Skip system checks.

Using the Loaddata Subcommand

The loaddata subcommand just needs the name of the backup file to read.

Shell

(aw) $ ./manage.py loaddata backup/myapp.json

I wrote this script to restore from the JSON backups. Seems I am scrubbing and recreating the database every few hours right now.

restore

#!/bin/bash

./manage.py dumpdata auth.user > backup/auth_user.json

APP=myApp
for X in model_1_name model_2_name model_3_name ; do
  ./manage.py dumpdata $APP.$X > backup/$APP.$X.json
done
for X in model_1_name model_2_name model_3_name; do
  FN="backup/$APP.$X.json"
  if [ -f "$FN" ]; then
    ./manage.py loaddata "backup/$APP.$X.json"
  fi
done