Enabling and Disabling the Django AWS Datastore

Published 2021-03-13.
Time to read: 1 minutes.

This page is part of the django collection.

After writing this I discovered that the Django AWS datastore had a serious bug, which necessitated the use of this feature in my development.

Managing Collected Files

If you invoke the collectstatic subcommand like this, pre-existing files in STATIC_ROOT are not deleted:

Shell

(aw) $ ./manage.py collectstatic \
  --verbosity 0
AWS datastore disabled; using local storage for assets instead.

You have requested to collect static files at the destination
location as specified in your settings.

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes

Providing the --clear option is a good idea because it removes all files from STATIC_ROOT before processing:

Shell

(aw) $ ./manage.py collectstatic \
  --force-local-storage \
  --clear \
  --verbosity 0
AWS datastore disabled; using local storage for assets instead.
STATICFILES_DIRS=['/var/work/django/frobshop']

You have requested to collect static files at the destination
location as specified in your settings:

    /var/work/django/frobshop/static-test

This will DELETE ALL FILES in this location!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel:

Quick Summary

It is quicker and easier to debug asset-related problems when you are certain that the assets would be found if a local filesystem was used as the datastore. To configure settings so the AWS datastore can be temporarily disabled, only 3 things are required:

Implementing an optional command-line parameter for manage.py, which I call --force-local-storage.
A check for the presence of the optional parameter in settings for when it is evaluated by manage.py.
Conditional logic for setting AWS_STORAGE_BUCKET_NAME, DEFAULT_FILE_STORAGE and STATICFILES_STORAGE.

--force-local-storage settings

Add the following code to the top of settings, so the optional parameter is handled properly:

Excerpt from settings/base.py

import sys

if "--force-local-storage" in sys.argv:
    print("AWS datastore disabled; using local storage for assets instead.")
    FORCE_LOCAL_STORAGE = True
    sys.argv.remove("--force-local-storage")
else:
    print("Using AWS datastore for assets.")
    FORCE_LOCAL_STORAGE = False

Settings Modifications

Replace the previous settings for AWS_STORAGE_BUCKET_NAME, DEFAULT_FILE_STORAGE and STATICFILES_STORAGE with the following near the top of settings:

Excerpt from settings/base.py

if "--force-local-storage" in sys.argv:
    print("AWS datastore disabled; using local storage for assets instead.")
    FORCE_LOCAL_STORAGE = True
    sys.argv.remove("--force-local-storage")
else:
    print("Using AWS datastore for assets.")
    FORCE_LOCAL_STORAGE = False

This goes after the definition of INSTALLED_APPS

Excerpt from settings/base.py

if not FORCE_LOCAL_STORAGE:
    AWS_DEFAULT_ACL = 'public-read'
    AWS_STORAGE_BUCKET_NAME = 'assets.ancientwarmth.com'
    STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'

Run Script

Here is my run script, which provides a help message for the optional parameter:

run

#!/bin/bash

function help {
  echo "
$(basename $0) - Run Django-Oscar webapp locally.
Syntax: $(basename $0) [OPTIONS]

Options are:
  --force-local-storage  Disable AWS datastore and use local storage instead
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [[ `service postgresql status` == **down ]]; then
  sudo service postgresql start
fi

./manage.py runserver --noreload $* 0.0.0.0:8001

This is the help message:

Shell

(aw) $ bin/run -h

run - Run Django-Oscar webapp locally.
Syntax: run [OPTIONS]

Options are:
  --force-local-storage  Disable AWS datastore and use local storage instead

Example Usage

Specifying the --force-local-storage option causes the collected assets to be collected and stored locally without regard to AWS.

Shell

(aw) $ ./manage.py collectstatic --force-local-storage
AWS datastore disabled; using local storage for assets instead.
(Input) STATICFILES_DIRS=['/var/work/django/frobshop/static']
(Output) STATIC_ROOT=/var/work/django/frobshop/collected_static_assets

You have requested to collect static files at the destination
location as specified in your settings.

This will overwrite existing files!
Are you sure you want to do this? yes

(aw) $ ls collected_static_assets/
admin  debug_toolbar  django_extensions  django_tables2  oscar  treebeard