Django and Oscar
Mike Slinn

Enabling and Disabling the Django AWS Datastore

Published 2021-03-13.
Time to read: 1 minutes.

This page is part of the django collection.

After writing this I discovered that the Django AWS datastore had a serious bug, which necessitated the use of this feature in my development.

Managing Collected Files

If you invoke the collectstatic subcommand like this, pre-existing files in STATIC_ROOT are not deleted:

Shell
(aw) $ ./manage.py collectstatic \
  --verbosity 0
AWS datastore disabled; using local storage for assets instead.
You have requested to collect static files at the destination location as specified in your settings.
This will overwrite existing files! Are you sure you want to do this?
Type 'yes' to continue, or 'no' to cancel:
yes

Providing the --clear option is a good idea because it removes all files from STATIC_ROOT before processing:

Shell
(aw) $ ./manage.py collectstatic \
  --force-local-storage \
  --clear \
  --verbosity 0
AWS datastore disabled; using local storage for assets instead.
STATICFILES_DIRS=['/var/work/django/frobshop']
You have requested to collect static files at the destination location as specified in your settings:
/var/work/django/frobshop/static-test
This will DELETE ALL FILES in this location! Are you sure you want to do this?
Type 'yes' to continue, or 'no' to cancel:

Quick Summary

It is quicker and easier to debug asset-related problems when you are certain that the assets would be found if a local filesystem was used as the datastore. To configure settings so the AWS datastore can be temporarily disabled, only 3 things are required:

  1. Implementing an optional command-line parameter for manage.py, which I call --force-local-storage.
  2. A check for the presence of the optional parameter in settings for when it is evaluated by manage.py.
  3. Conditional logic for setting AWS_STORAGE_BUCKET_NAME, DEFAULT_FILE_STORAGE and STATICFILES_STORAGE.

--force-local-storage settings

Add the following code to the top of settings, so the optional parameter is handled properly:

Excerpt from settings/base.py
import sys

if "--force-local-storage" in sys.argv:
    print("AWS datastore disabled; using local storage for assets instead.")
    FORCE_LOCAL_STORAGE = True
    sys.argv.remove("--force-local-storage")
else:
    print("Using AWS datastore for assets.")
    FORCE_LOCAL_STORAGE = False

Settings Modifications

Replace the previous settings for AWS_STORAGE_BUCKET_NAME, DEFAULT_FILE_STORAGE and STATICFILES_STORAGE with the following near the top of settings:

Excerpt from settings/base.py
if "--force-local-storage" in sys.argv:
    print("AWS datastore disabled; using local storage for assets instead.")
    FORCE_LOCAL_STORAGE = True
    sys.argv.remove("--force-local-storage")
else:
    print("Using AWS datastore for assets.")
    FORCE_LOCAL_STORAGE = False

This goes after the definition of INSTALLED_APPS

Excerpt from settings/base.py
if not FORCE_LOCAL_STORAGE:
    AWS_DEFAULT_ACL = 'public-read'
    AWS_STORAGE_BUCKET_NAME = 'assets.ancientwarmth.com'
    STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'

Run Script

Here is my run script, which provides a help message for the optional parameter:

#!/bin/bash

function help {
  echo "
$(basename $0) - Run Django-Oscar webapp locally.
Syntax: $(basename $0) [OPTIONS]

Options are:
  --force-local-storage  Disable AWS datastore and use local storage instead
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [[ `service postgresql status` == **down ]]; then
  sudo service postgresql start
fi

./manage.py runserver --noreload $* 0.0.0.0:8001

This is the help message:

Shell
(aw) $ bin/run -h
run - Run Django-Oscar webapp locally. Syntax: run [OPTIONS]
Options are: --force-local-storage Disable AWS datastore and use local storage instead

Example Usage

Specifying the --force-local-storage option causes the collected assets to be collected and stored locally without regard to AWS.

Shell
(aw) $ ./manage.py collectstatic --force-local-storage
AWS datastore disabled; using local storage for assets instead.
(Input) STATICFILES_DIRS=['/var/work/django/frobshop/static']
(Output) STATIC_ROOT=/var/work/django/frobshop/collected_static_assets
You have requested to collect static files at the destination location as specified in your settings.
This will overwrite existing files! Are you sure you want to do this?
yes
(aw) $ ls collected_static_assets/ admin debug_toolbar django_extensions django_tables2 oscar treebeard


* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.