Published 2021-02-11.
Last modified 2021-04-06.
Time to read: 8 minutes.
django collection.
I discussed why I decided to check out
the django-oscar e-commerce framework in an
Shopping Carts Powered by Python and Django.
In this article I describe my experience setting up and running django v3.1.6 and django-oscar v3.0 on Ubuntu 20.04 with Python 3.8.6.
BTW, there are two online django-oscar user groups: the
Slack channels
and the Google Group.
Getting Started
The very first page in the
django-oscar documentation
describes a sample project called Frobshop.
This article describes my experience installing Frobshop.
Etymology: It was adopted by the community of computer programmers which grew out of the MIT Tech Model Railroad Club in the 1950s, and allegedly among the oldest existing words in hacker jargon.
– Definitions.net
I disabled Django Haystack for Frobshop, and I wrote Suggestions for Django-Oscar and Django-Haystack to explain why. I also did not install Apache Solr, as explained in Installing Apache Solr on Ubuntu 20.04.
Installing django-oscar
For manageability of Python runtime versions and libraries, a virtual Python environment should be set up.
The required virtual environment will house compatible versions of Python, Django, Django-Oscar and all their dependencies.
I decided to locate a virtual environment at ~/venv/aw as explained in
A Python Virtual Environment For Every Project:
$ newVenv aw created virtual environment CPython3.8.6.final.0-64 in 528ms creator CPython3Posix(dest=/home/mslinn/venv/aw, clear=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mslinn/.local/share/virtualenv) added seed packages: pip==20.1.1, pkg_resources==0.0.0, setuptools==44.0.0, wheel==0.34.2 activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
$ use aw
(aw) $
Next, I installed the Python libraries and their dependencies.
Django‑oscar installs Django as a dependency.
I also specified a few more dependencies that do not automatically get pulled in.
Pip‑tools is installed so dependencies can be managed properly.
I installed django.db.backends.postgresql because I wanted to use PostgreSQL instead of SQLite.
BTW, django.db.backends.postgresql_psycopg2 was renamed in django 1.9 to django.db.backends.postgresql,
and is installed by pip as django‑postgresql.
(aw) $ pip install django-oscar[sorl-thumbnail] \ pip-tools psycopg2-binary pycountry
BTW, the Frobshop instructions say to install Pillow,
but that library is
already specified as a django-oscar dependency,
so there is no need to install it manually.
Now the virtual environment also contains 2 Django executable images:
/home/mslinn/venv/aw/pyvenv.cfg /home/mslinn/venv/aw/bin: __pycache__ activate activate.csh activate.fish activate.ps1 activate.xsh activate_this.py django-admin django-admin.py easy_install easy_install-3.8 easy_install3 faker pip pip3 pip3.10 pip3.11 pybabel python python3 python3.8 sqlformat wheel wheel-3.8 wheel3 /home/mslinn/venv/aw/lib: python3.10 python3.11 python3.8 python3.9
Let’s see information about the versions of django and django-oscar that were just installed into the oscar virtual environment:
(aw) $ pip show django django-oscar Name: Django Version: 3.1.6 Summary: A high-level Python Web framework that encourages rapid development and clean, pragmatic design. Home-page: https://www.djangoproject.com/ Author: Django Software Foundation Author-email: foundation@djangoproject.com License: BSD-3-Clause Location: /home/mslinn/oscar/lib/python3.8/site-packages Requires: asgiref, sqlparse, pytz Required-by: django-treebeard, django-tables2, django-phonenumber-field, django-oscar, django-haystack, django-extra-views --- Name: django-oscar Version: 3.0 Summary: A domain-driven e-commerce framework for Django Home-page: https://github.com/django-oscar/django-oscar Author: David Winterbottom Author-email: david.winterbottom@gmail.com License: BSD Location: /home/mslinn/oscar/lib/python3.8/site-packages Requires: django-haystack, phonenumbers, django-widget-tweaks, django-treebeard, django-extra-views, Babel, purl, factory-boy, django-tables2, django-phonenumber-field, pillow, django Required-by:
Generating Frobshop
Now it was time to generate the frobshop project from the template:
(aw) $ mkdir $work/django && cd $work/django (aw) $ django-admin startproject frobshop (aw) $ cd frobshop
I now had these files and directories:
(aw) $ tree
├── db.sqlite3
├── frobshop
│ ├── __init__.py
│ ├── __pycache__
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
└── manage.py %}
For convenience, I made frobshop’s copy of the manage.py Django command-line utility for administrative tasks executable:
(aw) $ chmod a+x manage.py
Creating a Git Repository
I checked in the frobshop project into a private git repository of the same name:
(aw) $ git init (aw) $ hub create -p frobshop (aw) $ cat > .gitignore <<EOF **/__pycache__/ .idea/ .vscode/ EOF (aw) $ git add . (aw) $ git commit -m - (aw) $ git push origin master
As I worked through the tutorial I used git commits, branches and tags to insulate me from messing things up while experimenting.
Git made it easy to restore the project to previous states, and to see what had changed between commits.
My commit script saved a lot of typing.
#!/bin/bash
# Originally written May 9/05 by Mike Slinn
function help {
echo "Runs git commit without prompting for a message."
echo "Usage: commit [options] [file...]"
echo " Where options are:"
echo " -a \"tag message\""
echo " -d # enables debug mode"
echo " -m \"commit message\""
echo "Examples:"
echo " commit # The default commit message is just a single dash (-)"
echo " commit -m \"This is a commit message\""
echo " commit -a 0.1.2"
exit 1
}
function isGitProject {
cd "$( git rev-parse --git-dir )/.."
[ -d .git ]
}
BRANCH="$(git rev-parse --abbrev-ref HEAD)"
MSG=""
while getopts "a:dhm:?" opt; do
case $opt in
a ) TAG="$OPTARG"
git tag -a "$TAG" -m "v$TAG"
git push origin --tags
exit
;;
d ) set -xv ;;
m ) MSG="$OPTARG" ;;
h ) help ;;
\?) help ;;
esac
done
shift "$((OPTIND-1))"
for o in "$@"; do
if [ "$o" == "-m" ]; then unset MSG; fi
done
if isGitProject; then
if [ "$@" ]; then
git add -A "$@"
else
git add -A .
fi
shift
if [ "$MSG" == "" ]; then MSG="-"; fi
git commit -m "$MSG" "$@" 3>&1 1>&2 2>&3 | sed -e '/^X11/d' | sed -e '/^Warning:/d'
git push origin "$BRANCH" --tags 3>&1 1>&2 2>&3 | sed -e '/^X11/d' | sed -e '/^Warning:/d'
else
echo "Error: '$( pwd )' is not a git project"
fi
if [ -f 0 ]; then rm -f 0; fi
(aw) $ commit -h Runs git commit without prompting for a message. Usage: commit [options] [file...] Where options are: -a "tag message" -d # enables debug mode -m "commit message" Examples: commit # The default commit message is just a single dash (-) commit -m "This is a commit message" commit -a 0.1.2
manage.py Help
The manage.py script can do many things.
Here are its sub-commands:
(aw) $ ./manage.py Type 'manage.py help <subcommand>' for help on a specific subcommand. Available subcommands: [auth] changepassword createsuperuser [contenttypes] remove_stale_contenttypes [django] check compilemessages createcachetable dbshell diffsettings dumpdata flush inspectdb loaddata makemessages makemigrations migrate sendtestemail shell showmigrations sqlflush sqlmigrate sqlsequencereset squashmigrations startapp startproject test testserver [haystack] build_solr_schema clear_index haystack_info rebuild_index update_index [oscar] oscar_calculate_scores oscar_cleanup_alerts oscar_find_duplicate_emails oscar_fork_app oscar_fork_statics oscar_generate_email_content oscar_import_catalogue oscar_import_catalogue_images oscar_populate_countries oscar_send_alerts oscar_update_product_ratings [sessions] clearsessions [staticfiles] collectstatic findstatic runserver [thumbnail] thumbnail
requirements.txt
I found it strange that Frobshop did not provide requirements.txt to install managed dependencies.
I created the file within the frobshop directory so I could replicate this installation on other machines:
(aw) $ pip freeze > requirements.txt
The file looked like this:
appdirs==1.4.3 asgiref==3.3.1 Babel==2.9.0 CacheControl==0.12.6 certifi==2019.11.28 chardet==3.0.4 colorama==0.4.3 contextlib2==0.6.0 distlib==0.3.0 distro==1.4.0 Django==3.1.6 django-extra-views==0.13.0 django-haystack==3.0 django-oscar==3.0 django-phonenumber-field==3.0.1 django-tables2==2.3.4 django-treebeard==4.4 django-widget-tweaks==1.4.8 factory-boy==2.12.0 Faker==6.1.1 html5lib==1.0.1 idna==2.8 ipaddr==2.2.0 lockfile==0.12.2 msgpack==0.6.2 packaging==20.3 pep517==0.8.2 phonenumbers==8.12.18 Pillow==8.1.0 progress==1.5 purl==1.5 pycountry==20.7.3 pyparsing==2.4.6 python-dateutil==2.8.1 pytoml==0.1.21 pytz==2021.1 requests==2.22.0 retrying==1.3.3 six==1.14.0 sorl-thumbnail==12.6.3 sqlparse==0.4.1 text-unidecode==1.3 urllib3==1.25.8 webencodings==0.5.1
I think that requirements.txt should have been provided with Frobshop.
If it had been provided, all one would have to do to install the proper version of all the Python modules would be to type:
(aw) $ pip install -r requirements.txt
This was a good time to commit my work to the git repository:
(aw) $ commit -m "First cut at dependencies"
Configuring Frobshop
Here is a comprehensive list of all django-oscar settings.
This is a list of all django-oscar default values.
Maximizing Django Oscar Settings Variables discusses some poorly documented settings.
The Frobshop documentation says to make massive changes to configuration files, without explaining why. No references are given, but I linked to all the documentation I could find. It seems some core apps, essential to proper operation, are undocumented.
Caution: the contents of these files change quite a bit between recent Django versions, so be warned that the config files are tightly bound to the Django version that it was generated for. This means that if you see a solution on StackOverflow to a problem that you currently have, blindly applying that solution may cause more problems instead of fixing them.
Here are the configured files that were altered, including a few changes I found necessary to make things work.
"""
Django settings for frobshop project.
Generated by 'django-admin startproject' using Django 3.1.6.
For more information on this file, see
https://docs.djangoproject.com/en/3.1/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/3.1/ref/settings/
"""
from pathlib import Path
from oscar.defaults import *
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.1/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'irs@7(qzjki2)gv7%b)zj5$t-6!l^kod0n4+!&hbgcku&b=$iv'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = ['*']
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'django.contrib.flatpages',
'oscar.config.Shop',
'oscar.apps.analytics.apps.AnalyticsConfig',
'oscar.apps.checkout.apps.CheckoutConfig',
'oscar.apps.address.apps.AddressConfig',
'oscar.apps.shipping.apps.ShippingConfig',
'oscar.apps.catalogue.apps.CatalogueConfig',
'oscar.apps.catalogue.reviews.apps.CatalogueReviewsConfig',
'oscar.apps.communication.apps.CommunicationConfig',
'oscar.apps.partner.apps.PartnerConfig',
'oscar.apps.basket.apps.BasketConfig',
'oscar.apps.payment.apps.PaymentConfig',
'oscar.apps.offer.apps.OfferConfig',
'oscar.apps.order.apps.OrderConfig',
'oscar.apps.customer.apps.CustomerConfig',
'oscar.apps.search.apps.SearchConfig',
'oscar.apps.voucher.apps.VoucherConfig',
'oscar.apps.wishlists.apps.WishlistsConfig',
'oscar.apps.dashboard.apps.DashboardConfig',
'oscar.apps.dashboard.reports.apps.ReportsDashboardConfig',
'oscar.apps.dashboard.users.apps.UsersDashboardConfig',
'oscar.apps.dashboard.orders.apps.OrdersDashboardConfig',
'oscar.apps.dashboard.catalogue.apps.CatalogueDashboardConfig',
'oscar.apps.dashboard.offers.apps.OffersDashboardConfig',
'oscar.apps.dashboard.partners.apps.PartnersDashboardConfig',
'oscar.apps.dashboard.pages.apps.PagesDashboardConfig',
'oscar.apps.dashboard.ranges.apps.RangesDashboardConfig',
'oscar.apps.dashboard.reviews.apps.ReviewsDashboardConfig',
'oscar.apps.dashboard.vouchers.apps.VouchersDashboardConfig',
'oscar.apps.dashboard.communications.apps.CommunicationsDashboardConfig',
'oscar.apps.dashboard.shipping.apps.ShippingDashboardConfig',
# 3rd-party apps that oscar depends on
'widget_tweaks',
'haystack',
'treebeard',
'sorl.thumbnail', # Default thumbnail backend, can be replaced
'django_tables2',
]
SITE_ID = 1
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'oscar.apps.basket.middleware.BasketMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
]
ROOT_URLCONF = 'frobshop.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
'oscar.apps.search.context_processors.search_form',
'oscar.apps.checkout.context_processors.checkout',
'oscar.apps.communication.notifications.context_processors.notifications',
'oscar.core.context_processors.metadata',
],
},
},
]
WSGI_APPLICATION = 'frobshop.wsgi.application'
# Database
# https://docs.djangoproject.com/en/3.1/ref/settings/#databases
DATABASES = {
'default': {
#'ENGINE': 'django.db.backends.sqlite3',
#'NAME': BASE_DIR / 'db.sqlite3',
'ENGINE': 'django.db.backends.postgresql',
'HOST': 'localhost',
'NAME': 'frobshop',
'PASSWORD': 'secret',
'PORT': '5432',
'USER': 'postgres',
'ATOMIC_REQUESTS': True,
}
}
# Password validation
# https://docs.djangoproject.com/en/3.1/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
AUTHENTICATION_BACKENDS = (
'oscar.apps.customer.auth_backends.EmailBackend',
'django.contrib.auth.backends.ModelBackend',
)
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.simple_backend.SimpleEngine',
#'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
#'URL': 'http://127.0.0.1:8983/solr/tester', # Assuming you created a core named 'tester' as described in installing search engines.
#'ADMIN_URL': 'http://127.0.0.1:8983/solr/admin/cores'
# ...or for multicore...
# 'URL': 'http://127.0.0.1:8983/solr/mysite',
},
}
# Internationalization
# https://docs.djangoproject.com/en/3.1/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'America/New_York'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.1/howto/static-files/
STATIC_URL = '/static/'
Here is documentation that helps explain what the above application-related settings above pertain to:
Django’s routing table is defined using urls.py:
"""
frobshop URL Configuration
The `urlpatterns` list routes URLs to views. For more information please see:
https://docs.djangoproject.com/en/3.1/topics/http/urls/
Examples:
Function views
1. Add an import: from my_app import views
2. Add a URL to urlpatterns: path('', views.home, name='home')
Class-based views
1. Add an import: from other_app.views import Home
2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
Including another URLconf
1. Import the include() function: from django.urls import include, path
2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
# path('i18n/', include('django.conf.urls.i18n')),
path('admin/', admin.site.urls),
path('', include(apps.get_app_config('oscar').urls[0])),
]
Time once again to commit my work:
(aw) $ commit -m "Initial settings."
Installing PostgreSQL
(aw) $ sudo curl https://www.pgadmin.org/static/packages_pgadmin_org.pub | sudo apt-key add (aw) $ sudo sh -c 'echo "deb https://ftp.postgresql.org/pub/pgadmin/pgadmin4/apt/$(lsb_release -cs) pgadmin4 main" > /etc/apt/sources.list.d/pgadmin4.list && apt update' (aw) $ yes | sudo apt install postgresql pgadmin4 (aw) $ sudo service postgresql start (aw) $ sudo -u postgres /usr/bin/psql -h localhost -c "ALTER USER postgres PASSWORD 'whatever';"
Creating the Database
I had previously installed PostgreSQL.
I just needed to create an empty database called frobshop.
One option for doing that would be to run psql and pass it parameters that indicate the network node and user to connect, then create the database:
(aw) $ /usr/bin/psql -U postgres \
-h localhost \
-c 'create database frobshop'
BTW, I had originally followed the Frobshop tutorial's suggestion to use SQLite.
I then tried to convert the frobshop SQLite database to PostgreSQL like this, but the conversion failed:
(aw) $ sudo apt install pgloader (aw) $ PGPASSWORD=secret /bin/createdb -U postgres -h localhost frobshop (aw) $ PGPASSWORD=secret pgloader -U postgres -h localhost \ db.sqlite3 postgresql:///frobshop (aw) $ PGPASSWORD=secret pgloader -U postgres \ ./sqlite.to.postgres
load database from db.sqlite3 into postgresql:///frobshop with include drop, create tables, create indexes, reset sequences set work_mem to '16MB', maintenance_work_mem to '512 MB';
It would make sense to just set up PostgreSQL straight away if you have any desire to understand how django-oscar works.
Since this step did not change any Frobshop files, there was nothing to commit to the git repository.
oscar_populate_countries Help
A django-oscar website will not work until country information is defined.
The oscar_populate_countries
subcommand of manage.py defines countries.
Here is the help information for that subcommand:
(aw) $ ./manage.py oscar_populate_countries -h usage: manage.py oscar_populate_countries [-h] [--no-shipping] [--initial-only] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks] Populates the list of countries with data from pycountry. optional arguments: -h, --help show this help message and exit --no-shipping Don<t mark countries for shipping --initial-only Exit quietly without doing anything if countries were already populated. --version show program<s version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn<t provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don<t colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.
Alternative Invocation Style
It is also possible to invoke the manage program like this:
(aw) $ python -m manage oscar_populate_countries -h
Defining Countries
I just want to be able to ship to Canada and the USA. Several steps are required to do that:
- Invoke
oscar_populate_countrieswith--no-shippingShell(aw) $ ./manage.py oscar_populate_countries --no-shipping Successfully added 249 countries.
- You can enable shipping to selected countries two ways:
-
Command-line instructions:
Shell
(aw) $ psql -c "update address_country set is_shipping_country = 't' where iso_3166_1_a2 in ('CA', 'US');" -
Web Admin instructions
-
Use
django-admin(for me that washttp://localhost:8001/admin/). -
Navigate to the
country editor.
- Click on Canada.
-
Enable Is shipping country and press Save.
- Click on Countries at the top of the left-hand menu.
-
Type
United Statesinto the text search field, click on United States, enable Is shipping country and press Save.
-
Use
-
Command-line instructions:
Since this step did not change any Frobshop files, there was nothing to commit to the git repository.
manage.py migrate Options
The Frobshop tutorial does not define data migrations very well. This is a gentle introduction to Django migrations. This is the official documentation, and this is information about writing database migrations.
The help for manage.py migrate is:
(aw) $ ./manage.py migrate -h usage: manage.py migrate [-h] [--noinput] [--database DATABASE] [--fake] [--fake-initial] [--plan] [--run-syncdb] [--check] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks] [app_label] [migration_name]
Updates database schema. Manages both apps with migrations and those without.
positional arguments: app_label App label of an application to synchronize the state. migration_name Database state will be brought to the state after that migration. Use the name "zero" to unapply all migrations.
optional arguments: -h, --help show this help message and exit --noinput, --no-input Tells Django to NOT prompt the user for input of any kind. --database DATABASE Nominates a database to synchronize. Defaults to the "default" database. --fake Mark migrations as run without actually running them. --fake-initial Detect if tables already exist and fake-apply initial migrations if so. Make sure that the current database schema matches your initial migration before using this flag. Django will only check for an existing table name. --plan Shows a list of the migration actions that will be performed. --run-syncdb Creates tables for apps without migrations. --check Exits with a non-zero status if unapplied migrations exist. --version show program’s version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn’t provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don’t colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.
Creating the Database from Migrations
(aw) $ ./manage.py migrate System check identified some issues:
WARNINGS: catalogue.ProductAttributeValue.value_boolean: (fields.W903) NullBooleanField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0. HINT: Use BooleanField(null=True) instead. Operations to perform: Apply all migrations: address, admin, analytics, auth, basket, catalogue, communication, contenttypes, customer, flatpages, offer, order, partner, payment, reviews, sessions, shipping, sites, thumbnail, voucher, wishlists Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying address.0001_initial... OK Applying address.0002_auto_20150927_1547... OK Applying address.0003_auto_20150927_1551... OK Applying address.0004_auto_20170226_1122... OK Applying address.0005_regenerate_user_address_hashes... OK Applying address.0006_auto_20181115_1953... OK Applying admin.0001_initial... OK Applying admin.0002_logentry_remove_auto_add... OK Applying admin.0003_logentry_add_action_flag_choices... OK Applying catalogue.0001_initial... OK Applying analytics.0001_initial... OK Applying analytics.0002_auto_20140827_1705... OK Applying analytics.0003_auto_20200801_0817... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying auth.0009_alter_user_last_name_max_length... OK Applying auth.0010_alter_group_name_max_length... OK Applying auth.0011_update_proxy_permissions... OK Applying auth.0012_alter_user_first_name_max_length... OK Applying sites.0001_initial... OK Applying partner.0001_initial... OK Applying customer.0001_initial... OK Applying basket.0001_initial... OK Applying basket.0002_auto_20140827_1705... OK Applying order.0001_initial... OK Applying offer.0001_initial... OK Applying voucher.0001_initial... OK Applying basket.0003_basket_vouchers... OK Applying basket.0004_auto_20141007_2032... OK Applying basket.0005_auto_20150604_1450... OK Applying basket.0006_auto_20160111_1108... OK Applying basket.0007_slugfield_noop... OK Applying basket.0008_auto_20181115_1953... OK Applying basket.0009_line_date_updated... OK Applying catalogue.0002_auto_20150217_1221... OK Applying catalogue.0003_data_migration_slugs... OK Applying catalogue.0004_auto_20150217_1710... OK Applying catalogue.0005_auto_20150604_1450... OK Applying catalogue.0006_auto_20150807_1725... OK Applying catalogue.0007_auto_20151207_1440... OK Applying catalogue.0008_auto_20160304_1652... OK Applying catalogue.0009_slugfield_noop... OK Applying catalogue.0010_auto_20170420_0439... OK Applying catalogue.0011_auto_20170422_1355... OK Applying catalogue.0012_auto_20170609_1902... OK Applying catalogue.0013_auto_20170821_1548... OK Applying catalogue.0014_auto_20181115_1953... OK Applying catalogue.0015_product_is_public... OK Applying catalogue.0016_auto_20190327_0757... OK Applying catalogue.0017_auto_20190816_0938... OK Applying catalogue.0018_auto_20191220_0920... OK Applying catalogue.0019_option_required... OK Applying catalogue.0020_auto_20200801_0817... OK Applying catalogue.0021_auto_20201005_0844... OK Applying order.0002_auto_20141007_2032... OK Applying order.0003_auto_20150113_1629... OK Applying order.0004_auto_20160111_1108... OK Applying order.0005_update_email_length... OK Applying order.0006_orderstatuschange... OK Applying order.0007_auto_20181115_1953... OK Applying customer.0002_auto_20150807_1725... OK Applying customer.0003_update_email_length... OK Applying customer.0004_email_save... OK Applying customer.0005_auto_20181115_1953... OK Applying communication.0001_initial... OK Applying order.0008_auto_20190301_1035... OK Applying communication.0002_reset_table_names... OK Applying communication.0003_remove_notification_category_make_code_uppercase... OK Applying communication.0004_auto_20200801_0817... OK Applying customer.0006_auto_20190430_1736... OK Applying customer.0007_auto_20200801_0817... OK Applying flatpages.0001_initial... OK Applying offer.0002_auto_20151210_1053... OK Applying offer.0003_auto_20161120_1707... OK Applying offer.0004_auto_20170415_1518... OK Applying offer.0005_auto_20170423_1217... OK Applying offer.0006_auto_20170504_0616... OK Applying offer.0007_conditionaloffer_exclusive... OK Applying offer.0008_auto_20181115_1953... OK Applying offer.0009_auto_20200801_0817... OK Applying offer.0010_conditionaloffer_combinations... OK Applying order.0009_surcharge... OK Applying order.0010_auto_20200724_0909... OK Applying order.0011_auto_20200801_0817... OK Applying partner.0002_auto_20141007_2032... OK Applying partner.0003_auto_20150604_1450... OK Applying partner.0004_auto_20160107_1755... OK Applying partner.0005_auto_20181115_1953... OK Applying partner.0006_auto_20200724_0909... OK Applying payment.0001_initial... OK Applying payment.0002_auto_20141007_2032... OK Applying payment.0003_auto_20160323_1520... OK Applying payment.0004_auto_20181115_1953... OK Applying payment.0005_auto_20200801_0817... OK Applying reviews.0001_initial... OK Applying reviews.0002_update_email_length... OK Applying reviews.0003_auto_20160802_1358... OK Applying reviews.0004_auto_20170429_0941... OK Applying sessions.0001_initial... OK Applying shipping.0001_initial... OK Applying shipping.0002_auto_20150604_1450... OK Applying shipping.0003_auto_20181115_1953... OK Applying sites.0002_alter_domain_unique... OK Applying thumbnail.0001_initial... OK Applying voucher.0002_auto_20170418_2132... OK Applying voucher.0003_auto_20171212_0411... OK Applying voucher.0004_auto_20180228_0940... OK Applying voucher.0005_auto_20180402_1425... OK Applying voucher.0006_auto_20180413_0911... OK Applying voucher.0007_auto_20181115_1953... OK Applying voucher.0008_auto_20200801_0817... OK Applying wishlists.0001_initial... OK Applying wishlists.0002_auto_20160111_1108... OK Applying wishlists.0003_auto_20181115_1953... OK
Since this step did not change any Frobshop files, there was nothing to commit to the git repository.
Mysterious Warning from manage.py
The following warning appeared every time manage.py ran:
WARNINGS: catalogue.ProductAttributeValue.value_boolean: (fields.W903) NullBooleanField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0. HINT: Use BooleanField(null=True) instead.
Thanks to StackOverflow, I was able to make this
django-oscar warning
disappear.
This is what the offending line looked like (line folded for legibility):
value_boolean = models.NullBooleanField(_('Boolean'),
blank=True, db_index=True)
This is what I understood the suggested change to be (line folded for legibility):
value_boolean = models.BooleanField(_('Boolean'),
blank=True, db_index=True)
Here is the incantation I used, which relies on an environment variable called
oscar to point to the Python virtual environment.
Line 1043 of abstract_models.py
was modified, within the definition of a class called AbstractProductAttributeValue.
(aw) $ sed -i 's/models.NullBooleanField/models.BooleanField/' \
~/venv/aw/lib/python3.8/site-packages/oscar/apps/catalogue/abstract_models.py
Startapp Subcommand
The startapp subcommand of manage.py message is:
(aw) $ ./manage.py startapp --help
usage: manage.py startapp [-h] [--template TEMPLATE] [--extension EXTENSIONS] [--name FILES] [--version]
[-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color]
[--force-color]
name [directory]
Creates a Django app directory structure for the given app name in the current directory or optionally in the given
directory.
positional arguments:
name Name of the application or project.
directory Optional destination directory
optional arguments:
-h, --help show this help message and exit
--template TEMPLATE The path or URL to load the template from.
--extension EXTENSIONS, -e EXTENSIONS
The file extension(s) to render (default: "py"). Separate multiple extensions with commas, or
use -e multiple times.
--name FILES, -n FILES
The file name(s) to render. Separate multiple file names with commas, or use -n multiple
times.
--version show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output
--settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided,
the DJANGO_SETTINGS_MODULE environment variable will be used.
--pythonpath PYTHONPATH
A directory to add to the Python path, e.g. "/home/djangoprojects/myproject".
--traceback Raise on CommandError exceptions
--no-color Don't colorize the command output.
--force-color Force colorization of the command output.
Adding a New Django App
I added a new Django app called pricing by using the startapp subcommand of manage.py:
(aw) $ ./manage.py startapp pricing
That created the following directories:
(aw) $ tree pricing pricing ├── __init__.py ├── admin.py ├── apps.py ├── migrations │ └── __init__.py ├── models.py ├── tests.py └── views.py
Nullable Field Problem
After adding the new Django app called pricing to frobshop,
I encountered this error:
(aw) $ ./manage.py makemigrations pricing You are trying to change the nullable field 'value_boolean' on productattributevalue to non-nullable without a default; we can’t do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one-off default now (will be set on all existing rows with a null value for this column) 2) Ignore for now, and let me handle existing rows with NULL myself (e.g. because you added a RunPython or RunSQL operation to handle NULL values in a previous data migration) 3) Quit, and let me add a default in models.py Select an option:
Update: now I would just provide a one-off default, or blow away with a delete SQL query,
since the database do not yet contain important information.
Instead, I continued to stumble forward:
I cancelled the makemigrations subcommand and changed the definition of the field to allow nulls,
so it looked like what was in the
django-oscar master branch
(line folded for legibility):
value_boolean = models.BooleanField(_('Boolean'), blank=True,
null=True, db_index=True)
The error went away the next time I ran makemigrations.
The sed command to make this change is:
(aw) $ sed -i \
-e 's/value_boolean = /models.NullBooleanField/models.BooleanField/' \
-e 's/value_boolean = /)/, null=True)/' \
~/venv/aw/lib/python3.8/site-packages/oscar/apps/catalogue/abstract_models.py
Creating the Django Superuser
The Frobshop documentation did not mention this topic. It is easy to do.
(aw) $ ./manage.py createsuperuser
Username (leave blank to use 'mslinn'):
Email address: mslinn@ancientwarmth.com
Password:
Password (again):
Superuser created successfully.
Since this step did not change any Frobshop files, there was nothing to commit to the git repository.
manage.py runserver Options
The help for manage.py runserver is:
(aw) $ ./manage.py runserver -h usage: manage.py runserver [-h] [--ipv6] [--nothreading] [--noreload] [--nostatic] [--insecure] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [addrport] Starts a lightweight Web server for development and also serves static files. positional arguments: addrport Optional port number, or ipaddr:port optional arguments: -h, --help show this help message and exit --ipv6, -6 Tells Django to use an IPv6 address. --nothreading Tells Django to NOT use threading. --noreload Tells Django to NOT use the auto-reloader. --nostatic Tells Django to NOT automatically serve static files at STATIC_URL. --insecure Allows serving static files even if DEBUG is False. --version show program's version number and exit -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions --no-color Don't colorize the command output. --force-color Force colorization of the command output.
Launching Oscar
Finally it was time to launch oscar on port 8000:
BTW, Django forks into 2 processes when it launches,
and each process loads the settings separately.
Use the --noreload option to force loading the settings only once.
(aw) $ ./manage.py runserver --noreload Watching for file changes with StatReloader February 11, 2021 - 15:45:01 Django version 3.1.6, using settings 'frobshop.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C.
Launch Failure
Frobshop failed to launch.
This was due to a serious django-oscar implementation deficiency:
-
django-oscarforms is hard-coded to requiredjango-haystack:Excerpt from oscar/apps/search/forms.htmlfrom haystack.forms import FacetedSearchForm -
While theOscar uses Haystack to abstract away from different search backends. Unfortunately, writing backend-agnostic code is nonetheless hard and Apache Solr is currently the only supported production-grade backend.
django-oscardocumentation describes various other fulltext search backends (ElasticSearch, Whoosh, and Xapian), apparently the official position is that they should not be used in production. ... or so the docs say. No-one seems to believe that statement. The documentation contains some strongly expressed views that are not widely held.
I wrote up an Suggestions for Django-Oscar and Django-Haystack, and posted to new GitHub discussion that I created for this topic.
Workaround
There is a django-haystack backend called
SimpleEngine
that uses ordinary SQL queries instead of fulltext search.
I installed and enabled django-haystack, and configured frobshop/settings.py to use SimpleEngine.
The next time I tried to run django-oscar, it presented itself in my browser at
http://localhost:8000.
I logged in as the superuser and saw:
Time once again to commit my work:
(aw) $ commit -m "Switched haystack.backends from SolrEngine to SimpleEngine."
Dashboard Navigation
Browsing throught the django-oscar, I stumbled across where the
menu structure of the dashboard navigation is defined, in JSON.
Order Pipeline
The Frobshop tutorial does not offer any information about how the order status of line items affects the status of orders. I found How to set up order processing, which addressed the issue. The tutorial should link to that page.
Reading “How to set up order processing”, it seems there is a
checkout app
and an order app.
A link to these from the article “How to set up order processing” would have been helpful.
This page
would be a good place to provide a discussion of how apps work together to enable django-oscar’s functionality,
but I could only find a generic explanation of Django apps in the Django documentation.
Unfortunately, “How to set up order processing” does not provide a sufficiently detailed explanation to understand the suggested settings in the Frobshop tutorial, which are:
OSCAR_INITIAL_ORDER_STATUS = 'Pending'
OSCAR_INITIAL_LINE_STATUS = 'Pending'
OSCAR_ORDER_STATUS_PIPELINE = {
'Pending': ('Being processed', 'Cancelled',),
'Being processed': ('Processed', 'Cancelled',),
'Cancelled': (),
}
Reading the Frobshop tutorial, I expected that if I copied the above settings into frobshop/settings.py that
“With these three settings defined in your project you’ll be able to see the different statuses in the order management dashboard.”
However, the information presented in “How to set up order processing” made me think that some custom programming would be required.
This Frobshop tutorial is horrible!
I pasted the above into frobshop/settings.py, and django-oscar started without errors or warnings.
I am unsure how to ‘order’ a product from this tutorial, however.
Time once again to commit my work:
(aw) $ commit -m "Defined order pipeline without understanding what I did."
More Clues
The django-oscar docs are sorely lacking for this topic.
I found this
buried in the source code
for django-oscar:
“The pipeline defines the statuses that an order or line item can have and what transitions are allowed in any given status.
The pipeline is defined as a dictionary where the keys are the available statuses.
Allowed transitions are defined as iterable values for the corresponding status.”
Additional settings in the same internal document include:
OSCAR_ORDER_STATUS_CASCADE
This defines a mapping of status changes for order lines which cascade from an order status change. For example:
OSCAR_ORDER_STATUS_CASCADE = {
'Being processed': 'In progress'
}
With this mapping, when an order has its status set to Being processed, all lines within it have their status set to In progress.
In a sense, the status change cascades down to the related objects.
Note that this cascade ignores restrictions from the OSCAR_LINE_STATUS_PIPELINE.
OSCAR_LINE_STATUS_PIPELINE
Same as OSCAR_ORDER_STATUS_PIPELINE but for lines.
Default: {}
Conclusion
Django-oscar seems well-designed, and seems to have all the features that I am looking for.
The community is active, responsive and helpful.
However, django-oscar, like many open-source products, suffers from poor documentation and has many problems.
Nothing is perfect.
However, django-oscar seems to be good enough for my needs.
