Mike Slinn
Mike Slinn

Publishing a Draft Article in a Jekyll Collection

Published 2021-12-20. Last modified 2023-12-02.
Time to read: 2 minutes.

This page is part of the jekyll collection.

I often want to have others review articles that I write before I make them visible to the public on my Jekyll-powered web site. There are many ways to do this:

  • Make a PDF (unfortunately, recipients do not see updates, unless a blizzard of emails ensues)
  • Set up a staging server (maintaining another server means more work)
  • Quietly publish to the production server without linking the article into the rest of the site (the subject of this article)

The remainder of this article shows the changes necessary to publish a draft page to a collection, without including it into the collection. The new page is not included in the site map, so the search engines are not notified. The new page is not included in any automatically generated content, so no links to it are inadvertently created. New links from the new page work. Reviewers can see the page if they know the URL.

Set the Flag

Add this to the front matter of any page in a collection that should not be publicly visible:

Front matter in Jekyll page
published: false

Use the Jekyll_draft Plugin

The jekyll_draft plugin makes working with drafts much easier, and adds some useful features.

Bash Scripts

This is the Bash script I wrote to make auto-reloading work in progress possible. My web site is served from AWS S3 buckets, and the AWS CLI is used to manage AWS services.

stage 
#!/bin/bash
#
# Author: Mike Slinn
#
# Before running this script:
# 1) Add "published: false" to front matter for new draft page
# 2) Run _bin/prod to upload images and other dependencies
#
# SPDX-License-Identifier: Apache-2.0

GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}"
git add .
git commit -m -
source _bin/loadConfigEnvVars
_bin/generate development

cd _site
FILES=$( grep -rl draftPost . | grep '[.]html$' | sed "s|^\./||" | grep -v publishing-drafts )
unset FILES2
for FILE in $FILES; do
  FILES2="$FILES2 /$FILE"
  echo "Uploading https://$DOMAIN/$FILE"
  aws s3 cp \
	--acl public-read \
    --quiet \
	"$FILE" "s3://$DOMAIN/$FILE"
done

# Only invalidate if -I not provided
if [ "$1" != -I ]; then
  aws cloudfront create-invalidation \
      --distribution-id "$AWS_CLOUDFRONT_DIST_ID" \
      --paths $FILES2
fi

No changes were required to my script for defining environment variables from the Jekyll YAML configuration file:

loadConfigEnvVars 
#!/bin/bash
#
# Author: Mike Slinn
#
# Defines the following configuration environment variables when sourced by a bash script:
# AWS_ACCOUNT_ID, AWS_CLOUDFRONT_DIST_ID, AWS_REGION, DOMAIN, LAMBDA_ARN, LAMBDA_IAM_ROLE_ARN, LAMBDA_IAM_ROLE_NAME, LAMBDA_HANDLER, LAMBDA_NAME, LAMBDA_RUNTIME, LAMBDA_ZIP, LAMBDA_PACKAGE_DIR, TITLE and URL
#
# SPDX-License-Identifier: Apache-2.0

function assertEnvVarSet {
  if [ ! -v "$1" ]; then
    echo "Error: no environment variable called $1 is defined"
    return 1
  fi
  return 0
}

function installYq {
  echo "Installing yq"
  # See https://mikefarah.gitbook.io/yq/
  sudo su
  VERSION=v4.2.0
  BINARY=yq_linux_amd64
  wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
    tar xz && mv ${BINARY} /usr/bin/yq
  exit
}

function lookupEnvVar {
  assertEnvVarSet "$(readYaml $1)" || { return 1; }
  eval echo -e "\$$(readYaml $1)"
}

function readYaml {
  # $1 - path
  yq eval ".$1" _config.yml
}

function writeYaml {
  # $1 - path
  # $2 - value
  yq eval ".$1 = $2" -i _config.yml
}

GIT_ROOT="$( git rev-parse --show-toplevel )"

if [ -z "$( which yq )" ] || [[ "$( yq -V )" != *4.* ]]; then
  VERSION=v4.2.0
  BINARY=yq_linux_amd64
  echo "Installing yq"
  # See https://mikefarah.gitbook.io/yq/
  sudo -iH bash <<EOF
  wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
    tar xz && mv ${BINARY} /usr/bin/yq
EOF
fi

# HOST
export PAAS_HOST="$(               readYaml host                         )"

# Linode modifiable values
export LINODE_BUCKET="$(           readYaml linode.bucket                )"
export LINODE_REGION="$(           readYaml linode.region                )"

# Linode computed values
export LINODE_HOST_BUCKET="%(bucket)s.#{LINODE_REGION}.linodeobjects.com"
export LINODE_WEBSITE_ENDPOINT="http://%(bucket)s.website-#{LINODE_REGION}.linodeobjects.com/"

# AWS modifiable values
export GATEWAY_STAGE="$(          readYaml aws.apiGateway.stage          )"

# AWS computed values
export AWS_ACCOUNT_ID="$(         readYaml aws.accountId                 )"
export AWS_CLOUDFRONT_DIST_ID="$( readYaml aws.cloudfront.distributionId )"
export AWS_REGION="$(             readYaml aws.region                    )"
export GATEWAY_ENDPOINT="$(       readYaml aws.apiGateway.endpoint       )"
export GATEWAY_REST_API_ID="$(    readYaml aws.apiGateway.restId         )"
export GATEWAY_RESOURCE_ID="$(    readYaml aws.apiGateway.resourceId )"
export GATEWAY_ROOT_ID="$(        readYaml aws.apiGateway.rootResourceId )"
export GATEWAY_NAME="$(           readYaml aws.apiGateway.name           )"

# Computed AWS Lambda values
export LAMBDA_ARN="$(           readYaml aws.lambda.addSubscriber.computed.arn        )"
export LAMBDA_IAM_ROLE_ARN="$(  readYaml aws.lambda.addSubscriber.computed.iamRoleArn )"

# Modifiable AWS Lambda values
export LAMBDA_HANDLER="$(       readYaml aws.lambda.addSubscriber.custom.handler      )"
export LAMBDA_IAM_ROLE_NAME="$( readYaml aws.lambda.addSubscriber.custom.iamRoleName  )"
export LAMBDA_NAME="$(          readYaml aws.lambda.addSubscriber.custom.name         )"
export LAMBDA_RUNTIME="$(       readYaml aws.lambda.addSubscriber.custom.runtime      )"

export LAMBDA_PACKAGE_DIR="${GIT_ROOT}/_package"
export LAMBDA_ZIP="${LAMBDA_PACKAGE_DIR}/function.zip"

# Misc modifiable values
export TITLE="$( readYaml title )"
export URL="$( readYaml url )"
export DOMAIN="$( echo "$URL" | sed -n -e 's,^https\?://,,p' )"

No changes were required to my script for publishing to production:

prod 
#!/bin/bash

# Push new Jekyll web site to AWS S3 bucket
#
# Author: Mike Slinn
# SPDX-License-Identifier: Apache-2.0

RED='\033[0;31m'
RESET='\033[0m' # No Color

function brokenLinkCheck {
  # See https://github.com/stevenvachon/broken-link-checker
  BROKEN="$( _bin/checkLinks | grep BROKEN )"
  if [ "$BROKEN" ]; then
    printf "\n${RED}Broken links found, aborting:$RESET"
    echo "$BROKEN"
    printf "\n${RED}=======================$RESET"
    exit 1
  fi
}

function checkDependencies {
  if [ -z "$( which npm )" ]; then
    # See https://github.com/nvm-sh/nvm
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
    [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion
  fi

  if [ -z "$( which node )" ]; then
    nvm install node
  fi

  if [ -z "$( which blc )" ]; then
    yes | npm install broken-link-checker -g
  fi

  if [ -z "$( which s3cmd )" ]; then yes | sudo apt install s3cmd; fi
  if [ -z "$( which aws )"   ]; then yes | sudo snap install aws-cli --classic; fi
  if [ -z "$( which jq )"    ]; then yes | sudo apt install jq; fi
}

function crawlSiteMap {
  # Ask Google and Bing to crawl the sitemap
  SITEMAP="$DOMAIN/sitemap.xml"

  # See https://developers.google.com/search/docs/guides/submit-URLs
  echo "Notifying Google of new sitemap.xml"
  wget -qO - http://www.google.com/ping?sitemap=https://$SITEMAP > /dev/null

  # See https://www.bing.com/webmaster/help/how-to-submit-sitemaps-82a15bd4
  echo "Notifying Bing of new sitemap.xml"
  wget -qO - http://www.bing.com/ping?sitemap=http%3A%2F%2F$SITEMAP > /dev/null
}


function invalidate {
  if [ "$AWS_CLOUDFRONT_DIST_ID" ]; then
    JSON="$( aws cloudfront create-invalidation \
      --distribution-id "$AWS_CLOUDFRONT_DIST_ID" \
      --paths "/*"
    )"
    INVALIDATION_ID="$( jq -r .Invalidation.Id <<< "$JSON" )"
    waitForInvalidation "$AWS_CLOUDFRONT_DIST_ID" "$INVALIDATION_ID" &
  fi
}

function maybeCreateS3Bucket {
  s3cmd ls "s3://$DOMAIN" 2>/dev/null >/dev/null
  if [[ $? -ne 0 ]]; then # create bucket
    _bin/makeAwsBucket "$DOMAIN"
  fi
}

function publishSite {
  set -b
  #set -e

  # Set current directory to project root
  GIT_ROOT="$( git rev-parse --show-toplevel )"
  cd "${GIT_ROOT}" || exit
  source _bin/loadConfigEnvVars

  checkDependencies
  git pull 3>&1 1>&2 2>&3 | sed -e '/^X11/d'  | sed -e '/^Warning:/d'
  #_bin/setTimes
  _bin/generate production
  if [ $? -ne 0 ]; then exit 1; fi

  brokenLinkCheck
  commit
  # maybeCreateS3Bucket
  sync
  _bin/redirects # No longer invalidates by default
  # _bin/stage -I
  invalidate
  if [ "$DOMAIN" != www.testJekyllTemplate.com ]; then crawlSiteMap; fi
  date '+Published %A, %B %d, %Y at %H:%M:%S.'
}

function sync {
  echo "Starting sync"
  # Get rid of any extra files in _site/
  find _site/ \( -name '*.Identifier' -o -name '.sprockets-manifest*' \) -delete

  # aws cli does not know the Content-Type for webp files so handle that filetype separately.
  # Do not sync mp4s because they are so large.
  aws s3 sync \
    --acl public-read \
    --exclude '*.ai' \
    --quiet \
    _site/ "s3://$DOMAIN"
  # Do not sync mp4s because they are so large.
  aws s3 sync \
    --delete \
    --exclude '*.mp4' \
    --quiet \
    _site/ "s3://$DOMAIN"
  echo "Ending sync"
}

function waitForInvalidation {
  if [ "$host" == aws ]; then
    echo "Waiting for invalidation $2 to complete."
    aws cloudfront wait invalidation-completed \
      --distribution-id "$1" \
      --id "$2"
    echo "Invalidation $2 has completed."
  fi
}

export host=aws
publishSite

No changes were required to my script for serving locally:

serve 
#!/bin/bash

# SPDX-License-Identifier: Apache-2.0

export PORT_LSB=1
unset FUTURE_POSTS

export FUTURE_POSTS="--future"
export HOST=0.0.0.0
export INCREMENTAL="--incremental"
export LIVE="--livereload"
export LIVE_PORT="--livereload_port 3572$PORT_LSB"
export PORT="400$PORT_LSB"

export DRAFTS="--drafts"
export UNPUBLISHED="--unpublished"
export QUIET="--quiet"
unset VERBOSE
export PORT="400$PORT_LSB"

export OPTIONS="$LIVE_PORT"

function isWindows {
  if [ "$( grep -i Microsoft /proc/version )" ]; then echo yes; fi
}

# Force polling if the script is running on a Windows drive
if [ "$( isWindows )" ]; then
  export OPTIONS="$OPTIONS --force_polling"
fi


function help {
   echo "Runs Jekyll. The default domain:port is localhost:$PORT.
  Options are:
    -F         Disable future posts
    -H domain  Run from given host, defaults to localhost
    -I         Disable incremental compilation
    -L         Disable live reload (use if cannot bind to server/port)
    -c         Clear cache and rebuild site
    -d         Debug mode
    -D         Disable drafts and unpublished
    -g 10      Generate only the last 10 posts
    -p 1234    Run from given port, defaults to $PORT
    -P         Generate a production site
    -q         Set Jekyll logging level to :error (default)
    -v         Set Jekyll logging level to :debug

  For example, to run on http://localhost:$PORT, type:
  _bin/serve
  _bin/serve -c  # Clean out previous build

  When quiet and verbose are both specified, log-level is set to :error
"
   exit 1
}

function installDependencies {
  if [ -z "$( find /usr/lib/x86_64-linux-gnu -name 'libmagic.so*' )" ]; then
    echo "Installing libmagic-dev"
    yes | sudo apt install libmagic-dev
  fi

  if [ -z "$( which ruby )" ]; then
    yes | sudo apt install ruby-full
  fi

  if [ -z "$( which bundle )" ]; then
	  gem install bundler $QUIET
  fi

  bundle install $QUIET
}

function isSiteUp {
  curl -sSf http://$1 > /dev/null 2> /dev/null
}

function makePluginDocs {
  rm -rf jekyll/docs/*
  cd _plugins/ > /dev/null || exit
  if [ "$QUIET" ]; then YARD_QUIET="--no-stats --no-progress"; fi
  yard doc $QUIET $YARD_QUIET \
    jekyll_command_template.rb jekyll_filter_template.rb jekyll_generator_template.rb \
    rawinclude.rb string_overflow.rb symlink_watcher.rb > /dev/null 2> /dev/null
  cd - > /dev/null || exit
}


# Restore default Python environment
if [ "$(declare -f deactivate)" ]; then
  if [ -z "$QUIET" ]; then echo "Deactivating Python virtualenv"; fi
  deactivate
fi

# Set cwd to project root
GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}" || exit

find . -iname '*Zone.Identifier' -delete

source _bin/loadConfigEnvVars

export MANWIDTH=70
export JEKYLL_ENV=development
while getopts "cdDhiFg:hH:ILp:Pqv\?" opt; do
   case $opt in
     D ) unset DRAFTS
         unset UNPUBLISHED
         ;;
     F ) unset FUTURE_POSTS ;;
     H ) export HOST="$OPTARG" ;;
     I ) unset INCREMENTAL ;;
     L ) unset LIVE ;;
     c ) if [ -z "$QUIET" ]; then echo "Removing temporary files from _site/"; fi
         #rm -rf _site/*
         bundle exec jekyll clean $QUIET
         ;;
     d ) set -xv ;;
     g ) export OPTIONS="$OPTIONS --limit_posts=$OPTARG" ;; # generate only the last 10 posts
     p ) export PORT="$OPTARG" ;;
     P ) export JEKYLL_ENV=production ;;
     q ) export QUIET="--quiet" ;;
     v ) export VERBOSE="--verbose" ;;
     * ) help ;;
   esac
done
shift "$((OPTIND-1))"

if [ "$HOST" ]; then OPTIONS="$OPTIONS --host $HOST"; fi   # Listen at the given hostname.
if [ "$PORT" ]; then OPTIONS="$OPTIONS --port $PORT"; fi   # Listen at the given port.
OPTIONS="$OPTIONS $FUTURE_POSTS"
OPTIONS="$OPTIONS $INCREMENTAL"
OPTIONS="$OPTIONS $LIVE"
OPTIONS="$OPTIONS $DRAFTS"
OPTIONS="$OPTIONS $UNPUBLISHED"
OPTIONS="$OPTIONS $QUIET"
OPTIONS="$OPTIONS $VERBOSE"

if [ -z "$QUIET" ]; then echo "OPTIONS=$OPTIONS"; fi

source use default > /dev/null
installDependencies
# _bin/make_public_plugin_docs

bundle exec jekyll serve $OPTIONS 2>&1 | \
  grep -Ev 'Using the last argument as keyword parameters is deprecated'
Front page
Expert witness USA
Studio technology USA
17 articles on LLMs
General topics
Ruby language USA
Jekyll-generated websites USA
Git and libgit2 USA
Django-powered e-commerce USA
Mainframe image; Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License by PekoeBlaze
Original music by Mike Slinn
* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.