Published 2020-08-15. Last modified 2022-04-04.
Time to read: 2 minutes.

This page is part of the posts collection, categorized under Git, Jekyll, Scripting, Ubuntu.

I first launched this website in 1996. Since then, it has been re-incarnated using many different technologies. Presently I use Jekyll to assemble the site, then push the image to a web-enabled AWS S3 bucket that is edge-cached by an AWS CloudFront distribution.

Until yesterday, the site contained images with a mixture of image formats. I decided to convert them all to the new webp format. Because there are hundreds of images in over 120 web pages, I wrote a bash script called toWebP to do the work. This posting provides the toWebP script plus instructions on how you could use it for your website.

The script converts image types gif, jpg, jpeg, png, tif, and tiff. It also modifies the HTML pages, CSS and SCSS that reference those images.

The conversions are set for maximum fidelity (lossless where possible), and maximum compression. This means the images look great and load quickly.

I also wrote toPng, which works in a similar manner as toWebp.

Caveat

The script assumes that all images are local to your website, which makes sense because the converted images need to be stored, and local storage is the only sensible option. It renames all references to images in HTML, CSS and SCSS files to webp format. If the images are remote (for example, on a CDN), they are not converted, but the image file types in the HTML, CSS and SCSS are adjusted anyway. I suppose I could fix the script, but I don't need to do that for myself. If someone needs that feature, go ahead and enhance the script... and please provide me the enhanced script, so I can update this articleing.

Prerequisites

You need to install the WebP package.

Mac

Use Homebrew or Macports.

Ubuntu (this is the default Linux distribution for Windows Subsystem for Linux)

At a shell prompt type:

Shell

$ yes | sudo apt install webp

Running toWebp

The program may emit warnings when it runs. Those warnings can be safely ignored.

Hopefully, your website is managed by git. I suggest that you commit your work before running the script. That way if something goes wrong you just have to type git stash to return your website to its previous state.

Usage

The general form of the command to convert all images and modify the HTML pages that they are referenced from is:

Shell

$ toWebp <directoryName>

Examples

To convert the website (images, html, scss & css) rooted at the current directory, type:

Shell

$ toWebp .

To convert the website called mySite rooted under your home directory, type:

Shell

$ toWebp ~/mySite

To just convert 1 specific image to webp, type:

Shell

$ toWebp images/blah.jpg

The toWebP Bash Script.

Put this file in one of the directories on your PATH, for example /usr/local/bin, or for Ubuntu users ~/.local/bin/:

toWebp

#!/bin/bash

# SPDX-License-Identifier: Apache-2.0

shopt -s extglob

export CMD="cwebp alpha_q 10 -exact -lossless -m 6 -short -q 100 -z 9"

function help {
  [[ "$1" ]] && >&2 printf "Error: $1\n\n";
  >&2 printf "$( basename $0 ) - Convert all images to webp files in place and update HTML, CSS and SCSS to suit

Usage: toWebp directory|imageFileName

Example: toWebp .  # convert images, html, scss & css in current directory tree

Example: toWebp images/blah.jpg   # just convert 1 specific image
"
}

function checkDependencies {
  if [ -z "$( which cwebp )" ]; then
    echo "Installing webp"
    yes | sudo apt install webp
  fi
}

function convertGifs {
  for F in $( find "$1" -iname '*.gif' ); do
    TOF="${F%.*}.webp"
    gif2webp -loop_compatibility -m 6 -mixed "$F" -q 100 -o "$TOF"
  done
}

function convertMost {
  # See "Extended pattern" extglob for Bash
  # https://wiki.bash-hackers.org/syntax/pattern#extended_pattern_language
  cd "$1" || exit 3
  for F in $( listImages ); do
    TOF="${F%.*}.webp"
    echo "Converting '$F' to '$TOF'"
    # Warning messages might be emitted, but don't worry
    $CMD "$F" -o "$TOF"
    if [ "$DELETE_OLD" ]; then rm "$F"; fi
  done
  cd - || exit
}

function listImages {
  # Does not return gifs because cwebp cannot handle them
  find . -iregex '.*\.\(jpg\|png\|jpeg\|tif\|tiff\)' -printf '%f\n'
}

function renameImage {
  sed -i "s,\b$1\b,.webp,g" "$2"
}

function swapImages {
  for F in $( find "$1" -iname '*.html' -o -iname '*.css' -o -iname '*.scss' ); do
    for X in .gif .jpg .jpeg .tif .tiff .png; do
      echo "Swapping $X images for .webp in '$F'"
      renameImage "$X" "$F"
    done
  done
}


unset DELETE_OLD   # TODO make this a command line option

if [[ -f "$1" ]]; then   # just convert a single image to webp
  F="$1"
  $CMD "$F" -o "${F%.*}.webp"
  if [ "$DELETE_OLD" ]; then rm "$F"; fi
elif [[ -d "$1" ]]; then # process all images, css and scss in directory
  shopt -s globstar
  convertMost "$1"
  convertGifs "$1"
  swapImages "$1"
else
  help "You must either specify a valid file or a directory"
fi

Make it Executable

Remember to make the toWebp script executable before trying to use it:

Shell

$ chmod a+x /usr/local/bin/toWebp

Help Message

Shell

$ toWebp
Error: You must either specify a valid file or a directory

toWebp - Convert all images to webp files in place and update HTML, CSS and SCSS to suit

Usage: toWebp directory|imageFileName

Example: toWebp .  # convert images, html, scss & css in current directory tree

Example: toWebp images/blah.jpg   # just convert 1 specific image 

The toPng Bash Script.

toPng

#!/bin/bash

# Convert all webp images to png in place; does not update HTML, CSS or SCSS
# Usage: toPng directory|imageFileName
# Example: toPng .  # convert images in current directory tree
# Example: toPng images/blah.webp   # just convert 1 specific image
#
# SPDX-License-Identifier: Apache-2.0

shopt -s extglob

export CMD="dwebp"

function help {
  [[ "$1" ]] && >&2 printf "Error: $1\n\n";
  >&2 printf "$( basename $0 ) - Convert all images to webp files in place and update HTML, CSS and SCSS to suit

Usage: $( basename $0 ) directory|imageFileName

Example: $( basename $0 ) .  # convert images, html, scss & css in current directory tree

Example: $( basename $0 ) images/blah.jpg   # just convert 1 specific image
"
}

function checkDependencies {
  if [ -z "$(which dwebp)" ]; then
    echo "Installing webp"
    yes | sudo apt install webp
  fi
}

function listImages {
  find . -type f -iname '*.webp'
}

function convert {
  # See "Extended pattern" extglob for Bash
  # https://wiki.bash-hackers.org/syntax/pattern#extended_pattern_language
  cd "$1" || exit
  for F in $( listImages ); do
    TOF="${F%.*}.png"
    echo "Converting '$F' to '$TOF'"
    # Warning messages might be emitted, but don't worry
    $CMD "$F" -o "$TOF"
    if [ "$DELETE_OLD" ]; then rm "$F"; fi
  done
  cd - || exit
}


# Set cwd to project root
GIT_ROOT="$( git rev-parse --show-toplevel )"
cd "${GIT_ROOT}" || exit
[[ -f _bin/loadConfigEnvVars ]] && source _bin/loadConfigEnvVars;

unset DELETE_OLD   # TODO make this a command line option

checkDependencies

if [ -f "$1" ]; then   # just convert a single image to png
  F="$1"
  $CMD "$F" -o "${F%.*}.png"
  if [ "$DELETE_OLD" ]; then rm "$F"; fi
elif [ -d "$1" ]; then # process all images, css and scss in directory
  convert "$1"
else
  help "You must either specify a valid file or a directory."
fi

Make it Executable

Remember to make the toWebp script executable before trying to use it:

Shell

$ chmod a+x /usr/local/bin/toPng

Help Message

Shell

$ toPng
Error: You must either specify a valid file or a directory.

toPng - Convert all images to webp files in place and update HTML,
CSS and SCSS to suit.

Usage: toWebp directory|imageFileName

Example: toWebp .  # convert images, html, scss & css in current directory tree

Example: toWebp images/blah.jpg   # just convert 1 specific image 

Share to Twitter, Hacker News, LinkedIn.