Mike Slinn

Production Infrastructure Scripts

Published 2020-03-01. Last modified 2024-11-26.
Time to read: 2 minutes.

This page is part of the av_studio collection.
The Production Infrastructure Overview and Production Infrastructure Directories articles describe the environment that the scripts on this page operate within.

Create Song Structure

The following script creates the subdirectory structure for a song.

Example Usage

This script can be called from any directory.

Shell
$ song_create $songs/test
Creating a new song subdirectory structure in '/mnt/e/media/songs/test'
  /mnt/e/media/songs/test/daw
  /mnt/e/media/songs/test/dvrArchives
  /mnt/e/media/songs/test/inputs/images
  /mnt/e/media/songs/test/inputs/videos
  /mnt/e/media/songs/test/outputs/mp3s
  /mnt/e/media/songs/test/outputs/stems
  /mnt/e/media/songs/test/outputs/youTube
Opening E:\media\songs\test 

Source Code

#!/usr/bin/env ruby

require 'colorator'
require 'fileutils'

# Creates a new music video (song) subdirectory structure.
# Does not create any files, so this script is idempotent;
# you can accidently call it several times without problems.
#
# Creates this structure:
# ├── daw
# │   └── beats  # Experiments
# ├── dvrArchives  # DaVinci Resolve project archives
# ├── inputs  # For video editor (DaVinci Resolve) project
# │   ├── images  # QR code, etc.
# │   └── videos  # A and B rolls
# └── outputs  # Renders, mostly by DAWs and video editor
#     ├── mp3s  # Pro Tools renders
#     ├── stems  # Zipped audio files
#     └── youTube  # DaVinci Resolve renders
#
# Sample output:
# $ song_create $songs/test
# The subdirectory structure '$songs/test' already exists. No problem!
#   /mnt/e/media/songs/test/daw
#   /mnt/e/media/songs/test/dvrArchives
#   /mnt/e/media/songs/test/inputs/images
#   /mnt/e/media/songs/test/inputs/videos
#   /mnt/e/media/songs/test/outputs/mp3s
#   /mnt/e/media/songs/test/outputs/stems
#   /mnt/e/media/songs/test/outputs/youTube
# Opening E:\media\songs\test
#
class SongStructure
  def initialize(song_name)
    unless song_name
      puts "Error: No song name was provided. Aborting.".red
      exit 1
    end
    @song_name = song_name

    @songs_dir = ENV.fetch('songs') # root of all songs
    unless @songs_dir
      puts "Error: Environment variable 'songs' is not defined. Aborting.".red
      exit 2
    end
    if @songs_dir.empty?
      puts "Error: Environment variable 'songs' has no value. Aborting.".red
      exit 3
    end

    unless Dir.exist? @songs_dir
      puts "Error: Directory #{@songs_dir} does not exist. Aborting.".red
      exit 4
    end

    @song_dir = "#{@songs_dir}/#{@song_name}" # root of for the new song
  end

  def create
    if File.exist? @song_dir
      puts "The subdirectory structure '#{@song_dir}' already exists. No problem!".yellow
    else
      puts "Creating a new song subdirectory structure in '#{@song_dir}'".yellow
    end

    [
      'daw',
      'dvrArchives',
      'inputs/images', 'inputs/videos',
      'outputs/mp3s', 'outputs/stems', 'outputs/youTube',
    ].each do |dir|
      puts "  #{@song_dir}/#{dir}".yellow
      FileUtils.mkdir_p "#{@song_dir}/#{dir}"
    end
  end

  def display_dir
    win_song_dir = `wslpath -w '#{@song_dir}'`.chomp
    puts "Opening #{win_song_dir}".yellow
    `cmd.exe /c 'explorer.exe /separate, #{win_song_dir}'`
  end
end

if __FILE__ == $PROGRAM_NAME
  song_structure = SongStructure.new(ARGV[0])
  song_structure.create
  song_structure.display_dir
end

Create Song Webpage

The following script creates a new web page on this website for a new song. Referring to the production overview image, it creates a new web page with the given name in the right-hand area pertaining to Jekyll.

A QR code is also created for the webpage URL, which begins with https://mslinn.com/songs/.

Example Usage

This script can be called from any directory. It overwrites any previous .html file of the same name in the collections/_songs/ subdirectory of the website. Users should commit their changes before calling this script, just to be sure.

Shell
$ song_page_create my_song
Wrote /var/sitesUbuntu/www.mslinn.com/collections/_songs/my_song.html 

Source Code

#!/usr/bin/env ruby

require 'colorator'
require 'fileutils'

# Creates $msp/collections/_songs/#{song_name}.html from song_template.html in the same directory.
# Overwrites the file if present.
# Creates a QR code image.
#
# TODO: update template contents with qr code and other strings. Right now manual editing is necessary
#
# Sample output:
# $ song_page_create space_truckin
#   Creating /var/sitesUbuntu/www.mslinn.com/space_truckin.html
#   Opening the new web page in VSCode
#
class SongPageCreate
  def initialize(song_name)
    unless song_name
      puts "Error: No song name was provided. Aborting.".red
      exit 1
    end
    @song_name = song_name

    @msp = ENV.fetch('msp') # root of mslinn.com (website)
    unless @msp
      puts "Error: Environment variable 'msp' is not defined. Aborting.".red
      exit 2
    end
    if @msp.empty?
      puts "Error: Environment variable 'songs' has no value. Aborting.".red
      exit 3
    end

    unless Dir.exist? @msp
      puts "Error: Directory #{@msp} does not exist. Aborting.".red
      exit 4
    end

    @webpage_dir = "#{@msp}/collections/_songs" # path of the directory containing the web page for the new song
    @template_path = "#{@webpage_dir}/song_template.html"
    unless File.exist? @template_path
      puts "Error: '#{@template_path}' does not exist. Aborting."
      exit 1
    end

    @webpage_name = "#{@webpage_dir}/#{@song_name}.html" # path of the web page for the new song
    @qr_image = "#{@webpage_dir}/images/qr_#{@song_name}.png"
    @webpage_url = "https://www.mslinn.com/songs/#{@song_name}.html"
  end

  def create
    if File.exist? @webpage_name
      puts "The webpage '#{@webpage_name}' already exists. IT WILL BE NUKED AND REPLACED NOW.".yellow
    else
      puts "Creating a new song webpage in '#{@webpage_name}' from 'template.html' in the same directory.".yellow
    end
    FileUtils.cp @template_path, @webpage_name
  end

  def display_dir
    webpage_name_win = `wslpath -w '#{@webpage_name}'`.chomp
    puts "Opening the directory containing #{webpage_name_win}".yellow
    `cmd.exe /c 'explorer.exe /separate, #{webpage_name_win}'`
  end

  # Overwrites
  def qr_code
    # yes | sudo apt install qrencode
    `qrencode -o #{@qr_image} '#{@webpage_url}'`
  end
end

if __FILE__ == $PROGRAM_NAME
  spc = SongPageCreate.new(ARGV[0])
  spc.create
  spc.display_dir
  spc.qr_code
end

Song Template

View the rendered song template for new songs, and examine its source code:

---
bpm: 120
history: 2024-11-28
date: 2024-11-28
key: C
description: "___, an original composition by Mike Slinn. All rights reserved."
guitar_pro_file:
headerImage:
last_modified_at: 2024-11-28
layout: songs
live_file:
mp3:
musescore:
musicxml:
pdf:
png:
pro_tools:
published: false
qr_code: /songs/images/qr_song_template.png
stems:
title: Song Template
---
<!-- #region about -->
{% capture about %}
<p>
  This is the song template.
</p>
{% endcapture %}
<!-- endregion -->


<!-- #region audio -->
{%- comment -%}
  {% capture audio %}
    <p id="mp3_long">
    </p>
    {% include audio.html mp3="/songs/mp3s/space_truckin.mp3" %}
  {% endcapture %}
{%- endcomment -%}
<!-- endregion -->


<!-- #region music_videos -->
{% comment %}
  {% capture music_videos %}
    {% include video.html youtube_id='6wMf1AHCJXE' %}
  {% endcapture %}
{% endcomment %}
<!-- endregion -->


{% include song.html
  about=about
  audio=audio
  change_log=change_log
  competition=competition
  how_to_play=how_to_play
  learning_videos=learning_videos
  lyrics=lyrics
  music_videos=music_videos
  planned=planned
  tools_used=tools_used
%}

Zip Stems

The following script packages audio stems into a zip file. It performs the functionality denoted by the dashed line with the label zip in the production overview image.

Pro Tools generates audio stems with a common prefix. The zip file created by this script is automatically named from the common prefix of all the audio stem files.

Example Usage

This script can be called from any directory.

Shell
$ song_stems $songs/my_song
Wrote /mnt/e/media/songs/my_song/outputs/stems/my_song_stems_120_bpm.zip 

The following is identical to the above:

Shell
$ song_stems 'E:\media\songs\one_year_older\outputs\stems'
Wrote /mnt/e/media/songs/my_song/outputs/stems/my_song_stems_120_bpm.zip 

Source Code

#!/usr/bin/env ruby

require 'colorator'
require 'find'
require 'zip'

# gem install rubyzip
# gem 'rubyzip'

# Zips audio files in songs\your_song\outputs\stems into
# $msp/collections/_songs/stems/ and deletes the original files
#
# If this program is run from either of these song directories:
#   /mnt/e/media/songs/one_year_older/
#   /mnt/e/media/songs/one_year_older/outputs/stems/
# then the defaults will overwrite any zip file in
# the corresponding www.mslinn.com directory
#   U:\var\sitesUbuntu\www.mslinn.com\collections\_songs\stems
#
# In other words, this invocation:
#   $ cd $media/songs/one_year_older
#   $ song_stems
# are the same as:
#   $ song_stems E:/media/songs/one_year_older
#   $ song_stems E:/media/songs/one_year_older U:/var/sitesUbuntu/www.mslinn.com/collections/_songs/stems
#   $ song_stems $media/songs/one_year_older
#   $ song_stems $media/songs/one_year_older $msp/collections/_songs/stems
#
# @param src [String] Can be either a Windows path or a WSL path
# @param dest [String] Defaults to common stems directory for all songs in mslinn.com
# Will be overwritten if it already exists.
class SongStems
  def initialize(src = nil, dest = nil)
    @src = src.nil? ? Dir.pwd : expand_env(src)
    unless File.exist? @src # Did the user provide a WSL path?
      # TODO: wslpath fails if @src does not exist; need to handle this
      @src = `wslpath '#{@src}'`.chomp # Try again with a Windows path
      unless File.exist? @src
        puts "Error: directory '#{@src}' does not exist.".red
        exit 1
      end
    end
    @src += '/outputs/stems' if File.exist?("#{@src}/outputs/stems")

    @dest = if dest.nil? || dest.empty?
              "/var/sitesUbuntu/www.mslinn.com/collections/_songs/stems"
            else
              dest = expand_env dest
              dest.exist? ? dest : `wslpath '#{dest}'`.chomp
            end
    unless File.exist? @dest
      puts "Error: directory #{@dest} does not exist.".red
      exit 2
    end

    @input_filenames =
      Dir
        .glob("#{@src}/*")
        .select { |f| File.file?(f) && !f.end_with?('.zip') }
        .map { |f| File.basename f }
    if @input_filenames.empty?
      puts "There are no audio stems in #{@src}, so this program has nothing to do.".red
      exit
    end

    zipname = common_prefix(@input_filenames)
                .gsub('_stem', '_stems')
                .tr(' ', '_')
                .delete_suffix('_')
    @zip_path = "#{@dest}/#{zipname}.zip"
  end

  def expand_env(str)
    str.gsub(/\$([a-zA-Z_][a-zA-Z0-9_]*)|\${\g<1>}|%\g<1>%/) do
      ENV.fetch(Regexp.last_match(1), nil)
    end
  end

  def make
    FileUtils.rm_f(@zip_path)
    Zip::File.open(@zip_path, create: true) do |zipfile|
      ::Zip.sort_entries = true
      @input_filenames.each do |filename|
        zipfile.add(filename, File.join(@src, filename))
      end
    end
    puts "Wrote #{@zip_path}".green
  end

  def delete_originals
    @input_filenames.each { |f| File.delete "#{@src}/#{f}" }
  end

  def common_prefix(strings)
    return '' if strings.empty?

    result = 0
    (0...strings.first.length).each do |k|
      all_matched = true
      character = strings.first[k]
      strings.each { |str| all_matched &= (character == str[k]) }
      break unless all_matched

      result += 1
    end
    strings.first.slice(0, result)
  end
end

if __FILE__ == $PROGRAM_NAME
  stems = SongStems.new(ARGV[0])
  stems.make
  stems.delete_originals
end

Linked Mp3s and Images

I wrote about cross-file system linking options in NTFS/ext4 Compatible Aliases.

The following script performs the functionality denoted by the solid line with the label symlink in the production overview image. Step by step:

  1. Automatically categories any media file passed to it.
  2. Categorization is done on the basis of file type.
  3. Files can be accepted with syntax compatible with Bash, or using NTFS directory and file naming conventions.
  4. Incoming files are linked to their proper place.
Shell
$ cd $one_year_older
$ song_link outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3
Wrote /var/sitesUbuntu/www.mslinn.com/songs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3 
Shell
$ song_link $one_year_older/outputs/mp3s/One\ Year\ Older_55sec_136_bpm_2024-11-26.mp3
Writing /var/sitesUbuntu/www.mslinn.com/collections/_songs/mp3s/One Year Older_55sec_136_bpm_2024-11-26.mp3 
#!/usr/bin/env ruby

require 'clipboard'
require 'colorator'

# If this program is run from either of these song directories:
#   $songs/one_year_older/
#   $songs/one_year_older/outputs/mp3s/
# then by default any pre-exising media file in
# the proper subdirectory of $msp/collections/_songs/
#
# The following is true for two equivalent values of the environment variable called songs:
#   export songs=/mnt/c/media/songs
#   export songs='C:\media\songs'
# This invocation:
#   $ cd $songs/one_year_older
#   $ song_link outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3
# Performs the same action as all of the following:
#   $ song_link $songs/one_year_older/outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3
#   $ song_link $songs/one_year_older/outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3 $msp/collections/_songs/mp3s
#   $ song_link C:/media/songs/one_year_older/outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3
#   $ song_link C:/media/songs/one_year_older/outputs/mp3s/One_Year_Older_136_bpm_2024-11-02.mp3 U:/var/sitesUbuntu/www.mslinn.com/collections/_songs/mp3s
#
# @param src [String] Can be either a Windows path or a WSL path; relative or absolute
# @param dest [String] Defaults to common mp3s directory for all songs in mslinn.com
#                      Will be overwritten if it already exists.
class SongLinker
  def initialize(src = nil, dest = nil)
    @msp       = compute_msp
    @src       = compute_src src
    @src_name  = File.basename @src
    @song_name = compute_song_name @src
    @dest_dir  = compute_dest dest
    @dest_path = "#{@dest_dir}/#{@src_name}"
  end

  def compute_dest(dest)
    dest = expand_env(dest)
    dest_www = if dest.nil? || dest.empty?
                 subdir = compute_subdir @src_name
                 "#{@wsl_vol}#{@msp}/collections/_songs/#{subdir}"
               else
                 win_exist?(dest) ? dest : `wslpath -m '#{dest}'`.chomp
               end
    return dest_www if Dir.exist? dest_www

    if File.exist? dest_www
      puts "Error: Destination '#{dest_www}' is a file, not a directory. Aborting.".red
      exit 1
    end

    puts "Error: Windows directory #{@dest_www} does not exist.".red
    exit 2
  end

  def compute_msp
    msp = ENV.fetch 'msp'
    if msp.nil?
      puts "Error: Environment variable msp is undefined".red
      exit 3
    end
    return msp if Dir.exist? msp

    if File.exist? msp
      puts "Error: Environment variable msp points to a file, not a directory: '#{msp}'".red
    else
      puts "Error: Environment variable msp points to a directory that does not exist: '#{msp}'".red
    end
    exit 4
  end

  def compute_song_name(src)
    result = src
              .delete_prefix(expand_env("$songs/"))
              .split('/')
              .first
    containing_dir = expand_env "$songs/#{result}"
    unless File.exist? containing_dir
      puts "Error: Song directory '#{containing_dir}' does not exist".red
      exit 5
    end
    result
  end

  def compute_src(src)
    src = Dir.pwd if src.nil?
    src = File.realpath expand_env src
    if src.empty?
      puts "Error: src file to copy has no value. Aborting.".red
      exit 6
    end

    unless File.exist? src # Did the user provide a WSL path?
      # TODO: wslpath fails if src file to copy does not exist; need to handle this
      src_tx = `wslpath '#{src}'`.chomp # Try again with a Windows path
      unless File.exist? src_tx
        puts "Error: file to copy '#{src}' does not exist.".red
        exit 7
      end
      return File.realpath src_tx
    end
    File.realpath src
  end

  def compute_subdir(src_name)
    case File.extname(src_name).delete_prefix('.')
    when 'gp'
      'guitar_pro'
    when 'gif', 'jpg', 'jfif', 'png', 'webp'
      'images'
    when 'mp3', 'aiff'
      'mp3s'
    when 'mccz'
      'musescore'
    when 'xml'
      'musicxml'
    when 'pdf'
      'pdfs'
    when 'zip'
      'stems'
    end
  end

  # Does not work
  def display_dir
    win_home_dos = `cmd.exe /c "echo %systemdrive%%homepath%" 2> /dev/null`.chomp
    win_home_linux = `wslpath -au '#{win_home_dos}'`.chomp
    Dir.chdir win_home_linux do
      @wsl_vol = wsl_volume_name
      win_dir = unc_to_win(@dest_path)
      puts "Opening #{win_dir}".yellow
      `cmd.exe /c 'explorer.exe /separate, #{win_dir}'`
    end
  end

  def expand_env(str)
    return nil unless str

    str.gsub(/\$([a-zA-Z_][a-zA-Z0-9_]*)|\${\g<1>}|%\g<1>%/) do
      ENV.fetch(Regexp.last_match(1), nil)
    end
  end

  def link
    if File.exist? @dest_path
      puts "Overwriting #{@dest_path}".green
      File.delete @dest_path
    else
      puts "Writing #{@dest_path}".green
    end
    File.symlink @src, @dest_path
    # Clipboard.copy(@dest_path)
  end

  def unc_to_win(path)
    return path.delete_prefix(@wsl_vol) if path.start_with? @wsl_vol

    return path.delete_prefix('//wsl$/Ubuntu') if path.start_with? '//wsl$/Ubuntu'

    path.delete_prefix('//wsl.localhost/Ubuntu')
  end

  def wsl_volume_name
    line = `cmd.exe /c net.exe use`
            .split("\n")
            .find { |x| x.include? 'Plan 9 Network Provider' }
    if line.nil? || line.empty?
      puts 'WSL has no drive letter mapped. Aborting.'
      exit 1
    end
    line.strip.split.first
  end
end

if __FILE__ == $PROGRAM_NAME
  linker = SongLinker.new(ARGV[0])
  linker.link
  # linker.display_dir
end
* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.