Mike Slinn
Mike Slinn

TouchDesigner For Music Video Creation

Published 2025-01-29. Last modified 2025-02-01.
Time to read: 23 minutes.

This page is part of the av_studio collection.

I have been searching for a truly all-singing, all-dancing music video generation capability. After obsessively immersing myself with the TouchDesigner for several days, I feel certain that it can take care of that somewhat nebulous requirement quite nicely.

TouchDesigner is a procedural modeling program. Alternative programs, some of which take a different approach, include the following:

Interesting related products include:

About TouchDesigner

TouchDesigner is a node-based visual programming environment used for real-time interactive multimedia and generative visuals. Developed by Derivative.ca, a Canadian company based in Toronto, TouchDesigner is widely used for creating interactive installations, VJ performances, projection mapping, and live visuals.

TouchDesigner has been developed over decades. The historical information I could find was a bit vague on details, but it seems that TouchDesigner was first released in 2002. It’s immediate ancestor, written by some of the same people, was Houdini 4.1. Houdini is now at release 20.5. I would like to hear the story from the Houdini people’s point of view.

Technology has changed and advanced quite a lot in the past 23 years. Many devices that were popular then are no longer available. It seems that TouchDesigner has retained compatibility with obsolete devices. I was surprised to see the extent of the AI support that TouchDesigner has incorporated for years. Something old, something new ... we seem to be halfway to a wedding vow.

There is a lot to like about what this program can do, and how it operates. I especially like the user interface and the documentation.

The feature matrix shows the features for the free version and all the other versions. Many of the really cool features require a paid license, but the free version has quite a lot of capability.

Non-Commercial Educational Commercial Pro
Price Free $300 USD $600 USD $2200 USD
Floating Cloud N/A $450 USD $900 USD $3300 USD
TouchPlayer Free N/A $300 USD $800 USD

Unfortunately, some core MIDI and video functionality seems to be unfinished. I discuss the specifics in this article. At this price point I expect more polish.

All of the TouchDesigner releases are shown here. The first use of the 099 branding was in 2019, 6 years ago. Here is some corresdance I had with Derivative:

Email Support Thread

From: Mike Slinn
Sent: February 9, 2025 10:36 AM
To: Derivative Support <support@derivative.ca>
Subject: Current TouchDesigner version

v099 was released in 2019. I searched for updates but did not find anything mentioned publicly. Does this mean there have been no updates for 6 years, or is it that the free edition is not kept up-to-date, and only paying customers get updates? If the latter, what is the current version?
Thank you,
Mike
On 2025-02-09 11:18 a.m., Malcolm Bechard wrote:


Hey Mike,

We dropped the 077/088/099 versioning, now it's just 2022.20000, 2023.10000 etc. We never stopped releasing builds for all users. You can see them all on our download page:
derivative.ca/download/archive

Malcolm
From: Mike Slinn
Sent: February 9, 2025 12:07 PM
To: Malcolm Bechard
Subject: Re: Current TouchDesigner version

Malcolm,

Thanks for the quick response. I am confused by the page you referred me to.
I see a column of build numbers followed by a column of dates. The most recent row shows dates that are about a year apart. It seems the most recent version, 2023.12120, was released 2024-12-16.

The version labeled "current release" available from the front page downloads TouchDesigner.2023.12120.exe. Help / About displays:
The version reports as 099, with build 2023-12120, created 2024-12-16 and copyright is claimed for 2025. These values do not line up.

The release notes show new features and bug fixes, so this build is truly is a new release.

The download page calls this "TouchDesigner 2023". That is confusing. There were 6 releases in 2024. Why was this not called "TouchDesigner 2024-12" or something similar?

Copyright date is the publication date. It can only be updated for a published revision, and has no relation to the date the program was last executed, so that information is incorrect.

Can you make sense of this for me?

Thanks,

Mike

Subject: RE: Current TouchDesigner version
From: Malcolm Bechard
Date: 2025-02-09, 1:27 PM
To: Mike Slinn <mslinn@mslinn.com>, Derivative Support <support@derivative.ca>
Hey Mike,

Thanks for your questions. The 099 that is still on the splash screen is just a leftover branding that we never got rid of. Our next major release series in 2025 will have it removed though. I'm not sure what you are referring to about `a year apart though'?

The builds we are releasing as our Official are still versioned 2023.1XXXX since they are still from the same 2023.10000 release series. We've continued to update that build series with features/bugs fixes, but major changes are coming in future release series. This is similar to many other softwares that use a year in their name (Office 2021 for example) but still give updates past that year, with the name staying 2021.

The fact the About page is saying copyright 2025 is and oversight. It doesn't say that on the main splash, but for some reason its auto updating on the about screen incorrectly.

Malcolm

Dependencies

The information in this section is generally not important for using TouchDesigner. However, knowing the components that were used to build a product helps develop an intuitive sense of that product’s capabilities, and often provides guidance on how to approach problems encountered while using the product.

Implementation details should not normally be visible to users. Because TouchDesigner seems to be extremely well designed, the dependencies that I show below merely provide the desired functionality without forcing users to contend with implementation details.

Example: Installation Debug

TouchDesigner uses Vulkan as a graphics engine. Vulkan is a low-level, low-overhead cross-platform API and open standard for 3D graphics and computing. Knowing that helped debug a problem.

I have several laptops. One of them is a 7-year-old Dell 9575 2-in-1. This model laptop has kinds of two video chips on the motherboard: a Radeon RX Vega M Graphics, and an Intel HD Graphics 630.

When I attempted to run TouchDesigner for the first time on the Dell 2-in-1, the following appeared:

To debug the problem, I installed and ran the Intel Driver Assistant on the 2-in-1 laptop. The software utility identified that the video driver for the Radeon RX Vega M Graphics chip needed updating.

😁 😁

After updating the video driver, TouchDesigner worked on the 2-in-1 laptop.

The Dell 9575 2-in-1 laptop can only hold a maximum of 16 GB RAM, so I do not expect it to be able to run demanding TouchDesigner networks at an acceptable frame rate. When running alongside Chrome and WSL, the laptop’s RAM pages quite a bit. Thankfully, the laptop’s NVMe storage has a high enough bandwidth that the constant paging does not impact performance too much.

Having two kinds of video chips means the Dell 2-in-1 laptop runs small TouchDesigner networks quite nicely. I do not expect that a demanding network would perform adequately on this laptop due to the limited memory available, but I have not tested that.

In theory there is no difference between theory and practice – in practice there is.

  – Yogi Berra

I am curious about test results.

Similar But Different

I also have an 8-year-old Dell XPS 15 9570 with 32 GB RAM and two other video chips. I think the 4K display feature that I opted for when I ordered the XPS 15 slows TouchDesigner down on this laptop quite a lot. This configuration of XPS 15 runs TouchDesigner less well than the 2-in-1 does, even though the XPS 15 has twice the memory of the 2-in-1.

I wonder what might happen if I connected a 4K external monitor to the XPS, and displayed TouchDesigner on it? Perhaps I might try that sometime.

In contrast, my custom-built desktop has 64 GB RAM and an MSI GeForce RTX 3060 Ventus 12G OC GGDR6 video card. It runs large TouchDesigner networks much quicker. Even though I continuously run Chrome and WSL in addition to TouchDesigner, I have not yet run out of memory. Surely this becomes an issue at some point.

Bill Of Materials

Following is a descriptive list of the DLLs that I found installed in %ProgramFiles%\Derivative\TouchDesigner\bin. As previously mentioned, I evaluated the free version 099, which is 6 years old. The current paid version, not publicly visible, might utilize different dependencies.

You could think of this list as similar to the list you might make when examining a car. Your list would consist of a brief description of the externally visible components, for example the engine, transmission, navigation, seats, doors, etc.

It seems that TouchDesigner provides support for a many types of devices, with many integration options. TouchDesigner still provides support for many discontinued devices. Kudos!

I was surprised to see the amount of neural network, deep learning and image recognition support that these dependencies provide. I installed TouchDesigner on Windows 10; Mac users will have some different dependencies.

  • 3Dconnexion SpaceMouse provides 3D mouse support.
  • Allied Vision Technologies Camera API provides the GigE Vision driver and API.
  • Alembic is a F/OSS library jointly developed by Sony Pictures Imageworks (SPI) and Lucasfilm Ltd, and available on GitHub. Alembic is a framework for storing and sharing scene data that includes a C++ library, a file format, and client plugins and applications.
  • Apache Xerces-C++ XML parser is a F/OSS validating XML parser written in a portable subset of C++.
  • Boost is a general-purpose F/OSS library of utility functions for C++.
  • Cubemos Skeleton Tracking Software; Intel has discontinued the Cubemos Skeleton Tracking software product, but TouchDesigner continues to support it.
  • DeltaCast VideoMasterIP SDK; is an SDK for high-performances multimedia capture and streaming. The IP Virtual Card is a new software framework allowing the use of 3rd-party NIC (Network Interface Card) for high-performances multimedia capture and streaming. It supports SMPTE ST2110 norms, including compliant packet pacing and PTP synchronization.
  • DepthAI allows users to develop and deploy vision pipelines.
  • E57 Scanning Format is a compact and vendor-neutral format that is used for storing point clouds, images, and metadata produced by 3D imaging systems, such as laser scanners. This file format is specified by the ASTM, an international standards organization, and it is documented in the ASTM E2807 standard.
  • EMVA GenICam provide a generic programming interface for all kinds of devices (mainly cameras), no matter what interface technology (GigE Vision, USB3 Vision, CoaXPress, Camera Link HS, Camera Link etc.) they are using or what features they are implementing. The result is that the application programming interface (API) will be identical regardless of interface technology.
  • FreeType is a F/OSS library to render fonts.
  • HeliosLaserDAC is a F/OSS node for working with the Helios Laser Dac, which is a USB to ILDA interface/adapter that connects laser show projectors to computers.
  • Intel nGraph is an open-source C++ library and runtime / compiler suite for deep learning ecosystems.
  • Intel RealSense provides stereo depth camera support. Intel just spun this group out; its future is unknown.
  • Intel Threading Building Blocks is a flexible performance library that simplifies the work of adding parallelism to complex applications across accelerated architectures.
  • LibAV is a F/OSS collection of libraries and tools to process multimedia content such as audio, video, subtitles and related metadata.
  • Libcurl; libcurl is a F/OSS client-side URL transfer library, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, HTTP/3, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos), file transfer resume, and http proxy tunneling.
  • Liblz4 is a F/OSS lossless compression algorithm, optimized for fast decompression.
  • Libmd is a multi-threaded math library for Microsoft Windows provided by Intel.
  • LIVE555 Streaming Media is a set of F/OSS C++ libraries developed by Live Networks, Inc. for multimedia streaming. The libraries support open standards such as RTP/RTCP and RTSP for streaming, and can also manage video RTP payload formats such as H.264, H.265, MPEG, VP8, and DV, and audio RTP payload formats such as MPEG, AAC, AMR, AC-3 and Vorbis. It is used internally by well-known software such as VLC and mplayer. The software distribution also includes a complete RTSP server application, RTSP clients and a RTSP proxy server.[2][3][4]
  • FFmpeg is a F/OSS suite of libraries and programs for handling video, audio, and other multimedia files and streams.
  • Microsoft DirectML; Direct Machine Learning (DirectML) is a low-level API for machine learning (ML).
  • Microsoft Kinect Common Bridge is an archived complement to the discontinued Kinect for Windows SDK that makes it easy to integrate Kinect scenarios in creative development libraries and toolkits.
  • Microsoft ONNX Runtime powers AI runtime training and inferencing.
  • MQTT is an OASIS standard messaging protocol for the Internet of Things (IoT). It is designed as an extremely lightweight publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth.
  • NVIDIA AR SDK utilizes the Tensor Cores on NVIDIA RTX GPUs to enable fast, high quality tracking through an AI network.
  • NVIDIA Broadcast SDK optimizes audio and video streams for broadcasting.
  • NVIDIA cuBLAS
  • NVIDIA CUDA and Cudart
  • NVIDIA cuDNN; the NVIDIA CUDA Deep Neural Network library is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.
  • NVIDIA cuFFT provides the CUDA Fast Fourier Transform library.
  • NVIDIA CUDA NPPCORE library is a library of functions for performing CUDA accelerated 2D image and signal processing.
  • NVIDIA Nsight Aftermath SDK is a library that integrates into a D3D12 or Vulkan game’s crash reporter to generate GPU “mini-dumps” when an exception or TDR occurs, exposing pipeline information to resolve an unexpected crash.
  • NVIDIA FleX is a particle based simulation technique for real-time visual effects.
  • NVIDIA Flow is a volumetric fluid based simulation of a burning gas system.
  • NVIDIA TensorRT is an ecosystem of APIs for high-performance deep learning inference. TensorRT includes an inference runtime and model optimizations that deliver low latency and high throughput for production applications.
  • NVIDIA Tools Extension Library is a cross-platform API for annotating source code to provide contextual information to developer tools.
  • NVIDIA® Vision Programming Interface (VPI) is a software library that implements computer vision and image processing algorithms on several computing hardware platforms available in NVIDIA embedded and discrete devices.
  • Oculus OVR Audio SDK was a tool for developing audio for Oculus devices. Oculus/Meta no longer supports this, but TouchDesigner still does.
  • OpenColorIO is a complete color management solution geared towards motion picture production with an emphasis on visual effects and computer animation.
  • OpenCV is a F/OSS computer vision library that contains over 2500 algorithms.
  • OpenEXR implements the EXR file format, a professional-grade image storage format for the motion picture industry.
  • OpenMP is a cross-platform library for shared-memory multiprocessing.
  • The OpenSSL Toolkit provides SSL support.
  • OpenVINO toolkit is a F/OSS toolkit for optimizing and deploying deep learning models. OpenVINO is developed by Intel and stands for "Open Visual Inference and Neural Network Optimization".
  • OpenVR SDK an API and runtime that allows access to VR hardware from multiple vendors without requiring that applications have specific knowledge of the hardware they are targeting.
  • OptiTrack NatNet SDK is a client/server networking SDK for sending and receiving OptiTrack data across networks.
  • Orbbec Femto Bolt is a high performance iToF 3D camera jointly developed by Orbbec and Microsoft. The camera uses the same depth module as Azure Kinect DK, continuing the same depth modes and performance.
  • Orbbec SDK is a cross-platform library supporting various platforms and languages for Orbbec 3D cameras.
  • Pixar Animation Studios Universal Scene Description a high-performance extensible software platform for collaboratively constructing animated 3D scenes, designed to meet the needs of large-scale film and visual effects production. USD enables robust interchange between digital content creation tools with its expanding set of schemas, covering domains like geometry, shading, lighting, and physics. USD’s unique composition ability provides rich and varied ways to combine assets into larger assemblies, and enables collaborative workflows.
  • Python 3.11.1
  • Rav1e is an AV1 video encoder.
  • RED R3D SDK is a RAW decoder for RED R3D files.
  • RPly is an ANSI C library that allows applications to read and write PLY files. PLY is a file format that stores geometric information, such as 3D models.
  • Scalable Display Technologies EasyBlend provides image perspective correction and blending for projection onto curved surfaces.
  • SICK LiDARs and Radar
  • Spout is a F/OSS library that provides realtime video routing for Windows. Using Spout you can send video between applications like Resolume, Max, Processing, MadMapper, TouchDesigner and more. Create graphics in Processing, mix them in Resolume and projection map them with Mad Mapper. Throw in some interactive content from TouchDesigner, Max or Unity while you are at it.
  • Steam Audio Core is a cross-platform software toolkit for spatial audio.
  • StereoLabs ZED SDK stereo camera for spatial perception.
  • Adobe Substance 3D Painter is a texturing app for 3D professionals and enthusiasts. This library was used by Houdini, TouchDesigner’s predecessor.
  • TeleDyne FlyCapture SDK; for machine vision cameras.
  • TeleDyne Spinnaker SDK; The TeleDyne Spinnaker SDK library supports machine vision applications that require FLIR USB3, 10GigE, and most GigE area scan cameras.
  • Vulkan is a low-level, low-overhead cross-platform API and open standard for 3D graphics and computing. It was intended to address the shortcomings of OpenGL, and allow developers more control over the GPU. It is designed to support a wide variety of GPUs, CPUs and operating systems, and it is also designed to work with modern multicore CPUs.
  • Ximea SDK provides support for Ximea machine vision cameras.
  • WebRTC; C bindings for Google Chrome WebRTC Native.
  • Zlib is a F/OSS software library for data compression. It is used by libpng and is widely used to compress files to the gzip format.

Getting Started

The written documentation for TouchDesigner is good; however, visually oriented programs like TouchDesigner benefit from video tutorials. I added comments to the following quote; my comments look like this.

 – Heavily edited excerpt from the TouchDesigner User Guide

After installing TouchDesigner, a good place to start learning is the Beginner Crash Course by Interactive and Immersive HQ. Just let each lesson autoplay. The pacing is just right, is not repetitive, and has many practical reasons for why each morsel of information is useful.

Lots Of Help Is Provided

Microsoft customers are used to getting help by selecting something they want to learn about, then pressing F1. If you do that when running TouchDesigner, an empty, black projector window will open. Closing the window by pressing the UI control exits TouchDesigner.

It is better to press the ESC button to exit the projector window. TouchDesigner has lots of help available – just do not press F1 when you want to view it.

Help information for a node can be found by right-clicking on the node and selecting Help....

A new tab will appear in your computer’s default web browser, displaying the appropriate page from docs.derivative.ca.

You might see an icon that resembles a small grid sprinkled into the text of the help page. I believe that this icon is meant to resemble a plus sign within a square. Click on an instance of this icon to reveal more information. The icon does not change when you click on it to indicate that it has been expanded. The UI designer could have done a better job here.

Contextual help is also available from the parameters’ pane, which displays two help icons in the left-most portion of its title bar. The question mark icon over a black background opens the above-mentioned help page.

See the small Python-colored square?
See the small Python-colored square?

Help for Python and C++ programmers is displayed when you click on the question mark icon that looks like it is in front of the Python colors (blue and yellow, separated by a thin white bar). This icon looks rather like a square Ukrainian flag with a question mark in front.

The Help / Operator Snippets menu item is particularly useful. It opens a new TouchDesigner window in which you can find many useful networks with embedded Python snippets. You can paste these subnetworks into your network.

Operator Snippets window<br>Mouse over the image to enlarge it
Operator Snippets window
Mouse over the image to enlarge it

Programming Interfaces

TouchDesigner provides a nice visual programming environment, into which snippets of code can be sprinkled. Larger chunks of code can also be woven into a network (of nodes).

The main programming interface supported by TouchDesigner in 2025 is Python. The version bundled with TouchDesigner is Python v3.11. C++ is also supported. TypeScript support is deprecated.

Python Snippets

You can experiment with Python code within a TouchDesigner Textport, which can be opened by typing ALT-T.

To obtain a reference to a node, pass the fully qualified path of the node to the op function.

TouchDesigner Textport
TouchDesigner Build 2023.12120 compile on Mon Dec 16 15:05:18 2024
Python 3.11.1 (heads/3.11-Derivative-dirty:82b0389147, Jan 25 2023, 22:34:27) [MSC v.1929 64 bit (AMD64)] 

python >>> r = op('/project1/moviefilein1')

Nodes have attributes and methods. Two commonly used attributes are name and opType.

TouchDesigner Textport (continued)
python >>> r.name
'moviefilein1' 

python >>> r.opType
'moviefileinTOP'

Python programmers are familiar with how to obtain the list of all available attributes and methods for a node: Just pass a node reference to dir. Python programmers might expect that a similar result could be returned by __dir__; however, many TouchDesigner objects do not support __dir__.

TouchDesigner Textport (continued)
python >>> dir(r)
['OPType', '__add__', '__bool__', '__class__',
'__delattr__', '__dir__', '__doc__', '__eq__', '__format__',
'__ge__', '__getattribute__', '__getstate__', '__gt__',
'__hash__', '__init__', '__init_subclass__', '__le__', '__lt__',
'__mul__', '__ne__', '__new__', '__radd__', '__reduce__',
'__reduce_ex__', '__repr__', '__rmul__', '__setattr__',
'__setstate__', '__sizeof__', '__str__', '__subclasshook__',
'_customPages', '_pages', 'activeViewer', 'addError',
'addScriptError', 'addWarning', 'allowCooking', 'aspect',
'aspectHeight', 'aspectWidth', 'base', 'builtinPars', 'bypass',
'changeType', 'children', 'childrenCPUCookAbsFrame',
'childrenCPUCookTime', 'childrenCPUMemory', 'childrenCookAbsFrame',
'childrenCookTime', 'childrenGPUCookAbsFrame', 'childrenGPUCookTime',
'childrenGPUMemory', 'clearScriptErrors', 'cloneImmune', 'closeViewer',
'color', 'comment', 'cook', 'cookAbsFrame', 'cookEndTime',
'cookFrame', 'cookStartTime', 'cookTime', 'cookedPreviousFrame',
'cookedThisFrame', 'copyParameters', 'cpuCookTime', 'cpuMemory',
'cudaMemory', 'curBlock', 'curPar', 'curPass', 'curSeq', 'current',
'currentPage', 'customPages', 'customParGroups', 'customPars',
'customTuplets', 'dependenciesTo', 'depth', 'destroy', 'digits',
'display', 'dock', 'docked', 'error', 'errors',
'evalExpression', 'expose', 'ext', 'family', 'fetch', 'fetchOwner',
'fileFolder', 'fileHeight', 'filePath', 'fileWidth', 'findHeader',
'getHeader', 'gpuCookTime', 'gpuMemory', 'hasAudio',
'hasDecodeErrors', 'height', 'icon', 'id', 'index', 'indexFraction',
'inputConnectors', 'inputs', 'iop', 'ipar', 'isBase', 'isCHOP',
'isCOMP', 'isCustom', 'isDAT', 'isFileOpening', 'isFilter',
'isFullyPreRead', 'isInvalid', 'isLastFrame', 'isLoopFrame', 'isMAT',
'isMultiInputs', 'isObject', 'isOddField', 'isOpen', 'isOpening',
'isPanel', 'isPreloading', 'isSOP', 'isTOP', 'label',
'lastIndexUploaded', 'licenseType', 'lock', 'maxInputs', 'minInputs',
'mod', 'name', 'newestSliceWOffset', 'nodeCenterX',
'nodeCenterY', 'nodeHeight', 'nodeWidth', 'nodeX', 'nodeY',
'numChildren', 'numChildrenRecursive', 'numFrames', 'numHeaders',
'numImages', 'numSeconds', 'numpyArray', 'op', 'opType',
'openMenu', 'openParameters', 'openViewer', 'ops',
'outputConnectors', 'outputs', 'pages', 'par', 'parGroup', 'parGroups',
'parent', 'pars', 'passive', 'path', 'pixelFormat', 'preload',
'python', 'rate', 'recursiveChildren', 'relativePath',
'render', 'replicator', 'resetNodeSize', 'resetPars',
'resetViewer', 'sample', 'save', 'saveByteArray', 'scriptErrors',
'selected', 'seq', 'setInputs', 'shortcutPath', 'showCustomOnly',
'showDocked', 'sourceChannels', 'start', 'storage', 'store',
'storeStartupValue', 'subType', 'supported', 'tags', 'time', 'timecode',
'totalCooks', 'trueIndex', 'trueNumImages', 'type', 'unload',
'unstore', 'unstoreStartupValue', 'valid', 'var', 'viewer',
'visibleLevel', 'warning', 'warnings', 'width']

The above gives us the names of 186 attributes and methods provided by moviefilein nodes. Many of them are described in Working with OPs in Python

Calling help() and passing a node reference yields many screens of documentation. This is too much information to view in a programming interface. I discussed how to view this help information above.

MIDI Controllers

Since I use DAWs a lot, I have many MIDI devices and find it natural to use their knobs and sliders to manipulate parameter values. TouchDesigner provides support for external MIDI controllers.

There are two MIDI In ops: a MIDI In CHOP and a MIDI In DAT. Initially, they will be empty. As each key, knob or slider on a connected MIDI controller is adjusted, those controls will be added to the list of known controls in all MIDI In nodes.

The TouchDesigner MIDI documents are not the best place to look for information about how to get MIDI controllers to work with TouchDesigner.

Matthew Ragan made 4 helpful videos about using MIDI controllers with TouchDesigner in 2017, 8 years ago. He explains many things, but does not consummate the information by telling you how to actually control something with a MIDI device. I figured out a way, which I will show you in a moment.

A few important points from the above video:

  • MIDI In CHOP provides MIDI data from a MIDI control ranging from 0 to 127.
  • MIDI In Map CHOP provides normalized MIDI data from a MIDI control ranging from 0 to 1. This op provides a naming convention for controls that are stored within a TouchDesigner project. If we open the /local/ network, the midi Bas can be saved to a separate file and then used in other projects by (somehow) merging midi Bases.

The MIDI Device Manager Dialog (ALT-D) shows a MIDI learn button, but it was always deactivated for me. Supposedly this feature was added so long ago that the exact date is lost in the mists of the distant past. On the other hand, the above video did not show MIDI learn functionality, and it walks through the painful details of coping without MIDI learn. Perhaps I will learn more about this one day.

Not having a MIDI learn button meant that much painful typing is required. Moreover, I found that the input sequence does not flow well, as there are idiosyncrasies about how a new control is recognized. I had to fiddle with controls until they all worked. Defining a new knob would deactivate recently defined knobs, so I would have to re-jiggle them in the order they were added before new knobs would function. This dialog needs more work; TouchDesigner seems unfinished in this regard.


Obtain The Value Of A Slider

To obtain the value from a mapped slider on a Novation LaunchControl XL, pass the value through the following linear network:

  1. midiin1
  2. select1 (I used this op to rename the MIDI data channel to v1)
  3. math1
  4. null3

The incantation I used for colorr (red) in a color picker is:

color_picker.colorr expression
op('/project1/null3')['v1']

I typed the above into a TextPort, so fully qualified references should be used. References to other components in a different scope can use relative paths. References within the same scope do not need a path.

The incantation I used for colorg (green) in a color picker is:

color_picker.colorg expression
1-op('/project1/null3')['v1']
Mouse over this image to enlarge it
Mouse over this image to enlarge it

I found an open-source MIDI learn function for MIDI controllers, packaged as a tox file; the project has a GitHub page. I have not tried it yet.

Projector Output

Projectors with resolution compatible with the free version of TouchDesigner are quite inexpensive.

Connect a Window node COMP (found at the bottom of the Other column) to the TOP node that you want to project, and then adjust the Window node parameters.

This is an excellent video on how to set up projector output:

Exporting MP4

TouchDesigner’s video recording capability is weak, and using it is more difficult than it should be. This capability seems to have been implemented decades ago, and not much work has been put in since it was originally created.

There are at least four ways to record FFmpeg or QuickTime movies with TouchDesigner.
 – From TouchDesigner User Guide

The Export Movie dialog (discussed below) is a user interface built around the Movie File Out TOP. The Movie File Out TOP is more flexible than the Export Movie dialog, which has issues.

Recording starts at the current playhead location. This is almost never what you want for a music video. Be sure to rewind before recording.

Exporting video will render in real time unless Realtime mode is disabled. This toggle should be automatic. Otherwise, if Realtime mode is enabled (which is the default), exporting a video will take five minutes to process a five-minute-long video, and quality might be impacted when heavy processing is required.

The default extension is mov, which causes the video to be written using the QuickTime container. If you change the extension to mp4, that container format will be used. However, MP4 videos are not supported by the free edition of TouchDesigner.

Although most MP4 files have an MP3 audio stream, TouchDesigner’s MP4 implementation does not support MP3 so MP3 audio streams cannot be used in videos that use the mp4 container.

Another popular audio codec for MP4s is AAC, but TouchDesigner does not support that either.

In the upcoming 2023.30000 series (or 2024.20000 likely by the time it’s out), there is a new ‘Movie Container’ parameter in the Movie File Out TOP which makes this easier to control.
 – From forum.derivative.ca

I did not see the Movie Container parameter in the Export Video dialog.

Movie File Out TOP

I am searching for a better way of exporting video. Several alternative approaches are being considered.

The Movie File Out TOP saves a TOP stream out to a movie file (.mov/.mp4) using a variety of codecs, including the H.264/H.265, Hap Q, NotchLC, Apple ProRes and Animation video codecs. It can also save single frame images, image sequences, or stop-frame movies.

For codecs that support Alpha, use the 'Movie Pixel Format' parameter to select a format that includes alpha.

To record movies with audio using the Movie File Out TOP, a Time Sliced CHOP with mono or stereo channels of audio is required. If TouchDesigner is running at a lower frame rate than the target video frame rate and a CHOP is specified for audio, the Movie File Out TOP will automatically repeat video frames to ensure the video and audio stay in sync.

Recording a movie without frame drops can be done in non-realtime by turning off the Realtime flag at the top of the user interface. The length of the video is not predetermined and depends on the amount of time the Record parameter is on.

H264/H265 NOTE: Encoding movies in H.264/H.265 codec is only available with a Commercial or Pro license. Nvidia graphic hardware is also required.

 – From docs.derivative.ca

There is no warning when a user attempts to export an MP4 without a paid license. Instead, the render process proceeds, but no video file is produced. This needlessly confuses and frustrates users. Derivative missed a strategic opportunity to upsell free users, and instead leads users who are evaluating the product to doubt its integrity.

Video codec choices are:

  • H.264 (NVIDIA GPU) creates compressed H.264 video files for you without stressing out your CPU, but requires an NVIDIA GPU.
  • MPEG 4 (Part 2) creates video files without an NVIDIA GPU.
  • For the best playback performance, use HAP or HAP Q.

Export Dialog

CTRL-M brings up the Export Movie dialog, which only outputs QuickTime movies.

This dialog allows you to export QuickTime movies from any viewer in TouchDesigner.

This export to movie does not run in real time. The rendering takes the necessary time required to display and write each frame completely. This allows movies to be produced at the highest resolutions the graphics hardware can support, while avoiding choppy playback and low frame rates that these resolutions can experience in realtime.

 – From docs.derivative.ca

Drag a TOP overtop the field labeled TOP Video and drop. For me, this resulted in the field being set to the value /project1/multiMix. This is the fully qualified name of the TOP that was created by the multiMix menu item of the Tools palette.

I dropped the aoudiodevout2 onto the CHOP Audio field, thereby setting it to /project1/audiodevout1.

I set the video Codec to H.264 (NVIDIA GPU). Using this video codec was a mistake, as I found out.

The only filetypes offered were mov and mp4, which makes sense. However, output files were only created when I selected mov, and when I did not specify an output directory.

The frame rate should be equal to the source frame rate. The best way to do this is to specify the frame rate as a Python expression: me.time.rate.

The default Filename is movieout.$N.mov. Do not mess with this value.

The video was encoded in the RLE format, which is not supported by Windows Media Player.

QuickTime Animation format (also known as QuickTime RLE) is a video compression format and codec created by Apple Computer to enable playback of RGB video in real time without expensive hardware.
 – From QuickTime RLE on Wikipedia

Motion JPEG (M-JPEG or MJPEG) is a video compression format in which each video frame or interlaced field of a digital video sequence is compressed separately as a JPEG image.

Originally developed for multimedia PC applications, Motion JPEG enjoys broad client support: most major web browsers and players provide native support, and plug-ins are available for the rest. Software and devices using the M-JPEG standard include media players, game consoles, digital cameras, IP cameras, webcams, streaming servers, video cameras, and non-linear video editors.

Motion JPEG is simple to implement because it uses a mature compression standard (JPEG) with well-developed libraries, and it is an intraframe method of compression.[citation needed]

It tolerates rapidly changing motion in the video stream, whereas compression schemes using interframe compression can often experience unacceptable quality loss when the video content changes significantly between each frame.[citation needed]

Minimal hardware is required because it is not computationally intensive.

JPEG is inefficient, using more bits to deliver similar quality, compared to more modern formats (such as JPEG 2000 and H.264/MPEG-4 AVC).

Technology improvements can be found in the designs of H.263v2 Annex I and MPEG-4 Part 2, that use frequency-domain prediction of transform coefficient values, and in H.264/MPEG-4 AVC, that use spatial prediction and adaptive transform block size techniques. There are also more sophisticated entropy coding than what was practical when the first JPEG design was developed. All of these new developments make M-JPEG an inefficient recording mechanism.

 – From Wikipedia: Motion JPEG

I set Audio Codec to MP3. Sometimes audio was recorded, but most often the result was a video without audio.

I learned that this dialog is quite fragile. Do not change a value unless you are willing to spend time figuring out what combination of things produces usable output. When I used the default Photo/Motion JPEG video codec the resulting video looked much better. This functional component of TouchDesigner would benefit from more work being done on it.

Other Ways to Export Video

I wonder if TouchDesigner works with:

  • Virtual video cables?
  • NDI video routing?
  • OBS Studio
    The easiest way I've found is to use OBS or streamlabs as your main streaming platform. In TouchDesigner, send your video/output out through a NDI/ Syphon/ spout out TOP which can be picked up in OBS/streamlabs. Then OBS/streamlabs can stream out to whatever platforms you want. Someone feel free to correct me if I'm wrong, but I think one of the biggest bottlenecks you can run into is graphics card, so hopefully your rig has some decent graphics processing power.
     – From @steakbird r/TouchDesigner

    Here is the Syphon Spout Out TOP. That seems worthwhile to follow up on.

Transcoding

TouchDesigner renders better video using macOS codecs. Videos using those codecs can be transcoded to more universal codecs using ffmpeg.

Here are a few incantations to convert movieout.1.mov to MP4, with AAC audio:

Shell
$ ffmpeg \
  -i movieout.1.mov \
  -crf 17 \
  -strict -2 \
  movieout.1.mp4

$ ffmpeg \
  -i movieout.1.mov \
  -vcodec h264 \
  -acodec aac \
  movieout.1.mp4

$ ffmpeg \
  -i movieout.1.mov \
  -c:a aac \
  -c:v libx264 \
  -movflags +faststart \
  -vf format=yuv420p \
  movieout.1.mp4

$ ffmpeg \
  -i movieout.1.mov \
  -c:a libfdk_aac \
  -c:v libx264 \
  -movflags +faststart \
  -vbr 3 \
  -vf format=yuv420p \
  movieout.1.mp4
-c:a aac
Use encoder aac to output AAC audio. -c:a aac is a low quality encoder; libfaac is better, and libfdk_aac (the Fraunhofer FDK AAC codec) is even better.
-c:a libfdk_aac
Use the Fraunhofer FDK AAC codec.
-c:v libx264
Use encoder libx264 to output H.264 video
-movflags +faststart
These options enable progressive download. After encoding completes, some information that is normally written at the end of a video file is moved to the beginning so the video can begin playing while it is still downloading.
-crf 17
h264 defaults to -preset medium -crf 23
-vbr 3
Target a quality, rather than a specific bit rate. 1 is lowest quality and 5 is highest quality.
-vf format=yuv420p
chooses YUV 4:2:0 pixel format. libx264 supports many pixel formats and by default will choose a pixel format that most resembles the input. Not all pixel formats are supported by all players. yuv420p is the most compatible and universally supported.

Miscellaneous

  • FunctionStore_tools on GitHub; a collection of TouchDesigner hacks and tools that can help you improve the workflow as well as manipulate the default behavior of TD
  • TDMorph; toolbox for enhanced parametric exploration, preset storage, composition and cueing in the TouchDesigner platform. It is meant to help in creating and performing generative content via various methods and archiving interesting parameter configurations using a JSON format. Furthermore, the library provides a set of tools that developers can use to create their own custom systems.

Reopening The Previous Project

Jump list
Jump list

There is no option for TouchDesigner to automatically reopen the previous project. That is a pity.

For Windows 10 and 11 users, however, jump lists are supported, which is very helpful. You can access a program’s jump list by right-clicking on its icon in the Start menu or the toolbar.

More to come...

The mice are pedaling as fast as they can!

Popup Image
Front page
Expert witness USA
17 articles on LLMs
Studio technology USA
Original music by Mike Slinn
General topics
Ruby language USA
Jekyll-generated websites USA
Git and libgit2 USA
Django-powered e-commerce USA
Mainframe image; Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License by PekoeBlaze
* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.