How this project came about

I first became interested in networking for audio when I learned of the "distributed audio processing" features built into Apple's Logic Pro DAW. I tried to assemble a test setup, and was highly disappointed when I discovered that this would require a second Mac with near-identical specs and OS version, which was simply beyond my means at the time.

The NetVST project

Around the end of 2016 I learned of the thousands of Windows-only VST plug-ins that had been created (many of them using Jeff McClintock's SynthEdit program, many available for free), and wondered how I might make use of this bounty on my preferred Macintosh/Logic platform. Remembering what I had read about Logic Pro's distributed audio processing, I conceived of a project called NetVST, for which I created my first network-capable Audio Unit and VST2 plug-ins (without JUCE, coding to the low-level APIs).

Just for fun, here's my very first NetVST video, created when the system barely worked. Other videos and much more information can be found on the NetVST wiki.

This slightly later video shows that server-side chaining was already possible in the NetVST system:

For remote hosting of VST plug-ins, I added some networking code to an early, open-source version of Hermann Seib's popular VSTHost. Hermann helped me create a German-language version of the NetVST wiki, but we have since fallen out of touch somehow, so I was unable to secure his agreement to open-source the modified host program, which is called NetVSTHost. I have, however, open-sourced the NetVST plug-ins on GitHub at the at AudioKitNet repo (in the NetVST folder).

Microsoft has been evolving its Windows operating systems quite rapidly, leading to concerns that many unsupported legacy VST plug-ins (especially 32-bit-only ones) might soon be unusable. I decided to see if NetVSTHost could run under Linux using Wine, and discovered that, with a few minor tweaks, it could. Furthermore, I found that a great many Windows-only VST plug-ins could also run in this configuration, and that nearly all compatibility problems which did arise involved only the plug-ins' GUI code. This got me thinking about the notion of splitting off just the DSP aspects for remote execution.

Once I knew NetVSTHost and plug-ins could run on Linux, it was a natural next step to see if it could even run natively on the Macintosh, and I was able to use Mike Kronenberg's terrific WineBottler tool to make that happen. This hastily-recorded YouTube video showing the result turned out to be surprisingly popular (though nowhere near as popular as Arminator, the free VST instrument it features):

 

Tidal changes in the computer market

The NetVST project got me thinking more broadly about distributed processing for audio, especially as I began to notice that while plug-ins' CPU demands kept increasing, the fantastic year-on-year jumps in CPU power we all enjoyed prior to about 2015 seemed to have stopped coming. I noticed that all the "action" in computer design seemed to have migrated away from desktop machines to the newer mobile platforms (phones and tablet) and servers, with the former emphasizing thin-and-light, and only the latter emphasizing brute CPU power.

It occurred to me that these were "tidal" changes, which the audio-software industry would not be large enough to influence. I started to think about distributed systems, which had been a hot research topic in the 1980s, but was mostly forgotten after about 1990.

After my experience with NetVST (and its commercial equivalents such as Vienna Ensemble Pro), which involved VST plug-ins' GUIs appearing in a separate host program—usually on a separate computer—I realized that true distributed systems for music would only be practical if plug-in GUIs could somehow remain in the host DAW, with only their DSP code migrating to an auxiliary CPU.

Early efforts to separate GUI from DSP

At first, I thought there might be a way to separate GUI and DSP without changing a plug-in's code. This would be essential for legacy VST plug-ins, many of whose authors had actually passed away without ever open-sourcing their creations. This video shows an early attempt to achieve this, by running one copy of a plug-in locally, and another remotely, wrapping both in some networking code derived from NetVST, which drew some odd comments from random YouTube viewers (who had no idea of the context). The trick to this is harnessing parameter-automation callbacks in the plug-in API.

AudioKitNet and Web GUIs

The approach shown in that video didn't work very well, and could not be cross-platform, because it required two identical copies of the original plug-in, and of course it would be bound to whatever workstation operating system that plug-in was built for.

Earlier this year (2018) I began contributing to the open-source AudioKit project, which, as a macOS- and iOS-only framework, had its own issues with being platform-bound. I began searching for ways to free at least the core DSP code of this wonderful library for use on other platforms, and of course the notions of distributed processing and "headless" DSP servers were still very much on my mind.

I looked into the notion of using web-programming techniques to create "remote GUIs" which could run on any platform capable of supporting a browser. My first efforts worked but were visually un-inspiring, but with some help from a very capable web programmer named Chris Johnson, I eventually had working Web GUIs like this:

 

AudioKitNet is an AudioKit sub-project which includes an alternate implementation of plug-in networking which is not based on JUCE. (The JUCE version demonstrated at ADC2018 was derived from this.) Its handful of demonstration DSP-service programs are designed to run in conjunction with websocketd, which makes a control interface available to JavaScript web programs via the WebSocket protocol.

WebSocket-based, browser-hosted GUIs should not be considered a substitute for native GUI implementations in plug-ins, but rather as an interesting option for secondary GUIs which can run on whatever device may be convenient for the user, e.g. a touchscreen tablet in a live performance setting.

JuceNetSynth

A final related project is JuceNetSynth (not yet open sourced, coming soon to GitHub). This is a JUCE-based standalone program and plug-in to provide native GUIs for remote DSP-services of the type developed in AudioKitNet and in the demo code for my ADC2018 presentation. (Ignoring the WebSocket aspect of AudioKitNet, these use the same protocol.) The standalone program includes an interactive GUI editor, which creates an XML description file, which the plug-in can load.

The following video shows the standalone JuceNetSynth with a replica of the GUI of a classic Windows-only VST phase-distortion synthesizer called Phadiz, connected to a network DSP service which hosts an actual instance of Phadiz.

This second video shows the same GUI loaded into the JuceNetSynth, compiled as a macOS Audio Unit and hosted in Logic Pro X, connected to the same DSP service which itself hosts the Windows-only Phadiz plugin (under Linux!).