[FR] Use Cuda and / or OpenCL for plugins and VSTi

It would be helpful to use CUDA and or OpenCL for plugins, channel strips, resampling and VSTi like Halion.

CUDA and OpenCL are software API to access GPU (video card) parallel processors. Most mid range video cards do have those processors inside today.

This should greatly offload the main processor and help to reduce or eliminate overloads without buying expensive external processing hardware.

There are a couple VST plugins that use CUDA or openCL so it is definitely possible :

http://gpuimpulsereverb.de/

Softwares in the graphics and video industry are using them as well since a few years (Photoshop, Edius …).

Ideally channel Strips and Reverbs could become the first things with CUDA / OpenCL support.

It’s tantalising knowing all that GPU power is there but there’s a snag: the latency introduced by transferring data to and from the GPU for processing. that step is much slower than to and from the CPU, which is why developers don’t tend to use the GPU for audio DSP. It might be more useful for offline bounce but less practical for realtime.

The LiquidSonics guys acknowledge this problem on the site you linked to. They write:

“GPU Edition: Status Notes
Zero latency mode is not supported by the GPU Edition due to a combination of factors including the mechanisms involved with transferring data to and from the GPU being much more efficient with larger blocks, a current requirement to run CUDA VST plug-ins in a separate thread (making larger block processing more efficient) and the lack of coherency in buffering schemes used by various different VST hosts complicating the above issues. It may become more practical to implement this feature in future using newer GPU architectures.”

Zero latency seems to me only necessary for performers monitoring during recording.

Today this is correctly achieved using hardware interfaces that can bypass the workstation during recording to reduce the monitoring latency to a few ms or less. Today most if not all serious digital audio interfaces can do that.
In the other end a digital audio workstation alone cannot reliably deliver a small enough latency for recording. Even without inserted plugins, as soon as you try to reduce hardware I/O buffers to a very small amount to simulate zero latency, you start to have drop outs.

Those interfaces are using fast ADC / DAC and internal digital switching / mixing. Most even have basic EQ / reverb fast internal processing that can be eventually remote controlled from the DAW so that the performers monitoring can be processed for more musician comfort.
Another solution is to use a direct analog path between microphones / instruments and the headphones using a traditional analog mixing desk. Typically this is the case for recording studios when they still have large analog desks.

For mixing zero latency is generally not necessary. Latency can be compensated (to a limit particularly if mixing for live events). This latency compensation mechanism is implemented inside all modern DAW and digital audio mixing desks.

Do you have an idea about the amount of added latency using Cuda or OpenCL in a setup with a consequent amount of tracks / plugins and an optimized thread / buffer management ?

If this value stay inside reasonable limits, and because of the large amount of processing power available inside GPUs, there is no reason to not use this technology for audio processing and or VSTi. Eventually enhanced for audio use by Steinberg, like they did for ASIO drivers where they created a new standard followed by most hardware manufacturers, GPU processing could help to give a more reliable and more powerful DAW.

For video, the amount of available processing power is impressive and does allow to manage multiple full HD video streams in real time with processing enabled on each stream on a Intel PC. So we can imagine that for audio, this could translate to a phenomenal power because audio tracks are nothing compared to full HD video streams bandwidth.

I agree. For me in post I do care about latency but no that much. Processing power is more important.

I can imagine some future plugins being extremely resource hungry and it being ok if GPUs can be accessed. Latency wouldn’t matter in mixing simply because, well, it doesn’t really.

Having said that I have no hope that it’d be implemented any time soon. I just don’t think anyone wants to spend the resources to code for it when there are other things that can be done instead. But yeah… +1 I guess…