Streaming Pure Data GEM Video Via OBS

28th December 2020

Recently I’ve been doing a bit of streaming with my modular synth and wanted something to make the video side a bit more interesting for anyone watching it. After a bit of searching I discovered Pure Data (aka Pd), and the GEM plugin for it, which adds graphical elements you can create and manipulate in Pd, so I thought I’d give it a go.

Getting audio in was pretty simple, I already use JACK for my audio routing, so all I had to do was configure Pd to use JACK as well, and connect everything up in qjackctl. Then in my patch I created an adc~ object which I could then route the left and right channels from my mixer out from.

After getting the gem package installed and tweaking the settings of Pd to actually load it I managed to get a quick demo up and running fairly quickly and gazed in wonder at the cube spinning in front of me. And then I hit my first problem. I stream at 1280x720 resolution, so wanted to set the GEM window to that size. Looking it up I saw that I could send it a dimen 1280 720 message before creating it and that would change the size of the window. That worked really well, until I tried to make my cube bigger, and straight away I noticed that no matter what I did it would get clipped out where the old window was. So although the window was bigger, I couldn’t put anything in the new space.

This took quite a long time to figure out, and in fact I never did figure it out. I ended up having trouble getting the vanilla Pd distributed with Debian to run with the plugins you need to be equivalent to Pd-extended and ended up installing something else entirely, Purr Data. Not only did it give me access to the objects I needed, it also fixed the annoying window problem, so I’d say Step 1: Install Purr Data.

Now I had a few cubes spinning around in the GEM window, I wanted to get those to show in OBS Studio, which is a really excellent program you can use to combine various audio and video sources in to a stream which you can push to YouTube, Twitch etc etc etc. My first attempt was to use the XWindow capture source in OBS. This worked, but not very well. I don’t know if it’s my window manager or just X11 but it was very flickery, which is no good at all. The next part of this blog is the whole reason I’m writing it, because it took me a stupidly long time to figure out and get working.

So far I’ve been enjoying Pd, but I’ve found resources online a bit lacking. It turns out the help in Pd (or at least Purr Data) itself is pretty good, though looking for things online didn’t really give me much joy, though I did find the v4l2loopback Linux kernel module. If you’re using Debian like me, then you can get this easily by installing the v4l2loopback-dkms package. I then loaded the module and set it up to create a device at /dev/video10. After some failed attempts I found this combination of options to work best:

sudo modprobe v4l2loopback devices=1 video_nr=10 card_label="OBS Cam"

Now, the label bit was copy and pasted from the v4l2loopback wiki, though the example there is using it as a destination for OBS rather than a source, so although the label isn’t strictly accurate for what we’re doing my OBS scenes are now configured to use it so I’m not changing it now.

To add it as a source in OBS it’s really very simple, you just add a Video Capture Device (V4L2) source and choose OBS Cam in the drop-down it comes up with. Now all we need to do is get GEM to send the video output to it.

Again I didn’t find that much info about doing this online, though a thread on the Pure Data Mailing List pointed me in the right direction.

The first thing I learnt is that the way to send the video to a v4l2loopback device is to use the pix_record object with the file to record to set to the device our modprobe command created earlier, /dev/video10. This object however expects a picture, not an object like our cube. So to get video of our cube to be sent via pix_record we need to render each frame as a picture, and map that as a texture on to another object, which is then sent to pix_record. To do this we use pix_snap to take a snapshot of what we’ve rendered, then pix_texture to use it as a texture, before finishing with a rectangle object. In order to preserve the aspect ratio of the original frame this needs to be rectangle 16 9. I translate this object so it’s outside of the view in the Gem window using translate 1 0 0 3, though you can leave it where it is if you want, it just looks a little odd. Finally this is sent to the pix_record object. We then send messages to that object to start and stop recording.

If anyone wants to use this method, I’ve attached an example patch gem-record-example.pd that should get you up and running, here’s the gist of it:

Install Purr Data
Install v4l2loopback - the v4l2loopback-dkms package on Debian derived distros
Create a v4l2loopback device
Make your scene in Purr Data with gemwin, gemhead etc
Add another gemhead with a higher priority number
Add a trigger a b to that, and connect both outouts to pix_snap set to capture the whole window n.b. the docs say it defaults to doing this but that didn’t seem to work for me
Add a pix_texture to that, then translate 1 0 0 3, then a rectangle 19 6. This is where the capture will be applied as a texture, it doesn’t need to be visible in the window, which is what the translate is for.
Add the final object, pix_record to the end of that chain. To start sending video to the device send the message codec v4l2, file /dev/video10, auto 1, record 1. To stop sending it send record 0.
Open OBS, and add a Video Capture Device (V4L2) source and choose OBS Cam.
Use OBS as normal, you should now be able to see the scene you creted in Gem, without flickering.