osc-chat: a multiplayer osc message sender and receiver 1
This page and the project are a work in progress. 2
Quoting myself on the about page:
osc-chat is a real-time ORCA-compatible multiplayer sound generator.
osc-chat receives simple OSC messages via the server, and broadcasts them to all connected clients, and the clients then generate sound based on what they receive.
This is in no way a technical flex, it’s fun - it’s a labour of love. I love making things, I love user manuals and hand-writing documentation, I love the way the internet enables anonymous interactions, I love systems that force collaboration but restrict communication 3, I love pseudonyms and changing identities 4, I love chat rooms 5.
why?
I was messing around with ORCA and I wanted to try receiving the messages it sent and doing something with them, the library I was using gave me the option to broadcast those messages, I’d never made a framework-less frontend before, and the chat room as an interface is appealing to me.
how does it work?
The client side is a webapp that generates sound based on OSC messages it receives. Users send OSC messages using ORCA or a similar tool (or via the form on the frontend) and the server broadcasts those messages to all connected clients. The client then produces sound based on the messages it receives. The interface shows a set of channels and their properties, and depending on the contents of the OSC messages received, users can send messages that control how different channels are configured or that cause the client to produce sound.
It’s written in JavaScript, the server uses osc.js, to receive OSC messages over UDP, with osc.js and ws to broadcast those OSC messages to all the connected clients. The client also uses uses osc.js to receive the messages via ws, and generates sound from those messages with tone.js. I’m not using any frontend frameworks, just vanilla-ish JS, HTML, and CSS, because I’ve never not used one before and I wanted to give it a try. I’ll probably move it to TypeScript at some point.
It’s hosted on EC2 because I needed to be able to expose a UDP port and that was the easiest way, this is a little more expensive than I’d like, but it’s not too bad. I’m running osc-chat in a container, and I’m using nginx as a reverse proxy to handle the SSL. The server persists some messages in DynamoDB.
fun problems/ideas
persisting config between sessions (solved)
With multiple connected clients all responding in the same way to messages, you can’t persist changes to channel properties from the client side because they’d all be persisting the same change, but the server doesn’t have any concept of what a channel is. Solution was to have the server persist the messages that match the format of a channel property change, and then have the clients fetch those messages and play them back when they connect.
generating names for users (unsolved)
To emphasise the chat-like features, I wanted to assign users usernames, but I didn’t want to have to have users sign in or manage auth particularly. I could allow users send a name string as part of the OSC messages, but if I wanted this to stay ORCA-compatible I was limited to only using numbers in the messages. I can randomly generate names for users, but I needed some kind of attribute that uniquely identifies users to use as a seed to generate the name.
Currently it’s set up to use the user’s IP address, which works for users connecting with ORCA. But any messages sent via the frontend all come from localhost and are assigned the same name. So I need a way to either use the IP address if it’s not localhost, or if it is (and the message is coming from the frontend) I need to generate a name based on something else. 6
prioritising interference
Because the message format is pretty open, it would be easy to let users encode a lot of information in each message. A user could send a message to a synth channel with the note, duration, waveform, attack, delay, sustain, release, and volume all in one message. This would give users a lot of control over the sound that was produced, as well as increasing message complexity, and I don’t think it’s as interesting.
So I started thinking about prioritising interference/interaction between users as much as possible. Messages that produce sound are simplified down to just note and duration, and other properties of the sound are set separately and can be changed by other users. This means that if you’re sending a sequence of messages to a channel, another user can change the volume of that channel, or add effects, or change the base sound itself. This makes every sound that’s produced a result of the interaction between users.
giving users a way to interact via the web interface (mostly removed)
Since early iterations I’ve had a form text input in the frontend that allowed users to type in space-spaced OSC messages and send them to the server, primarily for testing. Then I experimented with adding a text-based sequencer to the frontend too, with four tracks of eight characters that you could toggle on and off, and textboxes that you could enter messages into like this:
I subsequently lost interest in the project for several months and I think the sequencer was the reason. Newer (undeployed as of 13/08/24) versions don’t have it 7 and while it makes it less accessible it’s better for it.
I might add re-implement the sequencer somewhere else, or provide another simple interface for sending compatible OSC messsages outside of osc-chat - something keyboard based?
adding effects channels (solved)
todo