Synchronizing multimedia playback on different machines

Synchronizing clocks of different computers in the order of milliseconds can be pretty daunting. This is what is mostly stopping me from doing same stuff on two different machines at the same time without huge latency issues, stuff that could get pretty interesting.

The world would be have been an amazing place to live if we were able to watch videos together from opposite poles on the Earth synchronized to milliseconds, listen to same music together from different places, play online games without any lag at all within the players, etc. Ah, sweet dreams. However, people have been trying to fix some of these problems as mentioned in this answer https://superuser.com/a/1082138/693992 to a similar question.

But before trying out external utilites, I decided to give it a try myself to synchronize audio playback. I primarily use mpv-player for everything multimedia. So, naturally I went on to tinker if I could somehow synchronize playback on different devices with mpv.

I went ahead with my Laptop and Raspberry Pi which have a pretty noticeable difference in startup times of mpv. I thought if I could make it work these, I could probably make it work with everything else. I synchronized my device clocks from an external NTP server and launched mpv with the following params:

$ mpv https://www.youtube.com/watch?v=ixUWy9qdi08 --input-ipc-server=/tmp/mpvsocket --no-video --pause

I imagined this would workaround mpv’s startup time, and once both are ready - I would instruct the IPC server to unpause the mpv on both my Laptop and Raspberry Pi at the same time. Boom! We would now have same audio being given off by both the devices. I installed at tool which allows you to run a command at a specific time. I added an entry to unpause mpv at a specific time on both devices. Something like this:

$ at 20:28
at> echo '{ "command": ["set_property", "pause", "no"] }' | socat - /tmp/mpvsocket
at> <EOT>
job 1 at Mon Apr  1 20:28:00 2019

I tested this but for some reason I would usually end up with reasonable gap between the audio like 1 second or so - enough for any human ear to detect the out-of-sync audio. I suspect this gap might occur from several reasons, at command only checks at a gap of every second to see if there is any job. Let’s say at previously checked at 20:27:59 and 900 milliseconds. The next time it checks for a pending job might be on 20:28:00 and 900 milliseconds. We’re already late by 900 milliseconds. I’m not sure whether this is the case but it could take some investigation. Second reason might probably be because of the small delays that occur on the Raspberry Pi when executing at and writing the data with socat to mpv’s IPC server. The last one would be as simple as the clocks of the two machines are not really synchronized in the order of milliseconds. Or a combination of these. Anyway, I couldn’t get it to work reliably with this approach.

I revisited the answer I mentioned in the first or second paragraph. I tried out one such software mentioned - Syncplay which supports multiple players and mpv being one of them. make went without problems. Although, I initially had trouble to get audio-only output working on my headless Raspberry Pi, but I found out you could pass additional parameters to the player and disabled the video output (I also created an issue #229):

$ syncplay --no-gui -a syncplay.pl -r randomroom --player-path mpv https://www.youtube.com/watch?v=ixUWy9qdi08 -- --no-video --vo=null

On my laptop, I launched the same command without any additional parameters to the player:

$ syncplay --no-gui -a syncplay.pl -r randomroom --player-path mpv https://www.youtube.com/watch?v=ixUWy9qdi08

This resulted in mpv running in pseudo GUI mode on my laptop and I could control the player with it (seek, un/pause). Adjusting the playback so would also cause similar changes to the syncplay client instance running on my Pi. It seems to work pretty well but can sometimes the audio may get out-of-sync. I found that seeking backwards/forwards a bit can help eliminate out-of-sync issues. Syncplay, however, does not seem whether it is intended for such a purpose (of syncing audio between machines). Its main purpose (as on their GitHub page) seems to mainly focus on video playing so that viewers can watch the same thing at the same time, as it states:

Solution to synchronize video playback across multiple instances of mpv, VLC, MPC-HC, MPC-BE and mplayer2 over the Internet.

Like with most softwares I find cool, I tried to get Syncplay running on my Android with Termux. However, mpv isn’t compiled with Lua support when installed with pkg on Termux and I didn’t want to go through the trouble of setting up the toolchain to compile it for Android. Turns out, I just needed to disable loading the Lua script as in https://github.com/ritiek/syncplay/tree/mpv-without-lua-support and somehow everything turned out to be fine!

Either way, I’ll keep Syncplay. :)

I still haven’t dug in the Syncplay codebase deep enough to understand how everything works yet. It might be a fun thing to do so in my spare time.

Written on April 1, 2019