v19 🚀🤯

Release notes for Version 19.0 of VDO.Ninja

https://vdo.ninja/v19/

Version 19 was a long-time coming, although during its development numerous auxiliary tools and services were also developed to help VDO.Ninja users. This releases attempts to push the boundaries on all fronts, including stability, feature requests, and remote control tooling.

If there are problems with version 19.0, the previous version of VDO.Ninja can be found at https://vdo.ninja/v183

Please report bugs and issues to Steve, whom is available on discord at https://discord.vdo.ninja. See project contact detail for other contact methods, although Discord is preferred.

Some highlights ☕

• co-director mode added

• new digital face effects

• better global coverage

• remote control of OBS itself

• http-based remote control API added

• ability to dock Director's room into OBS

• ability to transfer files to guests

• better Raspberry Pi support added

TURN relay server 📡

Several new TURN servers have been added. These include new locations, such as Japan, Singapore, and Brazil, along with added server capacity to LA, Australia, and Germany. TURN servers are used in bridging peer-to-peer connections when a firewall or networking issue prevents a direct connection. They also are used to power the speed-test service.

You can use the speed-test to select specific servers to run speed tests against. It's located at https://vdo.ninja/speedtest with options to change server location near the bottom of the page.

Director related 🎬

• Added an initial implementation of co-directing. Lots more of ideas and polish and improvements coming, but I hope it's a useful start: https://docs.vdo.ninja/director-settings/codirector

  • The main idea is there is a new URL parameter called &codirector or &dirpass, which lets you set a password for the director's room. Just be aware, the first person into the director's room is still the 'main director' and their password sets the password for the co-director.

  • The main director will be able to tell who is a co-director because their control box that appears will be tinted blue; other guests will be grey by default.

• The microphone gain, when set by the director, will persist after the performer reloads their browser. The saved setting is specific to the room and stream ID used.

• &thirds/&ruler/&grid will add an overlay to the self-preview in group-mode. a rule of thirds overlay. This only works for guests added to a room or faux-room currently; not basic push-links.

Rule of Thirds overlay helps guests stay centered

• Fixed a minor issue where if using &buffer or &sync in OBS the ability to mute audio in scenes failed. Sync/buffer aren't supported in OBS yet, so using them would previously break some of the audio code needed for muting.

• Added &noisegate as an option, which auto-mutes your mic if someone else is speaking already. The goal of this is to avoid echo and over-speak, so slap it on the annoying guests. Feedback welcomed.

• Added &screensharehide (&sshide); this just hides the local screen-share sub-window that appears when screen sharing in a room.

• Fixed a glitch where right-clicking the invite/scene links and selecting open-in-new-tab did not work correctly.

• Customizable director's dock for OBS example made:

  • https://vdo.ninja/?css=minidirector.css&cleanoutput&hidesolo&director=YOURROOMNAMEHERE

  • The minidirector.css file demos how you can customize the director's room, removing things you may not want from the layout.

  • You can create your own stylesheet, hosting it online, to configure which buttons or layout you want from the director's room.

  • Supports OBS as a dockable widget.

  • Links, like solo links, can be dragged from the dock directly into OBS for easy access to feeds.

  • I created an alias for the default mini director stylesheet &minidirector if you wish to use the official one. Feedback welcomed.

• You can change the order of videos in the director room with these little arrows now.

  • This is not the same as mix-order; this is just for the directors order.

  • It will impact the MIDI hotkeys targeting, since that logic is based on the directors order of videos.

• The 'mute in scene' director's button will now also mute solo-links; not just group scene links. This seems to align with how people want the button to work. It was trickier to get working than expected, requiring core code changes, so likely will need more testing.

• Added a "create timer" button to the director's room; this lets you enter a time in seconds, which then appears as a count down timer for that guest. It flashes red/black when it expires. The director can remove the timer anytime.

• Fixed a bug with the director's settings not reflecting the correct active audio/video devices always.

• &webp mode now still works if the director minimizes or changes focus on the window, without needing electron.

• Added the ability for the director to Request a guest to upload a file to them; just them. The idea is, if the director does a remote recording, the guest can upload it after it's done saving.

• Fixed a bug with the scene-stats were retransmitted kbps was incorrectly labelled.

• Added &hidesolo ; lets you hide the solo links from showing in the director's room.

• Optimized the custom scenes a bit more; audio will now also be disabled when the video is not added yet to the scene. (before audio was always preloaded in all scenes for all connected peers).

  • This should help a small bit with bandwidth and CPU when using many custom scenes.

  • Videos not yet added to scene 1 will have the audio bitrate drop to 16-kbps.

    • Scene 1 has always been special, as the video preloads there, so the quality ramps up immediately and guests appear immediately.

    • Other scenes (>1) or custom scenes will need to ramp up from 0 bitrate, which can take some time to load and ramp up, but this is needed as I can't have every scene pre-loaded else things get overloaded.

  • You can override all this logic I'm using for pre-loading bitrates of videos in scenes now with &preloadbitrate=500

    • 0 disables pre-loading, while -1 allows for unlocked bitrates during preloading.

• The mic gain of remote guests set by the director will re-set automatically after a guest does a page reload.

  • Only works if the same room, password, and stream ID are used by the client.

  • This allows for page refreshing of the guest and the mic volume not changing.

• Added a new setting to the director's room settings. The director can toggle 'highlight director', which will act a bit like a faux-broadcast mode.

  • Broadcast mode is still probably more stable, since it blocks video connections completely, but this mode can toggle between videos on and off, so more flexible.

  • It won't appear if the director is set to 'director is a performer" mode, as there is a different button for that.

• The drop-down selector will select the current audio/video device also; it won't just show the first item in the list, but rather the current active one.

  • Renamed the change-camera/mic "apply" button to "request", to better reflect what's going on when change remote devices, since when you request to change a video/audio device, the remote guest needs to accept the change first.

  • If a remote guest denies the request to change devices, the director is alerted to this fact.

  • Fixed some issues with the remote audio/video selector; when you change devices, it will clear old state more reliably and avoid a race condition so what you see is what you get.

• If you hover over a video as a director, it provides a reminder that you can CTRL/CMD + click on the video for its stats. (local dev)

• Added the &consent flag, which will let the director remotely change the camera/mic source of a guest without the user's acceptance each time. Instead, it just alerts the user when they join the room. (if using &autostart, the message is displayed as an overlay with a message that times out after a few seconds).

Raspberry Pi, Jetson, and Python support added

• The Raspberry_Ninja project has evolved, with increased compatibly and functionality.

• The $10 Raspberry Pi pocket-sized computer can be used to broadcast hardware-encoded video into VDO.Ninja at upwards of 1080p30.

• The Nvidia Jetson is also supported; fluid high-definition video at upwards of 30-mbps, with power-sipping hardware-encoding used.

• Microphone/audio support is available; code samples and documentation.

• Support for $10 HDMI to USB adapters is provided.

• Support for the official and private handshake server deployments is available.

• V19 of VDO.Ninja supports viewer disconnection detection.

• Sample code for Linux-desktop systems is available; much more coming soon to the project.

• https://github.com/steveseguin/raspberry_ninja and https://www.youtube.com/watch?v=J0qqXxHNU_c

General changes, fixes, and new features

• If a guest's video itself is highlighted, it will become full screen now; it won't remain unchanged like before.

  • Fixed an issue where 'highlight guest' caused a conflict with &broadcast mode. Changed it so &broadcast mode takes priority.

• Improved the re-connection logic some more; took a few nights, but some re-connection events should be faster now, and in some cases, near seamless.

  • Purposeful disconnection events, such as when hanging-up or refreshing, are communicated now also, avoiding the need for a timeout disconnection delay.

• Started keeping track of the number of hardware encoders being used on iPhones. There's a total of 3 on iOS devices, so once the h264 encoders are used up, the codec will default to vp8 (software) now.

  • This change applies mostly to scenarios where there is more than one group scene being used, as scenes make use of the iOS h264 encoder by default.

  • iOS devices can overheat or become sluggish if the VP8 codec is used heavily, so this change is a bit of a mixed blessing. It at least avoids video not loading at all.

• Added &stun and &addstun; works in the same was as &turn, except it's a bit more specific for STUN.

  • Using &stun overrides the default google stun servers, while &stun=false deletes the default stun server.

  • &addstun just appends an added stun server. So, you can have both google + custom, or two custom stun servers, with these two options.

  • A change to both &turn and &stun; you can pass &turn=turn:turnserver.com or &stun=stun:stunserver.com without a username or password. If you add the username and password, it will still use it, but if you don't include them, it will assume no credentials are to be used. Example:https://vdo.ninja/beta/?stun=stun:stun3.l.google.com:19302&addstun=stun:stun2.l.google.com:19302&turn=false

• A video recording won't stop if the user attempts to leave the page, but hits cancel when prompted to confirm.

• Documentation has been updated with a 'lite' version of the URL parameters; just the most commonly used ones, all in a list with short descriptions: https://docs.vdo.ninja/getting-started/cheat-sheet-of-basic-parameters

• Fixed an issue related to multiple audio tracks being selected.

  • Instead of trying to send multiple audio tracks over the connection, which has made the code complexity pretty high and uses more bandwidth, I'm merging audio inputs into a single track at the sender's side now.

  • If &stereo is used, this merged track will have stereo channels, with any mono-inputs filling stereo channels to remain mono; this should keep the behavior of the app consistent with before, yet be more stable and audio should stay better in sync.

  • Be warned though, the current master gain-node happens AFTER the mixing, so there may be an increased chance of clipping with loud multi tracks. In the future I can have different gain nodes per track, but that will be for another day.

• If multiple audio tracks are selected by a guest, they should each get their own selectors now for the basic 3 audio settings: aec, gain, denoise.

  • The web-audio effects nodes will apply to both tracks. (comp/gain/low-cut)

  • These new multi-track setting options are available to both remote director and publisher

  • You can hold CTRL (or cmd) to select multiple audio tracks as a publisher.

Publisher's new settings for multi-track audio

Director's new remote settings controls of multi-track audio

• Using &wc2 or &ss2, or using &wc or &ss AND &intro together, will show the "Share your Camera" or "Share your screen" button before asking the user to select camera/screen options. This preps the user for the permissions popup that comes next, but does add an extra button to press.

• Changed the error message seen for mobile users, if the camera request times out. The other desktop-focused error made no sense for mobile users, so the messaging is more helpful.

• Firefox browsers now support dynamic change of Echo Cancellation, Denoise, and Auto Gain.

• Video published with a Firefox browser can now also have their outbound video bitrates be set by the viewer using the &videobitrate parameter. Firefox required custom code to support this feature.

• Identified a bug with Firefox where changing bitrates dynamically too quickly causes it to get stuck, despite it saying otherwise. Applied a timeout delay , which seems to have fixed it.

• Fixed an incompatibility issue in Firefox, where specifying the audio device output destination in Firefox broke the mute speakers button for guests.

• Added the &zoomedbitrate and &zb flag, which lets you set the target bitrate for a guest when they 'zoom in' on a video using the full-window icon in the top-right of a video.

  • Default if the option is not used is 600-kbps I think.

  • Using the flag, with no value passed, will set it to 2500-kbps; or you can set your own.

  • The idea is, you might want to still have a group call, but occasionally share a high resolution screen. This will increase the load a lot on the guest who is being zoomed-in on, but it's an option if increasing &trb is not acceptable.

• The viewer of a stream has "quality limitation reason" added to the stats page of a video, along with info on when the encoder is hardware-based.

  • This should help debugging of guests , as you'll be able to better tell if issues are caused by packet loss or an overloaded CPU. This info was previously-sender side only.

• Added a button that lets a guest share a file with everyone in the group. While I am testing this, the button is shown by default in the control bar, but it can be disabled with &nofileshare or &nodownloads.

  • When someone shares a file with you, it will be listed in the Chat window as a download link; at least for now.

  • There is a progress bar, and clicking it while downloading will cancel the download (after accepting a confirmation prompt).

  • The option to share a file (and not as a media stream) is now provided also in the &fileshare mode. It's a bit crude at this point, but should work in a pinch. (located at bottom of the stream media file page).

  • If exiting (hangup), the app will prompt you to let you know if there are any active downloads are in progress; it will confirm if you want to stop them and exit.

• Audio-optimizations: In scene 1, audio bitrate drops to 16-kbps when not active (video to 400-kbps). In every other scene (2 or custom, etc), when not active, audio and video bitrate drops to 0.

  • You can override all this logic I'm using for pre-loading bitrates of videos in scenes now with &preloadbitrate=500 or whatever, where the value is in kbps. 0 is off. -1 is unlocked.

• Optimizations for iOS;

  • iOS systems will switch to VP8 as an encoder when H264 encoders are exhausted. This will probably increase CPU load issues, but should reduce the chance of an iPhone outputting a dead video stream.

  • Video resolution is lower now from iOS guests to other guests; this increases the frame rate from around 1-fps to around 10-fps. This seems like a better balance.

• Added a &safemode option for guests. This has the guest's camera start in a rather basic fashion, which might help solve problems with certain camera not being able to use VDO.Ninja.

  • Added to the director's customization options as well, under "Compatibility mode".

• The existing &cover parameter will now additionally force the preview video on mobile devices to fill the entire screen when streaming. Normally, the preview video during streaming is smaller to allow for the control bar to have its own space on the screen. With &cover added, the control bar floats on top of the video.

• Added a pop up notification to confirm with the user that they intended to exit/reload the &push page.

  • Added &easyexit or &ee as a flag to bypass this behavior. (also added to &cleanoutput)

  • The pop up confirmation won't show with scenes/view links; just publishers/directors.

• Added the 'not visible' list of guests to the &broadcast only mode view. Users seem to like having it.

• Added custom-made join and leave notification beeps, for when guests join and leave a group room. This just extends the existing &beep command with new beeps and triggering events.

• Added code to try to detect when a user has reloaded and reconnected, versus just reconnected; faster clean up can happen this way and fewer "beeps" needed on those reconnects.

• Turned on &animate by default; so videos move around all animated-like, which looks a bit more sophisticated I think. It is disabled for mobile users by default however, for CPU reasons, and can be disabled for other devices by passing false or 0 to the parameter.

• Added a &cursor option, which attempts to show the mouse cursor during screen shares (cursor is set to hidden by default). This flag is largely pointless though, as most browsers just ignores my settings and does what it want, breaking spec. It's on beta though; article on it here: https://docs.vdo.ninja/source-settings/cursor

• Made a sample web app that demos how to send data back and forth between two peers on the Internet, with very little code needed; just the VDO.Ninja iframe API. I'm hosting it as a GIST and also a fiddle file. It's 69-lines of code. https://gist.github.com/steveseguin/15bba03d1993c88d0bd849f7749ea625 https://jsfiddle.net/steveseguin/0t3ayvk8/31/

• Fixed the &margin function; correctly adds padding around videos now, i think. :/ well, I fixed an issue with it anyways.

• Added &rounded as another stylistic function, which rounds the edges of videos, but it currently only works if the window fits the video's aspect ratio correctly, so its a bit half-assed right now.

• Not sure if this change is great UX, but it should make things more secure. The password-entry field for incoming guests uses asterisks now. I just see so many twitch streamers screen sharing when not realizing it, seems like a requirement.

• When a guest shares a screen while in a room, the label of the webcam feed will carry over to the screen share label as well now. This was a requested change; not 100% sure about it, but makes enough sense I guess.

• Fixed an issue where the mini self preview would get hidden or go all awkward in size when using the &broadcast mode.

• Tweaked the font-label sizing a bit. The font won't go super-large if deviating from 16:9 window sizes anymore.

• Fixed an issue where the lower control bar could get 'stuck' to your mouse when moving it too fast; you can't accidentally drag the control bar off screen now, as well.

• Addressed a potentially critical problem ahead of an upcoming Chrome change, where the total number of audio/video elements will be limited to as low as 40 eventually by Chrome. I think I managed to avoid it from impacting rooms with fewer than 40 guests though.

  • Patched the Electron App so it won't be impacted by this Chromium change regardless

  • Reduced the memory utilization of VDO.NInja with larger rooms; a side effect of what Google forced me to do. (reduced use of video elements)

• Added VMix, "dummy" camera, and a couple other virtual cameras to the list of input sources to not mirror the self-preview.

  • Most camera types will mirror the camera by default, but virtual cameras I try to not mirror in the preview.

• Fixed an issue with the digital greenscreen effect slowing down after changing the camera a few times; there were some lingering effect processes that were not being cleaned up after a camera change.

  • Fixed an issue where on android, using the digital video effect caused some android devices problems with changing cameras.

• Reworked the stats bar; breaking things down into details. Outbound connections, audio out, video out, scenes connected, total upload bitrate, and if you're overloaded, it will say CPU OVERLOADED.

• Removed the option to select digital video filters, unless &effects is added to the URL.

• Spatial group-chat audio added via the &panning parameter (&pan).

  • You can pass a value between 0 to 180, of -1 for a random value.

  • What it does, is as a viewer, pans the audio to have a spatial left/right direction to it, based on the value passed.

  • &panning=-1 sets the pan to be random; otherwise its just center-stereo if left empty.

  • Setting panning to random easily allows you to create a spatial 'audience' effect, where everyone seems to be located someone specifically, making it easier to tell who is talking.

  • Added panning controls to the IFRAME API sandbox app, demoing how you can programmatically set it to dynamic control the effect.

• Mostly fixed a display name issue when highlighting guests; the font-size is a bit more appropriate in more cases now.

• Improved the digital video effects/filters so they work more often when the Chrome tab is running in the background. (applies to effects 4, 5, and 6)

• Added a 3D face effect; I still need to handle 'clean up', in particular when switching between different filters/effects, but the core functionality has been added. THREE.js-based. &effects=11 is the primary new effect, which is an anonymous face mask. Many more are coming, although this release is largely just a proof of concept; you should be able to add your own effects at some point.

• Added beta support to the invite.vdo.ninja link generator; also you can obfuscate view links now.

• I updated the MIDI sample page; made it a bit more like the IFRAME sample page, with test buttons and such.

  • Check it out at https://vdo.ninja/midi

  • Using the MIDI API, you can control VDO.Ninja with MIDI devices, or even a streamdeck.

  • Improved support for guests in the director room has been added as well

  • Added a new command, &midiremote, which is just like if &midi had a child with &midiin. It allows for remote control of the director/guest via a remote computer that's using &midiout. This mainly just avoids the need for a virtual midi driver on the receiving end.

  • Added a few more midi hotkey commands; up to scene 8 is available now.

  • Documentation has been updated and did a Youtube video demoing it all: https://youtu.be/rnZ8HM9FL4I (edited)

• Created a demo web app for the sensor data parameter, along with an example of how to access video data from the Iframe API. Video included on how to use it.

  • Demos how you can access the video data from a VDO.Ninja child iFrame from the parent window. This demos a rather unknown capability of iFrames, which can be quite powerful.

  • https://vdo.ninja/examples/sensors

  • https://www.youtube.com/watch?v=SqbufszHKi4

Panoramic photo taken in real-time using the gyroscopic sensor and video data obtained from VDO.Ninja

• Based on a request, added the stream ID to the remote monitor stats page.

  • This tool lets you remotely track the video quality of video streams in a graphical way.

  • Previous details on this feature can be found here: https://docs.vdo.ninja/release-notes/v17-release-notes#remote-monitoring-support-added

Electron Capture app

The Electron Capture app is made by Steve and is a downloadable app that functions as a replacement for the OBS browser source and Chrome. It's designed specifically for VDO.Ninja and unlike the OBS browser source, Electron Capture does not suffer from stream-killing audio-bugs or video corruption issues caused by packet loss. Links and details can be found here: https://github.com/steveseguin/electroncapture/

• Added a simple auto ad-skipper for youtube to electron capture.

• Added full-screen to the CLI option list.

• Fixed some fullscreen bugs.

• &quality and &framerate now work with the Electron capture app when screen-sharing.

• As an experimental addition, &q=-1 (negative 1) will allow for screen share to capture the display at the same resolution as the display, rather than being capped at 1080p or whatever. This feature is Electron Capture specific.

• Added the ability for the Electron Capture app to change audio, video, and audio output destinations via the right-click menu, dynamically. Elevated privilege's are needed and only works with VDO.Ninja.

• Added 32-bit Windows systems support also to the portable version (for older systems).

• Added a button for twitch and youtube that full-windows videos, making them ideal for window capture. (not the same as full-screen)

• The right click menu now has an option for muting the electron window. Works with any website.

Companion Ninja (previously OSC.Ninja)

• The https://companion.vdo.ninja service is an API and sample web-app that enables HTTPS / Websocket support to VDO.Ninja, providing Companion-compatible hot key support, all without needing to use a MIDI interface.

• You can send HTTPS GET requests to VDO.Ninja to remotely add guests to scenes, mute mics, and more. This is an alternative to the MIDI and IFRAME API that VDO.Ninja provides.

• Currently the app is the demo and documentation, via the source code, but proper documentation and guides are coming. This is all just the start.. feedback very welcomed.

Remote Control OBS using VDO.Ninja

• Created a proof of concept of being able to remotely control OBS via VDO.Ninja

• Requires OBS to have the OBS Websockets plugin installed (simple/quick to do).

  • https://github.com/Palakis/obs-websocket

• You can then remotely control OBS from this website, as if you were connecting to the Websocket plugin, but you can do so anywhere. You don't need to be on the same computer or same network.

  • https://steveseguin.github.io/remote_ninja/

• The code for the project is available at:

  • https://github.com/steveseguin/remote_ninja

• The provided link is the remote control interface: it works across firewalls, by using VDO.Ninja's IFRAME API to transmit the commands.

• The sample app that is hosted can monitor OBS metrics, go live, change scenes, and even change the audio level of remote sources.

• The option to share the OBS virtual cam is supported, allowing for the remote director to not only control OBS, but see the output in real-time using VDO.Ninja's video streaming.

• This concept expands on the basic websocket plugin, as it works across firewalls without needing to install anything additionally or having to change networking settings.

Login and Password for OBS websocket plugin

Connected to Websocket plugin; leave the window for remote directors to route commands to OBS

The remote director sees this view; options to change scenes, outputs, and settings are available.

Chat Overlay Ninja

• https://chat.overlay.ninja/ was updated in the chrome webstore to support Twitter, Instagram and Facebook live. It already supported Twitch, Youtube, Restream, and others.

• Maintained/supported by Steve, and allows for featured messaging overlays to be added to your streams. Mentioning it as it is a key feature for a lot of users looking to adopt VDO.Ninja, but don't otherwise yet have access to it.

Android native app updated 📱

• A new Android APK build has been created with a version of code updated by @jcalado

• Contains fixes some of the camera issues

• Adds some buttons to toggle the preview of the video on and off

• Still supports screen-sharing on Android

• Often works with VDO.Ninja when Chrome on Android will not

• Adds the ability to cycle cameras with a single button easily

• Added support for more camera types

• Improved visual styles

• Download link: https://drive.google.com/file/d/1M0kv5nWLtcfl2JOnsAGiG1zUmkeIVLyZ/view?usp=sharing (update: see https://docs.vdo.ninja/getting-started/native-mobile-app-versions)

Other minor changes

• The console shows some developer tips to help with common issues.

• Fixed a typo within some of the translation files, regarding a parameter option in the director's room.

• Minor polish; the "copy this URL into your browser" text fits better on small screens now, leaving more room for the video itself.

• Added some text to the 'remote record' button to let the director know it's experimental still; should have a backup record going.

Thank you to all the sponsors, volunteers, and contributors 🥰

A few of the faces and names are listed below; those who have helped report bugs, provided feedback, answered support questions, contributed code, or sponsored the project -- you're all rock stars to me. 😎 Thank you, sincerely,

~ Steve Seguin