Smooth Video (Experimental)

Warning

This is an experimental feature.

This feature allows video playback to be perfectly smooth by not dropping or duplicating frames and presenting frames at the same rate at the display refresh rate.

This is particularly important for very large video walls where a single glitch in playback can be very noticable.

Limitations

  1. This feature only works on Windows and uses the Media Foundation API.
  2. Hap/NotchLC codecs are not supported

Setup

To use this feature:

  1. Add the text AVPROVIDEO_SUPPORT_BUFFERED_DISPLAY to Edit > Project Settings > Player > Scripting Define Symbols. You can also enable Developer Mode in the Global Section of the MediaPlayer component, and then tick the Support Buffered Display option.

  2. Now a new Frame Selection option will be available in the MediaPlayer component . Set this to Elapsed Time Vsynced.

    alt_text

  3. Ensure that vsync is enabled in your application (Edit > Project Settings > Quality). In the Unity editor make sure it's also enabled in the Game View.

  4. Ensure that your display vsync rate is a perfect multiple of your media frame rate. For example a display that is 60 FPS is fine for media that is 60 or 30 FPS.
  5. For best results your video shouldn't contain audio, but if they do then the audio may slightly ahead of the video. In this case you can enable the option Use Audio Delay to counteract this.
  6. Don't run any scripts etc that delay the main thread, otherwise sync may be lost
Note

In the Unity editor it is difficult to achieve perfect playback. For best results play using a standalone IL2CPP build (ideally with full-screen mode set to exclusive).

How it works

  1. When the video loads it will being playing internally (pre-roll) and buffer those frames.
  2. Once pre-roll is complete the video can begin playback.
  3. Scripted logic selects which frame to display from the collection of buffered frames, based on vsync rate, Unity dropping frames etc.
  4. Meanwhile the buffers are continued to be filled up ahead of what is being displayed
  5. As long as the buffers are used and filled at the same rate, smooth playback should continue.

Debugging

We added some monitors to help understand how the buffers are logic are performing:

alt_text

The Buffers section shows the health of the buffers. For best results Buffered Frames should be at least 4 and Free Frames should be at least 3.

It can be difficult to spot dropped or duplicated frames, so the Presentation Quality section tries to detect these and give you an overview on the smoothness.