Encoding Notes
Correct video encoding is important to get the best results for different use cases. Mostly videos are encoded for the typical case: forward playback with support for slow random seeking. Alternatively you may want to be able to seek through the video quickly, so you would need to encode with different parameters for that use case.
There is a wide range video encoding software but we shall focus on the command-line tool FFMPEG as it is very mature and flexible, however it should be possible to apply the same concepts with others encoders.
Resources
FMPEG Basics
Below is a basic FFMPEG command. This will convert input.mov to output.mp4. The video track will be compressed using the H.264 video codec (libx264 is the name of this encoder), with frames encoded in the 4:2:0 format which is the most common. Since no other options are specified it will use some default settings for the encoder and based on the input video properties. Any audio in the source video will be compressed with the default audio compression for MP4 which is AAC.
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -y output.mp4
You can add many parameters to control how videos are encoded. These paramters will limit the output video to 4 seconds, remove the audio track, and change the frame rate to 30 FPS, the size to 1280x702 and with a quality factor of 14 which is quite high quality and will result in a slightly larger file.
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -t 4 -an -r 30 -s 1280x720 -crf 14 -y output.mp4
Windows Batch Files
You can create .bat files with FFMPEG commands so that videos can be processed by dragging the files into the batch files.
Using %1 for the input and output file names, this will become the path of the source file:
ffmpeg -i %1 -c:v libx264 -pix_fmt yuv420p -y %1-output.mp4
Fast Seeking
For a use case of fast seeking/scrubbing through the video, it's important to simplify the encoding so that it favours fast decoding over small file size. This is usually done by increasing the number of key frames and disabling some of the more complex encoding settings (eg B-frames).
Convert a video to H.264 with only key-frames and tuned to decode quickly at the expense of file size:
ffmpeg -i input.mov -pix_fmt yuv420p -c:v libx264 -crf 18 -tune fastdecode -x264-params "keyint=1" output.mp4
Convert a video to HEVC with only key-frames and tuned to decode quickly at the expense of file size:
ffmpeg -i input.mov -pix_fmt yuv420p -c:v libx265 -crf 18 -tune fastdecode -g 1 output.mp4
See the section about seeking for further information.
Transparency Packing
Convert a video with alpha channel (eg using ProRes4444) to a left-right alpha packing video:
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -vf "split [a], pad=iw*2:ih [b], [a] alphaextract, [b] overlay=w" -y output.mp4
Convert a video with alpha channel (eg using ProRes4444) to a top-bottom alpha packing video:
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -vf "split [a], pad=iw:ih*2 [b], [a] alphaextract, [b] overlay=0:h" -y output.mp4
Stereo Packing
Convert two videos, each representing one eye into a packed left-right stereo video (without stereo metadata):
ffmpeg -i left.mp4 -c:v libx264 -pix_fmt yuv420p -vf "[in] pad=2*iw:ih [left]; movie=right.mp4 [right];[left][right] overlay=main_w/2:0 [out]" output.mp4
H.265 / HEVC
Convert a video to HEVC (note that the -tag:v hvc1 is specified here for compatiblity with Apple platforms):
ffmpeg -i input.mov -pix_fmt yuv420p -crf 18 -vcodec libx265 -tag:v hvc1 -movflags faststart output.mp4
Frame Rate
The frame rate of your video should ideal match the refresh rate of your target display device. If that's not possible then it shoudl be a multiple of the display device rate. For example if you intend to display on a 60hz screen, then ideally your video frame rate should be 60 or 30. This will result in the smoothest playback.
VR / 360 / High Resolution Video
When encoding high resolution video (for example for 360 VR) you should usually consult the developer documentation for the target platform / device. Usually though a codec like VP9 or HEVC (H.265) is required instead of the standard H.264 as HEVC supports higher resolutions.
With FFMPEG to use HEVC codec:
ffmpeg -i input.mov -pix_fmt yuv420p -crf 18 -vcodec libx265 -tag:v hvc1 -movflags faststart output.mp4
or the VP9 codec:
ffmpeg -i input.mov -pix_fmt yuv420p -crf 18 -libvpx-vp9 -movflags faststart output.mp4
Colour-space Profiles
When encoding videos there are different standard for colour encoding. They can broken up into 4 main properties with these common values:
- Range
-color_range tv(limited range)-color_range pc(full range)
- Colour transfer
-color_trc bt709-color_trc bt2020-10(for 10-bit)-color_trc smpte2084(PQ)-color_trc arib-std-b67(HLG)
- Colorspace
-colorspace bt709-colorspace bt2020nc
- Colour primaries
-color_primaries bt709-color_primaries bt2020
The most common encoding for videos is to use the Rec709 standard. Convert a video to 10-Bit HEVC using Rec709 standard with full range:
ffmpeg -i input.mov -pix_fmt yuv420p10le -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range pc -crf 18 -vcodec libx265 -tag:v hvc1 -movflags +write_colr -movflags faststart output.mp4
10-Bit Videos
Convert a video to 10-Bit HEVC using Rec709 standard with full range:
ffmpeg -i input.mov -pix_fmt yuv420p10le -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 2 -crf 18 -vcodec libx265 -tag:v hvc1 -movflags +write_colr -movflags +faststart output.mp4
Quality and File Size - Constant Rate Factor
The constant rate factor will try to give you a target quality without caring about the final file size or bit rate. You can control quality using the -crf option where the lower the number [1..51] the higher the quality is. Also the -preset can be set to the slowest preset you can handle to get the best quality
Convert a video to HEVC with high quality and long encoding time:
ffmpeg -i input.mov -pix_fmt yuv420p -crf 8 -vcodec libx265 -tag:v hvc1 -preset slower output.mp4
Convert a video to H.264 with normal quality and short encoding time:
ffmpeg -i input.mov -pix_fmt yuv420p -crf 28 -vcodec libx264 -preset ultrafast output.mp4
Quality and File Size - Variable Bit Rates
If you have a target file size, then you can use 2-pass encoding to achieve the best quality for that size. First you need to caculate the maximum bit rate for your target file size:
BitRateInMbps = (TargetFileSizeInMegaBytes x 8) / VideoDurationInSeconds
You can then pass this bit rate into the -b:v parameter:
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -b:v 3.5M -pass 1 -an -f mp4 NUL
ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -b:v 3.5M -pass 2 -c:a aac output.mp4
Progressive MP4 Streaming
A video file can be prepared for streaming by applying the 'fast start' option to an existing video:
ffmpeg -i input.mp4 -acodec copy -vcodec copy -movflags faststart output.mp4
Hap Codec
FFMPEG doesn't support all the Hap format variants, but you can encode these types:
Convert video to basic Hap:
ffmpeg -i input.mov -vcodec hap -format hap output.mov
Convert video to basic Hap with alpha channel:
ffmpeg -i input.mov -vcodec hap -format hap_alpha output.mov
Convert video to basic HapQ for higher quality:
ffmpeg -i input.mov -vcodec hap -format hap_q output.mov
Convert video to basic HapQ for higher quality with 8 chunks for better multi-threaded decoding, and force using the snappy compressor for smaller file size:
ffmpeg -i input.mov -vcodec hap -format hap_q -chunks 8 -compressor snappy output.mov
For encoding other newer Hap formats you can find the plugins online including Jokyo, Original Quicktime plugin, Original AV Foundation tool.
Oculus Quest / Go
The Oculus Quest / Go have special encoding requirements which are detailed in this Blog Post.
Facebook Audio 360
Typically the Facebook Audio 360 Sparial Workstation would be used to create compatible MKV files with Opus audio. But in some cases ambisonic encoded videos can be converted using FFMPEG.
Convert a video with ambisonic audio (1st, 2nd or 3rd order) into a MKV file suitable for Facebook Audio 360 decoding and copying the existing video tracks:
ffmpeg -y -i input.mov -c:v copy -sample_fmt s16 -acodec libopus -mapping_family 255 output.mkv
Removing Audio
If audio is not needed, it's best to strip it out using the -an option:
ffmpeg -i input.mp4 -an -vcodec copy output.mp4
Remove Rotation
Rotation metadata can be removed:
ffmpeg -i input.mp4 -metadata:s:v:0 rotate=0 output.mp4