Before choosing a metric, you should understand what the scores tell you, and make sure
it tells you what you need to know.
Accessing the Metric
Don’t choose a metric without understanding
how you’ll access it and how much it will cost to
do so. In this section, I’ll briefly discuss the tools
that can compute the metrics above, starting
with FFmpeg, a free tool that can compute both
PSRN and SSIM.
The Moscow State University (MSU) Vid-
eo Quality Metric Tool (VQMT, $999 direct)
supports a range of metrics like PSNR, SSIM,
MS SSIM, and many others, including VMAF in
version 10, which is now in beta. The top win-
dow in Figure 3 (on page 48) shows the VMAF
score for two 1080p talking head files, one en-
coded at 4500Kbps, the other at 8500Kbps, with
the top graph showing the entire file, the low-
er graph the highlight region on the left in the
upper graph. The scores are very close, indi-
cating that additional 4Mbps spent on the high-
est-quality stream is a waste.
You can drag the playhead and visualize any
frame in the video, either side-by-side, as shown
in the bottom of Figure 3, or one atop the other.
This latter view makes it simple to switch between the two encoded files and the original,
which is better for visualizing minor differences like the color shift mentioned above. VQMT
offers perhaps the best interface of any tool for
making A/B comparisons between two encoded files (Figure 3), and its batch operation is
On the downside, VQMT can only compare
files of identical resolution, so if you’re analyzing lower-resolution rungs on your encoding ladder, you’ll have to manually scale them
to full resolution first, which takes time and lots
of hard disk space. In the beta, the implementation of VMAF is painfully slow, literally using