University of Bristol

Evaluating Video Codecs Through Objective and Subjective Assessments

We have evaluated the performance of two state-of-the-art video codecs, High Efficiency Video Coding (HEVC) Test Model (HM) and AOMedia Video 1 (AV1) through objective and subjective quality assessments. Nine source sequences were carefully selected to offer both diversity and representativeness, and their different resolution versions were encoded by both codecs towards pre-defined target bit rates. The compression efficiency and computational complexity of both codecs were first evaluated through objective quality assessment and further compared with those of another two codecs, Versatile Video Coding (VVC) Test Model (VTM) and Google’s VP9. The subjective quality of HM and AV1 compressed content was then tested by conducting psychophysical experiments. Moreover, the Dynamic Optimizer method was also employed to compare both codecs across different resolutions and wide bit rate range. The selected source sequences, compressed video content and associated subjective data have now been made available online, offering a valuable resource for compression performance evaluation and objective video quality assessment.

Parts of this work have been presented in the IEEE International Conference on Image Processing (ICIP) 2019 in Taipei and in the Alliance for Open Media (AOM) Symposium 2019 in San Francisco.

The sequences used for the codec comparison and the subjective evaluation can be downloaded here.

Papers to reference:

[1] A. V. Katsenou, F. Zhang, M. Afonso and D. R. Bull, “A Subjective Comparison of AV1 and HEVC for Adaptive Video Streaming,” 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 2019, pp. 4145-4149.

[2] F. Zhang, A. V. Katsenou, M. Afonso, and D. R. Bull, “Evaluating Video Codecs Through Objective and Subjective Assessments” (under preparation)