BVI-UGC: A Training Database for UGC Video Quality Assessment and Video Coding

About

In recent years, user-generated content (UGC) has become one of the major video types consumed via streaming networks. Numerous research contributions have focused on assessing its visual quality through subjective tests and objective modeling. In most cases, objective assessments are based on a no-reference scenario, where the corresponding reference content is assumed not to be available. However, full-reference video quality assessment is also important for UGC in the delivery pipeline, particularly associated with the video transcoding process.

In this context, we present a new UGC video quality database, BVI-UGC, for user-generated content transcoding, which contains 60 (non-pristine) reference videos and 1,080 test sequences. In this work, we simulated the creation of non-pristine reference sequences (with a wide range of compression distortions), typical of content uploaded to UGC platforms for transcoding. A comprehensive crowdsourced subjective study was then conducted involving more than 3,500 human participants. Based on this collected subjective data, we benchmarked the performance of 10 full-reference and 11 no-reference quality metrics. Our results demonstrate the poor performance (SROCC values are lower than 0.6) of these metrics in predicting the perceptual quality of UGC in two different scenarios (with or without a reference).

Useful Links

[LINK] sign form to get access to the data

Sample frames from BVI-UGC

In order to develop a database with diverse and representative source content, we defined 15 typical UGC categories following the scope of existing UGC databases and the genres on popular UGC platforms (e.g. YouTube, Tiktok, Flickr etc.).

Example frames in selected videos from 15 UGC catogories defined in this database. Every category contains both landscape and portrait videos (the ratio is approximately 2:1).

Citation

@article{qi2024bvi,
  title={{BVI-UGC}: A Video Quality Database for User-Generated Content Transcoding},
  author={Qi, Zihao and Feng, Chen and Zhang, Fan and Xu, Xiaozhong and Liu, Shan and Bull, David},
  journal={arXiv preprint arXiv:2408.07171},
  year={2024}
}[paper]