Toward a Subjective Assessment System for Closed Captioning Quality
Somang Nam, Deborah Fels, Mark Chignell
A novel quality assessment system design for closed captioning (CC) is proposed. CC was originally designed to serve hearing-impaired audiences. Traditional quality assessment models focus on empirical methods only, measuring quantitative accuracy by counting the number of word errors in the captions for a segment. However, hearing-impaired audiences have been outspoken about their dissatisfaction with the quality of current CC. One solution to this problem may involve inviting human evaluators, who represent different groups, to assess the quality of CC at the end of each segment, but in reality, this may be difficult to do and very labor-intensive. To solve this challenge, we propose the training and use of an artificial intelligence system to predict subjective CC quality based on human assessment data. In this paper, the design and development process, to be used in developing this system, is described.