Towards Automated Perceptual Shot Matching in Motion Pictures

We present an automated shot matching algorithm that considers perceptual effects and scene-specific context based on shot sequence. The algorithm focuses on viewer-centric elements, such as attention to faces or objects and the overall color mood across shots. Using publicly available deep learning models to detect salient image features, we employ a committee-based approach that matches these aspects independently before combining them into a single RGB balance and black level offset per shot. While fully automated, the system allows post hoc manual adjustments, including the strictness of face, object, and mood matching as well as temporal strictness. This transparent, parameterized approach avoids the black-box nature of many deep learning algorithms, providing intuitive, editable results that colorists can easily refine. We include visual examples to illustrate the algorithm's perceptual accuracy, and summarize a study with 5 professional colorists focused on assessing the quality of the match and potential time savings. Comparing manual grading from raw footage versus algorithm-assisted pre-matched footage, results indicate that a match comparable to that of a human colorist can be achieved in 20% less time. The algorithm achieved roughly 70% of the quality of the colorist match, as judged by the participants. The consistent demonstration of significant improvement underscores the value of future long-term studies that evaluate the impact on resource management and grading quality in real-world settings.

Published

2025-10-13

Content type

Original Research

Keywords

color matching, color grading, perceptual shot matching, machine learning

ISBN

978-1-61482-966-9