Detection
Boards are found as pixel-level masks, not boxes. Stadiums, leagues and camera setups all look different, so the models are tuned on real broadcast footage.
Advertising board replacement in sports video, built on computer vision. It works from the video itself: no camera tracking hardware.
The proof of concept is the result of over 400 code versions and approaches.
Every broadcast, stadium and lighting condition brought its own problems: moving cameras, floodlights, compression, players in front of the boards.
Boards are found as pixel-level masks, not boxes. Stadiums, leagues and camera setups all look different, so the models are tuned on real broadcast footage.
Detection alone is not enough. Each board is followed over time, keeping the perspective correct through fast camera moves and scene cuts.
People, balls and other moving objects are detected too, so that they stay in front of the replaced board.
Compression, floodlights, fast cameras and crowded frames are the normal case, not the exception. The pipeline is built around them.
Detection, tracking and masking were each tuned on real broadcast frames. Many attempts failed, and the pipeline is what survived.
Methods that work on clean test clips often break on compressed broadcast video with fast cameras and crowded frames.
Formats, resolutions and camera angles vary. So far it has been validated mainly on football.
More data and speed: real time is the goal, and other sports need to be validated.
The proof of concept works on real broadcast footage. The next step is testing it with partners on their own footage and getting it to real-time speed.