Automatic closed-captioning

Prototype for automatic closed-captioning of news broadcasts and public interest programs  [2002]

Summary: This project aimed at adapting CRIM’s speech recognition technology  to the specific area of French Québécois closed-captioning of news broadcasts and public interest programs, for Groupe TVA. During the course of this work, the RQST consulting expert, who is hearing-impaired himself, played an important role towards Groupe TVA by submitting valuable suggestions to the broadcaster. He assisted in the evaluation of the closed-captions generated by the system and tested its user-friendliness on a segment of the deaf and hearing-impaired population.

Sous-titres automatiques en direct. Christian Aubry, @ccéléraTIon, autumn 2002

Sous-titrage en direct. Jacqueline Bousquet, Découvrir, Sept.-Oct. 2002


Captioning live programming for Groupe TVA [2006-09] 

Definition of the client’s problematic

In Quebec, 750 000 people suffer from a hearing disability. Since there was no adequate technology available for close-captioning live broadcasts, deaf and hearing-impaired persons could not have access to recent or breaking news, descriptions of sporting events or any other event broadcast in real-time. 

To remedy this situation, the CRTC (Canadian Radio-television and Telecommunications Commission) instituted a new regulation that compelled the television networks to close-caption their programming.

In order to fulfill the new requirements of the CRTC, Groupe TVA approached us to find out if our speech recognition technology could be used to perform the closed-captioning of live broadcasts. 

Mandate challenge

To be understood, the closed-captions must not contain more of 10% of incorrect words. In addition, the delay between what is spoken and the corresponding transcription that is displayed must be minimal.

Within the context of a noisy environment involving several participants, it is impossible to achieve an error rate of less than 10% for spontaneous speech transcription. Out of vocabulary words pose yet another challenge. It often occurs that words that are unknown to the system (foreign words, regions, etc.) suddenly show up in the current news. These out of vocabulary words produce several transcription errors that must be avoided at all costs.

Solution developed by CRIM

An error rate of less than 10% can be achieved within the context of speech that is read in a controlled environment. In order to reproduce a similar environment, we make use of speech captioners. Their task mainly consists in repeating what the broadcasting interlocutor is saying.

Instead of processing the audio of a possibly unknown interlocutor, located in a potentially noisy environment and ad-libbing his text, we process an acoustic signal coming from a perfectly controlled environment.

To remedy the problem of out of vocabulary words that frequently occur in current news reports, our system crawls news Web sites in order to enrich its vocabulary. As a result, new words that are part of current news events are automatically detected and added to the closed-captioning vocabulary of the system.

Impact on the company

The integration of our software by Groupe TVA allowed its license renewal towards the CRTC. Now TVA is capable of making 100% of its televisual content accessible to deaf and hearing-impaired persons.

CRIM stands out

The system STDirect allows for the closed-captioning of live television broadcasts at a low cost. It automatically adapts itself to current news and is easily extensible to different languages.

Following integration of STDirect at Groupe TVA, a new company, SOVO technologies, was created that uses STDirect to provide a closed-captioning service for:

  • - All the live sports events on RDS, RDS2 and RDS Info Sports
  • - Several programs for Télé-Québec, TVA, Canal Vie, CBC, RDI, TFO, and CPAC
  • - Special events such as the World Cup FIFA 2010, live coverage of the Bastarache commission, or the Vancouver 2010 winter Olympic Games.

SOVO Technologies, created in 2010, is now a company employing nearly eighty people.


Remote, real-time closed-captioning [2002-10]

Result : We have designed a closed-captioning technology aimed at the deaf and hearing-impaired population. A first project begun in 2002 led to the development of  STDirect (Sous-Titrage en DIRECT de bulletins de nouvelles et d'émissions d'intérêt public) that was initially installed in the television studios of  TVA network. The STDirect system has garnered many awards:


  • IWAY 2004 prize, in the “Technologies aimed at supporting persons with disability”, awarded by CANARIE to Pierre Dumouchel, vice-president of scientific operations at CRIM and speech recognition researcher.
  • OCTAS Partenariat stratégique OSBL 2005 prize, awarded by the Fédération de l'informatique du Québec to CRIM,  Groupe TVA and RQST Conseil-expert. 
  • Innovation 2005 prize, in the catagory Partenariat, awarded by ADRIQ to CRIM, Groupe TVA and RQST Conseil-expert. 
  • 2005 CATA Alliance Innovation prize

CRIM began to commercialize its closed-captioning services in 2008. In 2010, SOVO Technologies was created as a result. Several live programs and events are now broadcast with closed-captions provided by STDirect, notably TVA news broadcasts of the Olympic Games, hockey games, FIFA world cup and tennis matches on the RDS network.


Synchronization of text/dialog for post-synchronization and dubbing [2010]

Summary: This project aimed the creation and development of a speech alignment technology and its integration to a computer-assisted post-synchronization and dubbing system for Ryshco Media (now DubSynchro). This firm specializes in dubbing for film and television.

Paroles de stars. Louise Desautels, Découvrir, May-June 2003

Quand la R-D mène une PME sur le chemin de la gloire. Jean Garon, @ccéléraTIon, autumn 2003



Recent news

  • UEAT and CRIM: A Win-Win Collaboration

    CRIM helps UEAT develop an online food delivery suggestion engine based on consumer preferences.


Upcoming event

  • DevOps - Événement Les Affaires
    9 December 2020 9:00
    Présentation en ligne
    Le CRIM est fier d'être partenaire de la 3e édition de la conférence DevOps - Passez de la théorie à la pratique pour une implantation réussie organisée par les Événement Les Affaires. Présentation en ligne.
  • Bravo !
  • Bravo @UeatSolutions ! Grâce à ce partenariat avec le CRIM, des milliers de restaurateurs ont accès à un outil de c…

Recent Publications

  • An end-to-end approach for the verification problem: learning the right distance

  • The Indigenous Languages Technology Project at NRC Canada: an empowerment-oriented approach to developing language software