Automatic closed-captioning


Prototype for automatic closed-captioning of news broadcasts and public interest programs  [2002]

Summary: This project aimed at adapting CRIM’s speech recognition technology  to the specific area of French Québécois closed-captioning of news broadcasts and public interest programs, for Groupe TVA. During the course of this work, the RQST consulting expert, who is hearing-impaired himself, played an important role towards Groupe TVA by submitting valuable suggestions to the broadcaster. He assisted in the evaluation of the closed-captions generated by the system and tested its user-friendliness on a segment of the deaf and hearing-impaired population.

Sous-titres automatiques en direct. Christian Aubry, @ccéléraTIon, autumn 2002

Sous-titrage en direct. Jacqueline Bousquet, Découvrir, Sept.-Oct. 2002

 

Captioning live programming for Groupe TVA [2006-09] 

Definition of the client’s problematic

In Quebec, 750 000 people suffer from a hearing disability. Since there was no adequate technology available for close-captioning live broadcasts, deaf and hearing-impaired persons could not have access to recent or breaking news, descriptions of sporting events or any other event broadcast in real-time. 

To remedy this situation, the CRTC (Canadian Radio-television and Telecommunications Commission) instituted a new regulation that compelled the television networks to close-caption their programming.

In order to fulfill the new requirements of the CRTC, Groupe TVA approached us to find out if our speech recognition technology could be used to perform the closed-captioning of live broadcasts. 

Mandate challenge

To be understood, the closed-captions must not contain more of 10% of incorrect words. In addition, the delay between what is spoken and the corresponding transcription that is displayed must be minimal.

Within the context of a noisy environment involving several participants, it is impossible to achieve an error rate of less than 10% for spontaneous speech transcription. Out of vocabulary words pose yet another challenge. It often occurs that words that are unknown to the system (foreign words, regions, etc.) suddenly show up in the current news. These out of vocabulary words produce several transcription errors that must be avoided at all costs.

Solution developed by CRIM

An error rate of less than 10% can be achieved within the context of speech that is read in a controlled environment. In order to reproduce a similar environment, we make use of speech captioners. Their task mainly consists in repeating what the broadcasting interlocutor is saying.

Instead of processing the audio of a possibly unknown interlocutor, located in a potentially noisy environment and ad-libbing his text, we process an acoustic signal coming from a perfectly controlled environment.

To remedy the problem of out of vocabulary words that frequently occur in current news reports, our system crawls news Web sites in order to enrich its vocabulary. As a result, new words that are part of current news events are automatically detected and added to the closed-captioning vocabulary of the system.

Impact on the company

The integration of our software by Groupe TVA allowed its license renewal towards the CRTC. Now TVA is capable of making 100% of its televisual content accessible to deaf and hearing-impaired persons.

CRIM stands out

The system STDirect allows for the closed-captioning of live television broadcasts at a low cost. It automatically adapts itself to current news and is easily extensible to different languages.

Following integration of STDirect at Groupe TVA, a new company, SOVO technologies, was created that uses STDirect to provide a closed-captioning service for:

  • - All the live sports events on RDS, RDS2 and RDS Info Sports
  • - Several programs for Télé-Québec, TVA, Canal Vie, CBC, RDI, TFO, and CPAC
  • - Special events such as the World Cup FIFA 2010, live coverage of the Bastarache commission, or the Vancouver 2010 winter Olympic Games.

SOVO Technologies, created in 2010, is now a company employing nearly eighty people.

 

Remote, real-time closed-captioning [2002-10]

Result : We have designed a closed-captioning technology aimed at the deaf and hearing-impaired population. A first project begun in 2002 led to the development of  STDirect (Sous-Titrage en DIRECT de bulletins de nouvelles et d'émissions d'intérêt public) that was initially installed in the television studios of  TVA network. The STDirect system has garnered many awards:

 

  • IWAY 2004 prize, in the “Technologies aimed at supporting persons with disability”, awarded by CANARIE to Pierre Dumouchel, vice-president of scientific operations at CRIM and speech recognition researcher.
  • OCTAS Partenariat stratégique OSBL 2005 prize, awarded by the Fédération de l'informatique du Québec to CRIM,  Groupe TVA and RQST Conseil-expert. 
  • Innovation 2005 prize, in the catagory Partenariat, awarded by ADRIQ to CRIM, Groupe TVA and RQST Conseil-expert. 
  • 2005 CATA Alliance Innovation prize

CRIM began to commercialize its closed-captioning services in 2008. In 2010, SOVO Technologies was created as a result. Several live programs and events are now broadcast with closed-captions provided by STDirect, notably TVA news broadcasts of the Olympic Games, hockey games, FIFA world cup and tennis matches on the RDS network.

 

Synchronization of text/dialog for post-synchronization and dubbing [2010]

Summary: This project aimed the creation and development of a speech alignment technology and its integration to a computer-assisted post-synchronization and dubbing system for Ryshco Media (now DubSynchro). This firm specializes in dubbing for film and television.

Paroles de stars. Louise Desautels, Découvrir, May-June 2003

Quand la R-D mène une PME sur le chemin de la gloire. Jean Garon, @ccéléraTIon, autumn 2003

 

Teams

Recent news

  • Vidéodescription - Entrevue avec le CRIM à l'émission Ça me regarde d'AMI-Télé
    22/02/2018

    Claude Chapdelaine, spécialiste en ergonomie cognitive et en interactions humains-ordinateurs au CRIM, était invitée à l’émission Ça me regarde, animée par Christine Rousseau et Kéven Breton et diffusée par AMI-télé.

    +

Upcoming event

  • CEO VISION 4.0 2018
    22 February 2018 15:00
    Hôtel Fairmont Tremblant
    The 18th edition of CEO Vision 4.0 will take place from the 20th to the 22nd
    +
  • #Partenariats collaboratifs: comment obtenir l’aide nécessaire pour développer de l’#IA ? @frlabont1, directeur gén… https://t.co/1p01GXHYFJ
  • Lire le communiqué: https://t.co/V95gb0mF2t

Recent Publications

  • État des lieux des technologies web

    +
  • La science des données pour la prédiction des temps de réponse dans la gestion des services incendies : l'exemple de Montréal

    +