Jakarto called on CRIM’s experts to help develop an automatic recognition system for objects such as street furniture in order to build detailed maps of cities and locate certain objects with a high degree of accuracy.
Your neighborhood recreated on the screen, automatically?
Everyone knows Google StreetView, a system based on a car that travels the streets of the planet and takes numerous photos that are then assembled to allow the user to “walk” virtually in the city of their choice, via a visualization platform.
The Jakarto company wants to take the concept of virtual modeling of urban space much further. Their platform aims to make the process of inventorying or locating urban equipment more efficient. This innovation would greatly facilitate the work of employees responsible for maintenance, urban planning and purchasing, among others.
Jakarto has therefore developed a very sophisticated data collection system that includes 5 ultra high definition (4K) video cameras mounted on a car, which film continuously. In addition, an extremely precise geolocation module is installed in the car: it allows you to know the vehicle’s position on the Earth’s surface to the nearest inch! Two LIDAR (light detection and ranging) systems complete the vehicle’s equipment: they accumulate data during each trip and use it to build a 3D model of the city or neighborhood traveled through. LIDAR is a technology used by autonomous cars to interpret their physical environment, among other things.
Jakarto’s idea is to combine high-resolution 2D images, geolocalization and 3D modeling to create a virtual model of the studied area that each type of image (2D or 3D) helps to enrich and enhance. However, to link a point in the 3D model to a point in a 2D image, it is necessary to know the exact position and orientation of each camera on the vehicle, their positioning in relation to the LIDAR, etc. CRIM experts had to exploit this mapping so that each object in the 3D model would be linked to its equivalent in the 2D images.
Two central goals
CRIM’s mandate in this project was twofold: first, use machine learning to train a model able to detect certain objects in the data: street furniture, benches, street lamps, fountains, street names, road signs, etc. To do this, the CRIM team used existing object detection models and modified them in order to train them to detect those specific objects that will be useful to clients of Jakarto. It then worked to properly integrate this detection module into the company’s data processing chain.
A cutting-edge tool for towns and cities
Producing an all-in-one solution for visualizing the virtual model of a city or area is an innovation that could prove very useful for city workers. The first model of the system was built using data collected in the Greater Montréal area, but this technology could of course be of use to many regions!