3D screens (displays) are often associated with immersive experiences that can be found in film or video games. They are also used in professional disciplines such as geomatics [1]. Studies [2] show that looking at a 3D object improves retention and understanding of the information being transmitted.
Looking at the array of activities in geomatics, it’s easy to understand why 3D technology is an enabler: it can help visualize urban space and guide decisions relative to infrastructure, investments, etc.
Did you know that geomatics also comprises:
– Emergency vehicle tracking
– Geomarketing for better targeting and client mapping
– Monitoring of flood risks during spring
– Augmented reality on smartphones and tablets
– Monitoring forest fires with satellite imaging
– Navigation aid on the St. Lawrence River
– Online travel management
– Roadwork planning
– Real estate sales
– Development of new residential boroughs
Box 1. Diversity of geomatic activities [3]
The main constraint behind this technology is the need to wear 3D glasses. Urban planners working with two screens – one displaying data, the other 3D objects – must constantly put on and remove 3D eyewear to switch between screens.
Until now. A Montreal-based company, AYE3D, changed the rules of the game by introducing FRE3DOM, the glasses-free 3D screen which is the result of years of software, optical and hardware development.
Its inventor, Jean-Etienne Gaudreau, now CTO at AYE3D, filed his first 3D-screen patent in 1994. It took another 20 years for the glasses-free screen to see the day in 2015.
AYE3D is the only company in Canada to market glasses-free 3D vision and possibly the only one in the world to offer such a resolution.
The AYE3D team, from left to right:
Mario Genest, CEO, Jean-Etienne Gaudreau, CTO, Guy Lussier, Product development Director.
In the glasses-free 3D screen market, we expect a 260% growth by 2025, which represents $206.6M USD in revenue [4].
As AYE3D always strives to stay one step ahead, they called upon CRIM to explore new possibilities for their technology using deep learning.
How does it work?
FRE3DOM has two cameras (see image below): the in-depth camera that assesses the distance between eyes and screen, and a second camera for facial recognition. Together, they detect the whole of the face and adapt the 3D image perspective to fit the user’s facial position.
Image: FRE3DOM cameras positioned at the top of the screen. Photo taken in front of glasses-free screen at AYE3D.
FRE3DOM Potential Users
Their first clients are geomatic experts, many of whom are in the process of replacing their obsolete equipment with these new user-friendly screens.
“By removing the repetitive task of putting and removing 3D glasses, human error decreases. Managers with whom we have spoken believe that glasses-free screens will help increase productivity by 10 to 20%.”
We could easily imagine other disciplines that can benefit from FRE3DOM: medicine, education, engineering, architecture, etc. In fact, a great number of organisations, here and across the world, are showing interest in FRE3DOM.
Let’s look at the medical sector. It is a lucrative one with an estimated market value of $500 million CDN [5] for 3D medical imagery. FRE3DOM offers a high-res view of the heart and lungs. Doctors can move the organ in every direction with a circular mouse and hence visualize with precision every artery and every vein, for instance.
Allowing the patient to see what the surgery entails is reassuring and conducive to a speedier recovery.
— Mario Genest, CEO of AYE3D.
Moreover, this screen can be useful during virtual dissections or for surgery planning which entails assessing its duration and risks, and practicing on-screen, all activities that a CT-scan alone cannot provide.
Image: 3D view of heart and lungs. Photo taken in front of a glasses-free screen at AYE3D.
FRE3DOM can make a significant impact in the construction sector as well. With the “digital twin” of a ship taken from the 3D collection at the Scottish Maritime Museum. (see below), the glasses-free screen helps teach best maintenance practices, provide training and guide engineers in building the vessel.
Image: 3D view of a ship taken from the collection at the Scottish Maritime Museum.
The Risk Mitigator
FRE3DOM brings precision and depth that reduce the risk of producing flawed products. It can be used for:
- design of electronic cards: the 3D screen will make it possible to see whether components and welded parts comply with spacing norms, reducing risk of error prior to starting production.
- aerospace: the 3D visualization of a machine or a part of an aircraft to verify its dimensions one final time before cutting.
- 3D printing: visualization to check the design before printing the part, which saves time and minimizes material waste.
Glasses-free 3D and Deep Learning
Innovation is a full-time activity at AYE3D! Currently, they are looking to increase the performance of their camera set and improve facial recognition for specific contexts, such as wearing a mask. They solicited the experts at CRIM on the choice of deep learning algorithms that will get them the best results in this specific context.
AYE3D is very interested in pursuing further innovation with CRIM. Their next step will be to analyze every other element in addition to performance, that could influence the choice of DL algorithm, such as power consumption and execution timeline.
Their main goal can be summed up in one word: flexibility. Flexibility to adapt FRE3DOM to new use cases that have yet to be discovered. One thing is for certain: the future looks bright with AYE3D and CRIM is proud to play a part!
This project was made possible thanks to the NRC Industrial Research Assistance Program (NRC IRAP).
