Télécom SudParis and elm.leblanc: the keys to a successful partnership

The theme of the industry of the future, and in particular the application of augmented reality, has been a Télécom SudParis research focus since 2016. Manufacturers such as elm.leblanc (Bosch Group), meanwhile, have been following a similar line of thought and have identified augmented reality as a driver of performance. Télécom SudParis and elm.leblanc therefore entered a partnership in 2017. A few years later, and the results are there. The expertise of academic researchers combined with that of industrial stakeholders has helped move innovation forward. This is a public research and industry partnership, supported by the Carnot label, one of the schemes in the State’s Future Investments program.

 The demonstrator: a first step

“In order to take up this great challenge at Télécom SudParis, we started by identifying researchers whose work was close to this field and encouraging them to move towards the field of the industry of the future”, recalls Olivier Martinot, Director of Innovation and Corporate Relations.

At the end of 2017, Télécom SudParis allocated part of its Carnot funding to setting up the first augmented reality demonstrator for training and maintenance in industry. The demonstrator was developed through the Télécom Sud Paris MyMultimediaWorld platform headed by Marius Preda, a lecturer-researcher in the ARTEMIS Department which specializes in image processing technologies.

This demonstrator is an essential step towards starting meaningful discussions with manufacturers. By showing the precision of the match between virtual images and real environment, one of the major issues of virtual reality, the demonstrator showed all the potential of augmented reality when applied to industry. A research project was therefore set in motion in partnership between Institut Mines-Télécom and elm.leblanc.

 

 

The aim was to study the conditions for the success of this transformation, drawing on the academic skills of the laboratories of Télécom SudParis and IMT Atlantique and their ability to collaborate with industry.

At the end of 2017, Télécom SudParis also gave its support to a start-up, Spectral TMS, housed in its incubator. Marius Preda helped with the production of the first prototypes. The start-up is now working with industrial partners such as Safran, an example that shows that Télécom SudParis is capable of working with both start-ups and large industrial companies

 

The challenge of elm.leblanc: an uncertain environment

As a leader in the heating industry, elm.leblanc must constantly adapt to a complex and uncertain environment by increasing the flexibility of its production. This aim can be met by making production line operators more versatile and by turning them into key players in the transformation of the industry. This means learning their work more effectively from their workstation.

“For example, our sales are immediately impacted by regulatory changes, such as the ban on fuel,” explains Emmanuel Bricard, IT Director at elm.leblanc.

“Similarly, responding to the market means gradually moving towards "customized" production, with small production runs or even one-off items. In addition, we must adapt to the seasonality of sales and resulting changes in the workload. We do this by hiring staff on short-term contracts during the heating season, but this requires new staff to be trained in our assembly processes in a very short space of time. The challenge is to limit the risks of disorganization that could result from our experts being taken up by supervising the training.”

 

The contribution of technology

“Our products are increasingly digital to improve the daily lives of our customers by offering them more practical applications”, says Emmanuel Bricard. “We also believe that technology provides some answers to the industrial challenges we are facing”.

Bosch and elm.leblanc are thus committed to the industry of the future and their aim is to develop “agile” industries capable of adapting their production to demand. This could help maintain the presence of industrial sites in countries such as France. This approach involves planning investments and developing staff skills to make them more versatile. To do so, it is essential that the training should become more efficient and reduce its impact on production.

 

The challenges of training

The aim of the Iron Men Project is to train operators more quickly and efficiently, directly at their workstations. Mixed reality became widely available on the market with Hololens headsets in 2016, and elm.leblanc identified this as a possible training method.

By equipping operators with augmented reality headsets, it could be possible to send them instructions in real time in the form of holographic projections. This would allow them to carry out their work and interact with their colleagues while getting the feedback on their work they would normally get from a supervisor. This tool should help develop the versatility of staff, which is a key factor for competitiveness in industry.

 

A structuring framework

The framework agreement between the management of IMT and elm.leblanc led to a research partnership in Industry 4.0. and more particularly in augmented reality (AR).

At Télécom SudParis, the team led by Marius Preda had the skills to apply research methods to the study of these problems.

The Iron Men Project was thus born with a consortium comprising IMT, elm.leblanc and companies specializing in AR. As a winner of the Digital Challenges call for projects funded by the State as part of the Carnot Future Investments program, in the “Augmented Reality/Mixed Reality” category, Iron men started in 2018 for a planned period of 3 years.

It is a large-scale and long-term strategic business partnership (PSE). To date, it is the most ambitious industrial partnership for Carnot and involves several Cifre theses by PhD students hired by elm.leblanc and supervised academically by scientists from partner laboratories.

“The PhD students divide their time equally between the factory and the laboratory. When they are in the factory, they are close to the production line. In the laboratory, we analyze what they have discovered in the factory, we review the actions of the operator,” explains Marius Preda. “Partnerships are the best way to work together”.

A complementary fit between research and industry

For elm.leblanc, the roll-out of AR first required an overview of this new area for which it did not have in-house skills. Since the creation of its innovation unit seven years ago, elm.leblanc has been calling upon PhD students to work on various topics, such as hybrid boilers or the Internet of Things.

“Despite the cultural differences that exist between the worlds of public research and industry, we are convinced that their complementary fit can enable us to move very quickly on new topics,” says Emmanuel Bricard.

Designed in preparation for the roll-out of mixed reality at elm.leblanc, the Iron Men Project is unprecedented in its scope, with three theses being carried out in the laboratories of Télécom SudParis on aspects surrounding augmented reality applied to the industry and two at IMT Atlantique on connected lenses. A sixth thesis is even being envisaged to redesign the production line, once Iron Men has delivered its results.

“Our contacts combine academic skills with an experience of industrial realities, which is very valuable,” says Emmanuel Bricard.

Skills research

The aim is therefore to create methods and tools which facilitate and accelerate learning of the work on assembly processes, without the intervention of an expert.

“Surprisingly, I discovered through elm.leblanc, that Industry 4.0 is not only based on increased robotics, but on the contrary, that human workers remain our main priority,” says Marius Preda.

“As there is a need to produce many models of boilers in small production runs, operators are considered as the best fit to make the production process more agile, provided they are well trained.”

The challenge is really to design effective "AR experiences" upstream, in a context of real-time operator guidance. The key is presenting the right information at the right time, in the right place and in the right way depending on the operator's skills. In other words, transforming information and contextualizing it by adapting it to each operator.

 

 

“One of the theses will look at how to evaluate the different modalities used in AR to interact with the environment: voice, gestures, etc.” sums up Thierry Duval, professor and expert in interactions and 3D mixed reality at IMT Atlantique. “Another question is how well AR headsets will be accepted: how long can they be worn during the day? What are the effects on the human visual system and eye movement between the real and the virtual?”.

The Cifre theses are working on real case scenarios and have a precise aim.

“In our, case,” says Thierry Duval, “elm.leblanc wants to have a tool that can be used on a production line within four or five years.”

As assembly lines are constantly being reconfigured, it is also necessary to have the tools to facilitate the modelling of the workstation in order to maintain the coherence between the real and virtual worlds. This is why the topics covered by the theses include evaluating methods of viewing 3D images in different assembly scenarios, studying the simplification of data to allow their visualization in real time, the development of 3D semantic models aimed at improving the understanding of the environment from the images captured by AR headsets.

Putting it all to use

The various modules associated with the features described above (movement and environmental recognition, workstation modelling, modelling of work instructions in mixed reality) will be integrated into tools that meet the business needs of elm.leblanc and will be used at its Drancy and Saint Thegonnec factories.

Thanks to a design based on the existing standards, Iron Men's results can be transferred to other production lines. This transfer is an integral part of the IMT's mission and the culture of Télécom SudParis, and “also stimulates academic work by providing concrete problems to solve” according to Marius Preda.

“Applying the work to an industrial environment has significant benefits for scientific publications.”

The synergy at the heart of the Iron Men project reflects its ambition to contribute to the development of the industry of the future.