Fraunhofer Institute for Mechanics of Materials IWMMaterials engineering
Using deep learning to classify steel materials objectively
Rolling bearings are installed wherever something is in rotation. The wide range of applications extends from large wind turbines to small electric toothbrushes. These bearings, which consist of steel components, must be carefully selected and tested with regard to their quality and the application in question. The grain size has a crucial effect on the mechanical properties of the steel. Up to now, the size of the microscopic crystallites has been assessed by metallographers by way of visual inspection — a subjective and error-prone method. Researchers at the Fraunhofer Institute for Mechanics of Materials IWM, in collaboration with Schaeffler Technologies AG & Co. KG, have developed a deep learning model that enables objective and automated assessment and determination of the grain size.
The material typically used for rolling bearings is surface-hardened steels enriched with carbon. Surface hardening is used to make the material durable so as to prevent failure, fatigue and critical crack growth in the components due to cyclic loading, thereby averting serious accidents. Critical microstructural characteristics in steel alloys are typically non-metallic inclusions and larger-than-average grains. These crystallites are formed during the steel production process and are subject to constant changes throughout the value chain. The mechanical properties of steel are largely determined by the grain size, among other things. Therefore, for quality control purposes, it is essential that they are reliably assessed. Researchers at Fraunhofer IWM in Freiburg, Germany, in collaboration with Schaeffler Technologies AG & Co. KG, have developed a deep learning model for determining the grain size of martensitic and bainitic steels, i.e., steels with a hardened microstructure caused by abrupt cooling. This model is designed to supplement or replace the time-consuming visual inspection carried out by trained metallographers, who, until now, have determined and classified the defects visually. They primarily look for larger grains and other flaws as this is where the likelihood of failure is at its greatest. However, as an interlaboratory round robin test revealed, the grain size evaluations carried out by experts differ from one another. Aside from the grain size classification being inherently subjective, the expert's ratings were shown to be inaccurate at times and thus not sufficiently reliable — especially for safety-relevant applications. The standard inspection procedure is also prone to errors because it is based on small samples and, due to the large amount of work involved, the inspection of the entire component is not feasible.
High degree of automation, improved accuracy and reproducibility in defect detection
The deep learning model for grain size determination, on the other hand, can be used to assess arbitrarily large component areas and exhibits high accuracy and ideal reproducibility. To achieve this, the model was supplied with image data that previously had been classified by experts. These images from the industry partner were used to train the model to recognize and classify microstructures in steel. The innovation here is that the grain size can be assessed in a consistently objective and automated manner. The data are subject to annotation noise due to differences in the way metallographers assess grain sizes. However, when optimizing the model, the noise, i.e., the erroneous assessment, is filtered out. By continuously receiving images annotated with over- and underestimation of grain size, the model learns an average representation and is trained to assess the microstructures more confidently. “In this case, neither exceptionally clean data nor large volumes of data are required for training,” says Dr. Ali Riza Durmaz, a scientist at Fraunhofer IWM. A web application developed by Dr. Durmaz and his team visualizes the results. In the process, explainable artificial intelligence approaches provide greater transparency in the model’s decision-making process.
The smaller the grain size, the stronger the steel
The deep learning model is used to classify microstructure images into different grain size ranges. “The rolling bearings must meet the microstructural requirements, meaning that the grains must not exceed a certain size. The smaller the grain size, the greater the strength of the steel,” explains Durmaz. The higher the number of small grains, the greater the density of grain boundaries, i.e., the contact surfaces between the grains. A high density of grain boundaries prevents plastic deformation of the component even under very high loads. Even if the material were slightly but permanently deformed, the bearing would no longer run smoothly, and the frictional properties would be impaired, as would the energy efficiency.
In addition to grain size, the deep learning model is also able to distinguish between martensitic and bainitic states as well as between different steel alloys (variants of the 100Cr6 and C56 families). The model is currently being implemented in the industrial setting of Schaeffler Technologies. This provides the industry partner with a system that can be used in industrial processes to identify defects in rolling bearings in an AI-based and automated manner with previously unattainable reproducibility. The workflow, which involves adapting the AI model to specific materials, linking it to image processing and embedding the model in user-friendly interfaces, can be easily transferred to other areas of application. “Our deep learning model paves the way for AI-based and automated qualification, for example, in any situation where safety-critical components are subjected to high and cyclic loads, such as electric drive components or the B-pillar in vehicles,” Durmaz concludes.