Sensor Enabled Automation, Robotics, and Control Hub (SEARCH) at the Centre for Ultrasonic Engineering (CUE), Strathclyde has been driving the NDE automation for the past decade. In recent years, there has been a notable shift towards adopting automated solutions in the NDE workflows, leveraging advancements in robotics, Artificial Intelligence (AI), and other technologies, recognized as NDE 4.0. This transition aims to redefine the roles of human NDE operators, moving them towards more supervisory positions where they oversee and address specific parts of the process, while automated systems handle the bulk of repetitive tasks. The principal objective is to enhance efficiency while improving the precision and repeatability of the overall NDE pipeline. While significant research has focused on the automation of robotic systems for sensor delivery, data analysis for NDE is predominantly conducted at the operator assistance level and has seen limited uptake of new AI-based tools. Two key reasons for this are a lack of model trust, which includes concerns from both industry users and regulators, particularly in safety-critical processes, and the “black box” nature of AI models where the reasoning behind decisions is obscured. This leads to greater risks in evaluating critical components since faulty predictions by an automated system could result in material failures during use.

Current industry practices in the aerospace sector consist of automated robotic data acquisition, data preparation and preprocessing, and subsequent analysis. NDE inspectors monitor segments of the ultrasonic C-scans (top view images), and if defects or artefacts exceeding allowable size in the industry guidelines are spotted, individual B-scans (cross-sectional images) in the area of interest are further inspected. Lastly, areas of interest are extracted and used for the creation of quality reports. Automated robotic data acquisition for components like small composite wing panels (7 meters span) usually takes around 40 minutes, with data analysis taking a similar amount of time if the component is undefective. However, this step is extended by an additional hour if artefacts and defects are spotted. This additional time is allocated to further inspection of different views of the data, most notably individual B-scans around areas of interest, and the quality report generation process.

As one of the groups leading research in studying strategies for AI integration into the NDE automation workflow, SEARCH has developed a knowledge base for AI technologies best suited for ultrasonic data analysis, defect detection and characterisation at different levels of data structures; time series, images, and volumetric. Among these are supervised object detection for ultrasonic amplitude C-scans, unsupervised anomaly detection on ultrasonic B-scans, and a self-supervised AI model for processing full volumetric ultrasonic data.

Specifically, a Faster Region-based Convolutional Neural Network was used for object detection, trained exclusively on simulated data to mitigate data scarcity issues. Meanwhile, the anomaly detection model, implemented as a convolutional autoencoder, and the self-supervised AI model, designed as a forecasting model for time-series data, were both trained on pristine CFRP samples. These past successful developments in collaboration with Spirit AeroSystems as the industry partner, have created the drive and ambition for the academic/industry team to extend this work into a new phase where AI models can excel at NDE data interpretation, and also provide domain-specific knowledge to guide NDE operators at stages of testing, data acquisition, interpretation, and reporting.

Project Scope

The recent emergence of Large Language Models (LLMs) presents several opportunities to enhance NDE workflows. With the growing availability of open-source LLMs, one promising direction is the development of an NDE-specific model tailored to particular inspection modalities, materials, and industry-specific regulatory guidelines. Such a model could function as a virtual NDE assistant, consolidating domain-specific knowledge to assist with setting up inspection parameters, consulting relevant guidelines, and summarising knowledge of current academic advancements.  Beyond knowledge retrieval, LLMs could also improve documentation processes. By generating quality reports based on results achieved through manual or automated data analysis, LLMs can significantly reduce the workload of NDE operators by improving efficiency in reporting.

The integration of multi-modal LLMs further expands these capabilities by enabling models to analyse time-series and image data alongside textual information. By fine-tuning an agent on these diverse data types, a deeper level of understanding and reasoning can be achieved. Unlike conventional AI models, LLMs can provide explanations for their decisions, improving transparency and addressing the “black box” issue often associated with AI. This transparency would also enable meaningful human-AI interaction, allowing operators to challenge or refine AI-driven assessments when discrepancies occur, ultimately creating more reliable and interpretable inspection processes.  

The proposed PhD will address several primary research areas, with main objectives:

• Investigating how LLMs can be trained and deployed as NDE assistants, capable of interpreting ultrasonic testing data. This research will explore the use of both text and image prompts to guide LLMs in offering suggestions and making informed decisions on the interpretation of various incoming UT data.
• Assessment of Multi-Modal LLMs for Localised Deployment: A comprehensive review of the state-of-the-art in multi-modal LLMs is required to identify models and techniques suitable for localized deployment in NDE applications.
• Exploring how LLMs can be utilised to generate quality reports that adhere to relevant standards and guidelines. These reports will be designed for use in various downstream manufacturing tasks, while bearing in mind the compliance with industry requirements.
• LLM-Guided Inspection Parameter Optimisation and Defect Detection: Conducting extensive testing to evaluate how LLMs can assist in setting up and refining inspection parameters. The research will assess their ability to dynamically adjust parameters based on textual and visual data, handle anomalies, and guide users through the inspection process.
• Defect detection and analysis capabilities of LLMs should be benchmarked against traditional and other AI-based methods, with a specific focus on zero-shot and few-shot learning approaches. Further investigation will assess whether additional training in defect detection is necessary to enhance model performance.

Research challenges include:

• Understanding current state-of-the-art of automated analysis of UT data and current advancements in the field of LLMs
• Developing a framework to adapt and fine-tune open-source LLMs on curated datasets
• Gain an understanding of the technical, legal, and ethical considerations involved in constructing datasets for training LLMs, ensuring adherence to industry standards and best practices.
• Construct and adapt a dataset for training LLMs, incorporating industry standards, technical documentation, and relevant academic research.

This study has wide application scenarios in automation of NDE for manufacturing and in-service inspections across several industries. This research is directly relevant to some of the leading companies in aerospace, nuclear, and defense (DSTL, Rolls Royce, BAE), energy (BP, Shell, EDF Energy, IHI, and Petrobras), and in general NDE engineering (Jacobs). The project is underpinned by high-value infrastructure at the university and one of the UK’s largest manufacturing CATAPULTs (National Manufacturing Institute of Scotland), and is well aligned with the project team’s expertise and research plans. It is aligned with the RCNDE priority areas for long-term research: improved defect detection, improved inspection reliability, minimisation of human factors, robotic and automated inspection during manufacturing, and faster and more accurate large-area coverage. The project aims to a) develop and deliver more industry-focused NDE solutions to promote the partner's and UK's business growth, and b) to introduce development program for the student, where highly demanded skills by the industry, access to a network of NDE experts in academia and industry, access to the state-of-the-art research facilities, and specialized NDE training can be offered to the student.

The project will make extensive use of the £40 million cutting-edge Sensor Enabled Automation & Control Hub (SEARCH) hosting several advanced industrial robots and NDE equipment at the Centre for Ultrasonic Engineering (CUE)at the University of Strathclyde. The student will work closely with RCNDEindustrial network and participate in annual review meetings, presenting the research findings to the attendees, facilitating networking, engagement, and research direction steer for future efforts. The student will work within an internationally renowned and growing team of diverse and multi-disciplinary researchers and engineers, physicists, and mathematicians and will receive a NDE training with advanced industrial KUKA robots, different NDE controllers and sensor technologies.