An Assessment of Selected Research Programs and Goals of the Engineering Laboratory at the National Institute of Standards and Technology: Fiscal Year 2024 (2025)
Chapter: 5 Advanced Manufacturing Goal: Advanced Manufacturing Data Infrastructure and Analytics Program
5
Advanced Manufacturing Goal: Advanced Manufacturing Data Infrastructure and Analytics Program
Throughout the panel meeting, several overarching observations were noted. First and foremost, a clear takeaway was that the National Institute of Standards and Technology (NIST) Engineering Laboratory (EL) staff in the Advanced Manufacturing Data Infrastructure and Analytics (AMDIA) Program are highly skilled, motivated, and passionate about their work and organization, with particular emphasis on the early-career individuals who presented a spirit of high morale and satisfaction in their work and career choice. On the other hand, there appears to be a disconnect between the EL staff’s perception of the impact of their work on industry and the level of their engagement with industry, and this panel’s assessment, as guided by the provided materials. Based on the information provided to it, the panel does not believe that they are as effective as they believe they are.
Due to lean resourcing—in supply chain work in particular—it is not evident that the NIST EL mission to promote “the goal of promoting U.S. innovation and competitiveness by advancing measurement science, standards, and technology” (NIST 2021) can be achieved. The projects presented and assessed in the AMDIA Program showed little coordination between them and appeared to be a set of disparate research studies. Greater collaboration between projects could further enhance the outcomes and maximize the use of limited resources across the portfolio of this program’s work.
Recommendation 5-1: The Engineering Laboratory should increase internal collaboration across different programs within it to leverage existing expertise and resources to maximum effect.
Finally, much of the work performed by EL appears to be basic science with a technology readiness level (TRL) of 1–4. While it is not within the purview of this panel to address the type of work that EL does, the question remains whether the low TRL level work aligns with the stated mission of the NIST EL and overall NIST mandate.
ASSESSMENT OF TECHNICAL PROGRAMS
Four AMDIA topics were presented by EL for assessment. These are the following:
- Circular economy—comprising the Circular Economy Product Design and Digital Thread Project and the Circular Economy Closed Loop Recovery Project;
- Biomanufacturing—comprising the Data Infrastructure for Biomanufacturing Process Control Project and the Advanced Data Exchange Standards for the Biomanufacturing Supply Chain Project;
- Supply Chain Traceability—comprising the Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project; and
- The Augmented Intelligence for Manufacturing Systems Project.
When assessing the projects presented, the panel considered several factors: the objectives of the work, the market and industry relevance and need, the merit of the work, dissemination plans, and the potential impact of the work.
Accomplishments
Circular Economy Product Design and Digital Thread Project and the Circular Economy Closed Loop Recovery Project
The objective of circular economy work is to provide measurement science standards and tools that will enable the development of product design, manufacturing, and life-cycle assessment and information to support circular economy implementations. The effort by NIST EL in the past several years in the circular economy area is in line with NIST’s objectives and role in addressing emerging areas in the manufacturing area.
The project team’s participation in the new International Organization for Standardization (ISO) circular economy standards is good. The circular economy is an area where industry can benefit from common standards and definitions. The team has also made several efforts to connect and participate in industry forums to better understand industry needs. Other projects for the team are ongoing and promise to address other important industry needs. The focus on life-cycle data and design is an important area to address. Other examples are digital twins for sustainable manufacturing, improvements of system standards for design and manufacturing, and closed-loop recovery of manufactured products.
Data Infrastructure for Biomanufacturing Process Control Project and the Advanced Data Exchange Standards for the Biomanufacturing Supply Chain Project
The importance of developing standardized data interfaces and best practices to enable consistent and reliable data access across industries has been highlighted by national initiatives and policies. There is an ever-growing need for interoperable, agile, and cost-effective data solutions, especially as biomanufacturing becomes increasingly automated and data-driven. Currently, many companies in this sector use customized, frequently disjointed data management systems that impede effective integration and interoperability. These businesses stand to gain a great deal from an advanced data infrastructure, which will supply standardized, interoperable data systems that will increase overall efficiency and streamline procedures.
These projects aim to conduct measurement science that enables U.S. biomanufacturers to digitally connect supply chain systems more efficiently. They also seek to improve the capabilities of biomanufacturers to access data via the development of software tools, architectures, data frameworks, and various standards.
These projects have successfully created some of the standards and technologies required for the biomanufacturing sector to improve their data automation infrastructures. Other notable achievements include delivering the draft National Institute for Innovation in Manufacturing Biopharmaceuticals (NIIMBL) Core ontology, integrating with the National Aeronautics and Space Administration’s Quantity and Unit ontology, and releasing the Open Applications Group Integration Specification Metadata and Dataset Metadata standards. The NIST Score Tool 3.0 has allowed the more effective exchange of data and interoperability within industry. Through the creation of tools such as the score tool and ontology-based data standards, these projects provide a generally applicable approach to data management that overcomes the drawbacks of conventional, compartmentalized techniques. In addition, the focus of this work on the real-time release of data, continuous manufacturing, and digital transformation is in line with industry trends toward automation and data-driven decision-making. The standardized interfaces created by this project may help the biomanufacturers to integrate discrete complicated supply chain data sets more efficiently. As a result, there may be less chance of yield loss in production processes.
Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project
There is a great market need for supply chain traceability in industrial sectors such as aerospace and electronics manufacturing, food preparation, agriculture, and pharmaceuticals. Many industries have dealt with this problem for several decades and have created in-house solutions (e.g., aerospace and defense) to benefit their particular needs. Traceability allows material provenance and authenticity to be assured, mitigates counterfeit components, enables knowledge creation from the relationship between performance and history, and enables continuous feedback and improvement of the production life cycle.
The objectives of this NIST EL project on supply chain traceability are to create a robust traceability model that supports standards and tools that use consistent semantics and enable the sharing and integration of traceability data for insight across value chains with little effort.
A traceability model has been created that tracks grain processes from the origin to the customer, which builds upon a prior model for grain traceability. Because this project has been active for less than 1 year, continued work focused on testing, validating, and demonstrating the traceability model to non-agricultural use cases—such as circular economy, bio-manufacturing, or pharmaceuticals—is ongoing. Furthermore, the project is seeking to incorporate elements of the models into standards beyond agriculture.
The potential impact of this work resides in the ability of the supply chain traceability model to address industry needs and to fill current gaps so that it may be adopted in industry. Hence, a review of current industry best practices and industry gaps in supply chain traceability is needed to ensure that the model is addressing industry-relevant questions.
Augmented Intelligence for Manufacturing Systems Project
This project addresses a critical market need by tackling the issue of unplanned downtime in machine tools, which can lead to substantial industry losses. By emphasizing on-machine measurement and diagnostics, the project aims to optimize production and prevent the creation of defective parts. The initiative is well aligned with national priorities and policies, underscoring the necessity of enhanced health monitoring and control within the industry.
The primary objective of this project is to advance measurement science through the integration of sensor technologies and augmented intelligence within manufacturing systems. It aims to enhance manufacturing capabilities across various industry sectors, and it focuses on the following four areas: linear axis monitoring, spindle and cutting force monitoring, thermal drift monitoring, and semiconductor manufacturing. These efforts are designed to significantly improve manufacturing efficiency and precision.
The research team has made commendable efforts to connect its work with various industries, including aerospace, by collaborating with major companies like Boeing and Rolls Royce. International collaborations have also been initiated (e.g., with Fraunhofer and several European universities), broadening the scope and impact of the research. The project has facilitated technology and knowledge transfer through partnerships with manufacturers, promoting further collaboration and innovation in the field.
A significant accomplishment is the development of a portable system for monitoring linear axis positioning, which has led to the issuance of a U.S. patent and collaboration with an industry supplier to Rolls Royce for commercialization.
Challenges and Opportunities
Before addressing challenges and opportunities particular to specific work, some challenges apply to the AMDIA Program in general. There appears to be a disconnect between the work done by EL and the applicability of the outcomes of EL’s work to the challenges and needs of industry. It is not clear that the measurement science advanced by EL is useful in addressing industry’s needs. The projects under the AMDIA Program did not clearly define a problem statement coupled with industry needs. While such
problem statements may exist, they were not evident in any of the information provided to the panel, written or verbal. Furthermore, the panel felt that the deliverables (e.g., standards, journal articles, workshops, and presentations) were not well stated, and possibly not known by the staff performing the work.
It is not evident that the current framework of the research plan enables the successful development of a data infrastructure for manufacturing and analytics. A focus on the standardization of data formats would be of interest so that data infrastructure across a manufacturing data stream could be enabled; such an infrastructure is a currently key lack in manufacturing practices. There was little or no discussion on data formatting for data infrastructure for a circular economy or digital twin, which enables interoperability, a worthy goal that has been articulated.
Finally, there do not appear to be detailed schedules with specified deliverables that would make it possible to hold staff accountable for outcomes. It appears that work is evaluated internally in 3- or 5-year cycles, promoting a culture lacking a sense of urgency. It is advised that each project have a clearly defined schedule with milestone deliverables that are reviewed internally annually.
Given the above concerns related to the lack of problem statements, schedules, accountability, and connecting work to industry needs, it is felt that it would be beneficial for EL to put forth a strategy that defines project priorities and how the project outcomes will address industry needs. Furthermore, specific metrics need to be defined and implemented that assess the impact of the performed work as it relates to improving U.S. competitiveness in manufacturing.
An overarching opportunity is to create a plan to transition the results of measurement science to industry and other stakeholders for each of the projects, thereby ensuring the relevance of the work performed and its alignment with industry needs that promote the manufacturing competitiveness of the United States.
Recommendation 5-2: The Engineering Laboratory should assess the obstacles in current industrial practices preventing the advancement of a data infrastructure that will improve productivity, resiliency, and sustainability for manufacturing operations and supply chains, and identify targeted research programs to overcome these obstacles.
Recommendation 5-3: The Engineering Laboratory should create yearly a prioritized list of industry challenges and develop corresponding National Institute of Standards and Technology problem statements that align with those industrial needs. The list and problem statements should specify goals and include a detailed schedule and specify the deliverables that are to be disseminated. All of this should be used to obtain the resources—both people and funding—to achieve the goals in the problem statements.
Circular Economy Product Design and Digital Thread Project and the Circular Economy Closed Loop Recovery Project
It is not clear that an effective analysis of industry needs has been done to identify obstacles and develop a long-term plan to address those. There is a need to focus on areas of high impact that are compatible with the NIST mission. There are also significant efforts in the national and international arenas where strong collaboration between NIST and those efforts can optimize and extend NIST’s impact in this area. Partnerships and participation in existing forums and consortiums can ensure a better understanding of industry needs and integration with related efforts.
Examples of opportunities for collaborations include existing established organizations such as the REMADE Institute with its annual roadmap assessment of industry needs and, on the international front, the World Business Council for Sustainable Development efforts in circular economy. It is advised that a clear needs assessment be developed along with a timeline to address those needs.
Recommendation 5-4: The Engineering Laboratory (EL) should conduct a clear needs assessment to understand industry’s needs and how EL can best address them, integrating its efforts into the broader efforts under way in industry. A timeline to address those needs, reflecting the speed at which things move in industry, should also be developed.
There is a clear need for standardization of product life-cycle assessment and data as well as for circular and end-of-life processes. It is difficult to assess the strategic focus and potential impact of the team’s work based on the information provided on work in the area of life-cycle data and design. While certain life-cycle data exist, other circular and end-of-life processes need a unified data approach to support broad implementations of the circular economy. A unified data approach is of interest across industries to allow them to generate the data needed to develop a full life-cycle assessment of engineering products.
Data Infrastructure for Biomanufacturing Process Control Project and the Advanced Data Exchange Standards for the Biomanufacturing Supply Chain Project
The general challenges to establishing data exchange standards are rooted in the complexity of biopharmaceutical processes and the need to integrate an automated supply-chain data stream. It is stated that the industry uses hybrid mechanistic and data-driven process control models to achieve an elevated level of automation and lowered risk of yield loss. One of the critical approaches this project took to tackle the problem is creating an open-source life-cycle management tool for data exchange standards, named Score. This tool offers a data repository and a suite of functionalities that enable standards developers to create, publish, and maintain data exchange standards with higher precision, quality, and productivity. The Score tool was published publicly on GitHub in fiscal year (FY) 2020 and underwent significant development updates in 2023. It was stated that it has been used in production by the Open Application Group standards organization and several major manufacturing enterprises. However, given the lack of publicly available evidence, it is unclear how industrial partners use the tool and how deeply the tool is involved in industrial value-creating processes. Instead of focusing on continuing work on creating and improving tools and standards, it would be beneficial to both the project, and EL as an organization, to work with industrial partners to understand the added value delivered by the tools and standards resulting from the project’s work.
Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project
The impact of the current work appears to be limited to those in the agricultural community who are inclined to adopt the supply-chain traceability model produced in this work. This project is in the nascent stages of development and the fidelity of the model has not been validated beyond agricultural data. Additionally, the model adoption from agricultural stakeholders has not been evaluated. The ever-present challenges in obtaining real data exist and point to the need for increased collaborations with stakeholders of different industries.
It would be useful to understand the nature of the industrial state of the art. That is, how supply-chain traceability is currently addressed in different industries and how this project could benefit those industries that are already implementing their homegrown supply-chain traceability regimes and are unlikely to shift their decades-old practices for the benefit of conforming to industry semantics. The particular application of agricultural grain supply-chain traceability is important; however, measures of anticipated adoption by agricultural stakeholders have not been provided.
There is a significant opportunity to create a widespread industry-agnostic model that is supported by manufacturing sector agencies or organizations ensuring material provenance and manufacturing authenticity.
Augmented Intelligence for Manufacturing Systems Project
While this project aligns with industry challenges, the current efforts are somewhat basic. Modern commercial manufacturing and cutting tools already offer advanced health monitoring solutions with sophisticated sensor technologies. There is a notable gap in the lack of benchmarking against commercially available options, which could provide a clearer understanding of its competitive edge and areas for improvement.
This project would benefit from advancing its research to higher TRLs by incorporating more sophisticated sensor technologies that align with current industry advancements. This progression is essential to ensure the project’s relevance and competitiveness in the rapidly evolving manufacturing landscape.
Recognizing that NIST is the coordinating body for the Manufacturing USA institutes, a much stronger connection with industry is necessary. Strong collaborations with other established consortia and institutes are also advised to achieve a larger impact across industry. This goal can only be accomplished with proper strategic planning, appropriate staffing and staff assignments, and a clear timeline.
Recommendation 5-5: The Augmented Intelligence for Manufacturing Systems Project should develop strong collaborations with established industry consortia and institutes such as those comprised under Manufacturing USA.
ASSESSMENT OF SCIENTIFIC EXPERTISE
Accomplishments
The EL staff contributing to the reviewed projects have strong educational backgrounds and are highly qualified for their stated work objectives. Among the staff, accomplishments include several fellows of the American Association for the Advancement of Science and the Society of Manufacturing Engineers and several awardees of the Department of Commerce Bronze Medal. Additional awards include the Society of Manufacturing Engineers Outstanding Young Engineer Award, the American Society of Mechanical Engineers (ASME) Best Paper Award, and the ASTM1 E60 Award of Special Service. The staff has been instrumental in facilitating many important outcomes including but not limited to the NIST/ASTM Report on Standards Needs for Circular Economy, ISO standards on circular economy, ASTM standards for sustainable manufacturing, release of a supply chain ontology, the NIIMBL Draft Core ontology covering end-to-end biopharmaceutical processes, and a new ASME standard for linear axis performance. Besides facilitating standards and releasing ontology, the staff has a steady presence in conferences and prolifically publishes conference articles. Many, but not all, the staff actively publish in peer-reviewed journals. However, the outcomes of the projects are not particularly uniform or coherent. More published activity appears to emanate from the circular economy group. This is possibly due to disproportionate resourcing among the goals and projects.
Opportunities and Challenges
Overall, a greater staff understanding of the connection between the relevance of their work and industry’s needs would be beneficial. Such an understanding was not uniformly demonstrated.
World-class expertise in manufacturing and data analytics was not demonstrated, because the research performed (or planned to be performed in the case of the digital twin) is behind the current industrial curve. Several projects appeared to be under-resourced, which further hinders industry leadership in these areas. For example, only one staff member is working on the Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project and only two staff
___________________
1 ASTM International, formerly known as the American Society for Testing and Materials.
members are working on the Augmented Intelligence for Manufacturing Systems Project. More staffing and resources have been provided for the projects related to the circular economy and biomanufacturing, in a likely shift due to the shifting strategic direction of NIST. The thinly resourced projects appeared to be incomplete, with the Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project stalled at grain traceability in agriculture, where broader industrial applicability has not been demonstrated, and the Augmented Intelligence for Manufacturing Systems Project appears to be paused due to lack of investment or industry acceptance or support.
The biomanufacturing projects, despite their more generous resourcing, have produced a relatively lean number of peer-reviewed journal articles for the size of the team and length of the projects, and hence the opportunity lies in greater dissemination. The circular economy projects, while demonstrating several outcomes with respect to the number of peer-reviewed journal articles, seemed to be disconnected from how industry would apply such standards or implement the results from their work.
BUDGET, FACILITIES, EQUIPMENT, AND HUMAN RESOURCES
Budget
The Advanced Manufacturing Goal budget has decreased notably since FY 2022. This is due to a NIST reorganization in which three EL groups were moved to the Communications Technology Laboratory. Between FY 2018 and FY 2022, the budget was relatively flat, but these budget levels have not been adjusted for inflation. This likely means that the Advanced Manufacturing Goal has had less money to work with every year. The panel is concerned about there being adequate budgetary resources for EL to meet the needs in this goal area.
Facilities and Equipment
The AMDIA projects presented and assessed did not require the use of experimental research facilities except for computational resources and off-the-shelf sensors such as accelerometers. The panel did not visit the computing laboratories or hear of deficiencies therein, and it is presumed that the EL computing facilities and the equipment available to the research staff are adequate.
Human Resources
Although the panel meeting coincided with a work onsite day, the parking lot was largely empty and many buildings lacked a bustling and collaborative sense of activity, giving the impression that many employees continue to work from home. The panel is concerned about the effects of too much remote work on mentorship, apprenticeship, collaboration, and innovation. Attention needs to be paid to the balance of remote and onsite work to provide staff with the flexibility they desire, and that is proving critical to attracting and retaining top-level talent while also ensuring adequate opportunities for the mentorship of junior staff and postdocs and maximizing opportunities for collaboration to increase the efficiency of the program’s projects. This could significantly improve many metrics such as collaborative publications, possibly returning them to pre-COVID-19 numbers. The panel recognizes that this is a large issue and a matter of larger federal policy. As such, the panel understands that they cannot readily propose a single or simple solution.
Opportunities exist regarding initiating a mechanism for continuous upskilling so that the staff’s skills evolve at the pace of evolving technology. Finally, a common theme addressed by the EL staff was the inability to hire new talent, partially due to a lack of new funding.
EFFECTIVENESS OF DISSEMINATION EFFORTS
The dissemination of work varied by project and the length of time that each project has been under way. For example, the staff of the circular economy projects, which have been active for a few years, have participated in the ISO committee working on new standards in this area. Notably, one senior staff member working in circular economy and sustainability has authored or co-authored 11 peer-reviewed journal articles, 14 conference papers, and 3 NIST reports in the past 3 years. While this appears to be the most prolific researcher in the projects under review, colleagues in the same area are also actively participating in conferences and journal publications. On the other hand, dissemination of results from the Advanced Data Exchange Standards for the Biomanufacturing Supply Chain Project is relatively lean (e.g., one peer-reviewed journal publication in the past 5 years and nothing published in the past 2 years), which can be attributed to the shorter research period and lower staffing resources devoted to the project in comparison to other projects. The dissemination efforts of the other two projects reviewed fell between the two example cases mentioned.
The level of dissemination is in line with a university research group in similar areas. Given the national role of NIST and its ability to make a high-level impact, the dissemination level and impact were expected to be much higher. In addition, the reach to industry and business was limited.
Circular Economy Product Design and Digital Thread Project and the Circular Economy Closed Loop Recovery Project
These projects plan to share their findings through publications and share both findings and best practices through participating in relevant forums and conferences. The team also participates in standards organizations such as ISO and ASTM to develop standards in the circular economy area. The team’s strong participation in the ISO and ASTM standardization efforts may help to address many industrial implementation challenges. The design for the circularity project also can lead to useful findings with a focus on life-cycle data and measurement science.
While producing a respectable number of peer-reviewed journal articles, the project staff seemed to be disconnected from how industry would apply such standards or implement the results from their work, likely hampering the effectiveness of their work and the dissemination of the results.
Data Infrastructure for Biomanufacturing Process Control Project and the Advanced Data Exchange Standards for the Biomanufacturing Supply Chain Project
These projects plan to share their findings and aid in the dissemination of software tools, best practices, and standards by partnering with organizations like NIIMBL and the Industrial Ontologies Foundry. It also intends to make established ontologies and data management tools widely accessible by publishing toolkits on open-source platforms.
Despite their more generous resourcing compared to other projects, these projects have produced a relatively lean number of peer-reviewed journal articles for the size of the team and length of the project. There is an opportunity for greater dissemination.
Enabling and Using Traceability and Data Linking for Sustainable and Efficient Supply Chains Project
Interactions with EL staff during the meeting left the impression that the current dissemination plans of this project appear to be limited to dissemination to the agriculture community through agricultural publications and standards related to goods movement (not including processing or manufacturing steps). If disseminated more broadly, other non-agricultural industrial sectors may find value in the project results and potentially find ways to leverage the overall framework for their needs.
Augmented Intelligence for Manufacturing Systems Project
This project has successfully culminated in the development of an ASME standard, attracting significant industrial participation. Additionally, the work has led to the publication of numerous high-impact articles and the issuance of a patent. These dissemination efforts underscore the project’s contribution to the field and its potential to influence industry practices.
Recommendation 5-6: The Engineering Laboratory should make a plan to transition the results of its measurement science work to industry to ensure that its work is relevant to industry’s needs.
REFERENCES
NIST (National Institute of Standards and Technology). 2021. “About EL.” Updated June 2. https://www.nist.gov/el/about-el.
NIST. 2024. “Advanced Manufacturing Data Infrastructure and Analytics Program.” Updated May 7. https://www.nist.gov/programs-projects/advanced-manufacturing-data-infrastructure-and-analytics-program.
