While the previous post on ETL pipelines focused on the production and construction of data, truly leveraging these assets within the research ecosystem requires a deep dive into the FAIR data principles. As the volume of big data grows exponentially, finding and utilizing the specific data one needs has become an increasingly daunting task. The FAIR (Findable, Accessible, Interoperable, Reusable) principles were introduced as a definitive solution to this challenge.
The origin of these principles traces back to the 2014 workshop, “Jointly Designing a Data FAIRport,” held in Leiden, Netherlands. At this event, diverse stakeholders—including researchers, publishers, and funding agencies—grappled with a critical question: “How can we ensure research data continues to create value rather than being lost or forgotten?” Their deliberations led to the formal publication of the “FAIR Guiding Principles” in the journal Scientific Data in 2016. This marked a historic paradigm shift, moving the focus of data management from journals to the data itself.
Global momentum for the FAIR principles accelerated quickly. At the 2016 G20 Hangzhou Summit, a formal statement was issued supporting the application of FAIR principles to research. By January 2020, the European Commission’s European Data Strategy mandated that all beneficiaries of Horizon Europe funding adhere to open data guidelines, decreeing that all data generated within this framework must be FAIR by default. That same year, representatives from nine prestigious global university groups signed the Sorbonne Declaration on Research Data Rights. By including a firm commitment to FAIR data, they helped establish these principles as the essential global standard for research data management.
The efforts of Matthias Scheffler’s research team at the Fritz Haber Institute (FHI) and Claudia Draxl at Humboldt University Berlin were instrumental in establishing these European-born principles as a practical philosophy for infrastructure building in the field of materials science AI research. The Scheffler group defined the FAIR principles not merely as a set of guidelines, but as a core infrastructure for transforming vast amounts of experimental and computational data into “knowledge and value.” In particular, the NOMAD (Novel Materials Discovery) ecosystem they led was already practicing these core values even before the FAIR principles were formally proposed; since integrating with the FAIR concept, it has become a global benchmark.
A key point emphasized in the series of papers published by the Scheffler group (notably their 2022 Nature article) is that data must be in a form that is “machine-actionable”—readable and processable not only by humans but also by AI/ML. It calls for securing connectivity between datasets through rich metadata and standardized ontologies, going far beyond simple data storage. Notably, the ‘R’ in FAIR must transcend mere “Reusability” to reach the level of being “Repurposable,” where data can be immediately applied to new research questions entirely different from its original purpose.
Key components of the FAIR data principles
The FAIR principles present a new paradigm for data management and utilization, providing a foundation for researchers to use and develop data efficiently. Each element is essential for maximizing data value and is categorized into the following four components:
- Findable: This is the most fundamental and essential attribute of data management. Data must be easily searchable so that researchers can quickly find the information they need. This contributes to maximizing the value of the data and enhancing research efficiency. Searchability is strengthened by ensuring datasets can be discovered through persistent unique identifiers (e.g., DOI).
- Accessible: Data accessibility is closely linked to security and open data policies. The FAIR principles evolve in alignment with the Open Science movement, and “Accessible” can be seen as having a significant overlap with the concept of “Open Access.” However, while Open Access has primarily developed around research journals, the FAIR principles focus on the accessibility of the data itself. This means that data should be retrievable by authorized users through appropriate access management and security protocols.
- Interoperable: Rather than focusing solely on the dataset itself, the connectivity between the data, the platform, and the broader ecosystem is crucial. Data must be seamlessly exchangeable between different systems, which necessitates data standardization and the construction of ontologies. Interoperability allows for the integration of diverse datasets, enabling more comprehensive and multi-faceted analysis.
- Reusable: Within the NOMAD project, this is often referred to as being “repurposable.” The core philosophy here is that data realizes its true value when it is reused for a purpose different from its original intent. It is essential to provide clear metadata and licensing information so that data can be confidently applied to a wide range of new research questions, thereby maximizing its potential for reuse.
These FAIR principles serve as a guide for researchers to manage and utilize data more effectively and provide a vital foundation for the sustainable development of the data ecosystem.
Establishing Quality and Value in Materials Data
While establishing FAIR data and performing standardization processes share many similarities, there are significant differences between the two. FAIR data is not just a data management policy; it is an essential framework for building the infrastructure or platforms needed to guarantee findability, accessibility, interoperability, and reusability. This allows researchers to utilize data more efficiently and maximize its inherent value.
From this perspective, the entire ETL (Extract, Transform, Load) pipeline process must remain faithful to the FAIR principles. The ETL process encompasses the full workflow of collecting, transforming, and loading data into a repository. During this process, data should be designed based on FAIR principles to ensure its quality and utility. For instance, in the extraction stage, the required data must be accurately identified, and in the transformation stage, appropriate metadata must be added. Finally, when loaded into the repository, the data should be stored in a form that is accessible and interoperable according to FAIR standards.
In this regard, utilizing the Elastic Stack offers several advantages. The Elastic Stack is an integrated platform for data collection, storage, search, analysis, and visualization, making it highly effective for implementing FAIR principles. Data can be efficiently collected and analyzed through the Elastic Stack, thereby increasing findability and allowing for the effective management of large-scale datasets. Furthermore, the Elastic Stack supports interoperability with various data sources, strengthening the connectivity between data and contributing to increased reusability.
The comparison between the FAIR principles and Open Science is also crucial. Open Science is a concept that emphasizes openness and transparency in research, with its primary goal being the public release of research journals and data. In contrast, the FAIR principles center on the management and utilization of data, distinguishing themselves by requiring that data be designed to be useful for both machines and humans. In other words, while Open Science prioritizes the openness of data, the FAIR principles focus on maximizing its usability.
The Importance of Metadata and Its Relationship with FAIR Data Principles
Metadata—often described as “data about data”—plays an essential role in enhancing the understanding and usability of datasets. Particularly in fields like Machine Learning (ML), the construction and application of metadata have become core elements for maximizing the inherent utility of data. Metadata includes diverse information such as data provenance, structure, content, format, and quality, helping users (and systems) understand and properly utilize the data.
The FAIR data principles are intrinsically linked to metadata, reinforcing the overall value of data through the specific functionalities of each component. Let’s examine how metadata facilitates the application of each FAIR principle:
- Findable: Metadata is a key contributor to enhancing the discoverability of datasets. Well-structured metadata allows users to quickly locate the specific data they need within databases or search engines. For example, by including persistent unique identifiers such as DOIs (Digital Object Identifiers), researchers can ensure their data is easily found and cited by others.
- Accessible: Metadata plays a vital role in improving data accessibility. It provides essential information regarding the data’s storage format and specific access permissions, helping users understand how to retrieve the data. This connects directly to the goals of Open Science, fostering an environment where a broader range of researchers can access and utilize valuable data assets.
- Interoperable: Metadata provides the necessary information to facilitate interoperability between different datasets. It ensures that data can be exchanged seamlessly across various systems and platforms. By utilizing standardized metadata formats, the connectivity between disparate data sources is strengthened, significantly increasing the potential for large-scale data integration and cross-disciplinary analysis.
- Reusable: Metadata is critical for maximizing the potential for data reuse. By clearly documenting the original purpose of data collection and the specific transformation processes the data has undergone, metadata allows other researchers to confidently apply that data to a wide range of new research questions. In the case of the NOMAD project, rich metadata is what allows “consumed” data to be repurposed as a foundation for entirely new scientific discoveries.
In this way, metadata acts as a core element in implementing the FAIR data principles and significantly enhances the inherent utility of data. Its importance is increasingly emphasized across various research fields, including machine learning, where it has become essential for maximizing data value and usability. Beyond mere data management, the construction and management of metadata are indispensable prerequisites for the sustainable development of the data ecosystem.
A New Leap in Data-Driven Materials Research Led by FAIR Principles
To maximize the value of materials research data, we must move beyond simple storage toward a progressive shift in intelligent utilization. The FAIR principles represent a practical philosophy ensuring that data is not lost but transformed into meaningful knowledge for both humans and machines. To realize this, the advancement of standardization, metadata, and ETL pipelines is central to research competitiveness in the AI era.
By combining leading data ecosystems with high-efficiency analytical tools like the Elastic Stack, data can evolve from a one-time consumable into a “repurposable” asset that continuously generates new value. Ultimately, internalizing the FAIR principles will not only increase the transparency of materials science research but also serve as a solid foundation for accelerating data-driven innovation.