A Model for Metadata Organisation and Management for Compilation of Specialised Datasets from Big Data

The paper presents a model for the design and management of metadata that enables the efficient compilation of specialised datasets from large, heterogeneous data collections. The metadata are represented as a typed property graph that facilitates the FAIR principles in data compilation: Findable, Accessible, Interoperable, Reusable. The representation is general and independent of the modality and format of the data. The graph-based design of the metadata supports the incremental extension of categories and relationships without requiring the migration of existing data. Its feasibility is demonstrated through the use of a graph database, in which the metadata for 689,645 Bulgarian textual data units are combined with a web-based filtering interface. Metadata retrieval is implemented through Cypher queries executed as graph traversals, enabling the extraction of thematic and application-oriented data subsets based on combinations of selection criteria. The application validates the suitability of the graph-based metadata design for compiling specialised datasets for training and fine-tuning large language models and other NLP applications.