Semantic and technically interoperable data exchange in the Flanders Smart Data Space

Abstract

1. Introduction

In response to the growing complexity of the data landscape and the need to manage and share data more effectively, the concept of Data Spaces, understood as sociotechnical ecosystems modeling the relations and interactions of data actors within and across application domains, has been recently introduced [6,9]. Interoperable Europe, a body of the European Commission that supports and promotes interoperable sharing of assets for public administrations, defines a Data Space as "an environment bringing together relevant data infrastructures and governance frameworks in order to facilitate data pooling and sharing."

Data Spaces introduce a paradigm shift in the way organizations interact with data. By integrating semantic technologies into Data Spaces, data can be structured and organized by meaning and context, enabling complex questions to be asked, connections to be discovered and valuable insights to be generated. An important aspect of Data Spaces is their decentral nature, meaning that the data themselves may be scattered across different physical locations or sources. The use of semantic technologies however, may enable a virtual, semantic centralization that facilitates the collaboration, integration and meaningful exchange of data in a distributed environment.

To navigate today's data integration challenges and align with the evolving concept of Data Spaces, the Belgian region of Flanders introduced the Vlaamse Smart Data Space (VSDS). This initiative reflects Flanders' commitment to maintaining its leading position in digitization by fostering the exchange of semantic data in a flexible and scalable manner. In 2012, Flanders laid the initial foundations for the exchange of semantic data, by establishing the Flemish Interoperability Program, known as Open Standards for Linked Organizations (OSLO) [4]. The program is instrumental in creating data standards within Flanders, based on the principles of Linked Data. This approach ensures that each term in the data model is assigned a URI, either re-used from an international data standard or newly created by OSLO when no suitable international URI could be found.

While OSLO has contributed to infusing semantics into Flanders' data landscape, today's challenges with data integration are centered around practical necessities to facilitate consistent and uniform data exchange interfaces. In response to this need, VSDS collaborated with imec and Interoperable Europe to develop the Linked Data Event Stream (LDES) specification for publishing data [13]. To strengthen this initiative of semantic and technical interoperability, VSDS is currently developing open-source building blocks which serve a dual purpose: enabling the publication of LDESs, but also facilitating their consumption within the ecosystem. This strategic development marks a pivotal step towards a more cohesive and interoperable data-sharing environment in Flanders.

In this paper, we present an overview of VSDS and a description of the main building blocks, developed as open-source software components, that enable uniform exchange of data across organizational boundaries, by leveraging Semantic Web technologies. Furthermore, we show how the VSDS framework and building blocks were (and are being) applied in a real-world scenario with multiple stakeholders, namely the Traffic Measurements Data Space, and discuss the impact of VSDS on facilitating data integration tasks and application development.

The structure of this paper is outlined as follows: Section 2 offers an overview of alternative data exchange specifications and concurrent Data Space initiatives. In Section 3, we delve into a specific use case of VSDS, traffic measurements. Section 4 provides a comprehensive view of the proposed solution, in which the open-source building blocks play an essential role. Section 5 discusses the advantages and constraints of the current approach. Lastly, Section 6 presents our concluding remarks and outlines perspectives for future work.

Useful links referenced in this section: Interoperable Europe, SEMIC Data Spaces, VSDS portal, imec article, LDES on Joinup, LDES specification.

2. Related Work

The Horizon 2020 project Open DEI Aligning Reference Architecture, Open Platforms and Large-Scale Pilots in Digitising European Industry, led by the International Data Spaces Association (IDSA), introduced various building blocks which are considered to be the core elements that must be implemented by Data Space initiatives [8]. IDSA is a global non-profit organization promoting secure and sovereign data sharing in digital ecosystems through Data Spaces. They continue to develop the IDSA Reference Architecture Model (RAM), a conceptual model which serves as a blueprint with the purpose of providing common architectural principles and guidelines for organizations to create or join a Data Space. It may be considered as the current leading architecture reference for Data Space initiatives and can be seen as a set of technical components within the broader context of building blocks.

Data Space building blocks are further categorized into being part of the control plane or data plane, based on the type of management task they address. Within VSDS the focus is on the data plane, with the OSLO data standards, based on the principles of Linked Data, being the response to the Data Models and Formats building block. The Data Exchange APIs building block is responsible for making actual data exchange possible. IDSA acknowledges a high heterogeneity of data exchange alternatives, such as polling, publish/subscribe, event-based, and large dataset transfer, and advocates for common protocol definitions that allow multiple strategies to remain interoperable within Data Spaces.

In this direction, the VSDS adopts the LDES specification, co-developed together with imec and Interoperable Europe, into its architecture as the core API to exchange data. Some work is already in progress to align the VSDS approach with existing Data Space initiatives, for example, by extending the Eclipse Data Space Connector to support and interact with LDES. Technical details are out of scope for this paper, but more information can be found on GitHub.

The LDES specification combines the principles of event streaming with those of Linked Data, creating a model suitable for describing the life-cycle of data sources, and aimed to support replication and synchronization of such data sources [13]. LDES describes and publishes data sources' changes as a stream of immutable entities and their relationships, using semantically described and hypermedia-based data structures [5]. LDES is domain model-agnostic and works both for fast, such as sensor data, and slow, such as an address registry, moving datasets. LDES has been used, among others, to publish base registries in Flanders [13], the Marine Regions dataset [7], cultural heritage data [12], and time-series data [15].

The main goal of LDES is to redistribute costs across the different stakeholders, resulting in a more sustainable and cost-efficient approach to exchange data, inspired by the Linked Data Fragments conceptual framework [14]. Current data integration practices are often dictated by specific use cases. This entails a scenario where a data publisher tailors and manages a custom API according to the unique requirements of each use case. Subsequently, a data consumer must integrate this custom API into their application. Whenever a data consumer requires data from another data publisher, the likelihood is high that they will need to write new code and develop new interfaces. Concurrently, the data publisher faces the challenge of hosting and maintaining multiple custom APIs on top of the same dataset.

Additionally, the expenses incurred through querying these custom APIs typically fall upon the data publisher. Conversely, with LDES it is possible to lower the maintenance burden, giving data publishers the flexibility to opt for an LDES-only availability for their dataset, which was the main reason for Marine Regions to opt for LDES [7]. In this approach, if a data consumer requires a specific API, such as a WFS or SPARQL endpoint, they can replicate the data from the LDES and host it within their infrastructure. However, when it comes to integrating the data into their database, the consumer can create reusable code because the data exchange protocol is standardized, streamlining the integration process across various use cases. The VSDS is also developing the other building blocks related to the control plane, but these are out of scope for this paper.

FIWARE acts as an open-source community aimed at simplifying the development of smart applications and solutions. By offering standardized APIs, of which the Context Broker and Smart Data models are the most known, it provides another path toward interoperable systems. The API specification implemented by a context broker and Smart Data models is called NGSI-LD [1] and is standardized by the European Telecommunications Standards Institute (ETSI). FIWARE has made efforts towards implementing a Data Space architecture based on NGSI-LD interfaces [2,11], however to the best of the authors' knowledge, such implementations have not been taken beyond proof-of-concept state.

Another Data Space-related initiative is Gaia-X. This is a European initiative emerging from industry, policy-making and research, designed to promote a secure and federated digital infrastructure with its focus on data sharing and collaboration between organizations. Gaia-X and IDSA share similar goals regarding the development of a secure and sovereign digital ecosystem and both focus on data sharing and collaboration. Therefore, alignment efforts between Gaia-X and IDSA have been made [10,3]. Furthermore, IDSA, Gaia-X, and FIWARE currently collaborate in the so-called Data Space Business Alliance, aiming to unify the vision of Data Spaces.

3. Use Case

To speed up the ecosystem's adaptation to the paradigm shift brought about by LDES, VSDS aims to lower the barriers to get started with this new technology. The ultimate goal of the VSDS for Flanders is to lay out the groundwork for the ecosystem so that everyone can create their own domain-specific Data Spaces, tailored to what makes sense to them. Several initiatives are already underway, demonstrating the feasibility and value of this approach.

The Traffic Measurements Data Space

The VSDS framework is employed within the mobility sector, initiating a targeted exploration for a well-defined mobility use case amidst a comprehensive stakeholder analysis. This led to the selection of traffic measurement data as central to the data space's application, emphasizing its relevance. Monitoring of traffic flows leverages a variety of technological tools, each type of sensor utilizing its distinct communication protocol to relay measurement values alongside sensor details. However, the realm of traffic data measurement suffers from the absence of a unified, coordinated data model standard. Moreover, data acquisition is scattered across numerous entities. Specifically, within the Flanders region, it is estimated that over 500 stakeholders possess traffic measurement data. This encompasses both public entities, such as local governments and regional agencies, and private sector participants, all of whom utilize this data for a broad spectrum of applications.

Within the mobility sector alone, it serves as a foundational element for traffic control strategies, monitoring, analysis, simulation inputs, and the creation of digital twins. Additionally, traffic data underpins regional and economic development initiatives, noise and emission modeling, spatial planning permits, retail site selection, tourism activity monitoring, and even audience measurement for billboard advertising. Nevertheless, this wealth of traffic data remains siloed within various entity-specific databases or applications, with portions of the data isolated on individual staff members' laptops. The establishment of a traffic measurement data space is thus a pivotal step towards evolving the data ecosystem for this particular use case.

Uniting the scattered mobility data landscape begins with consensus-building among mobility experts on a uniform semantic framework. Leveraging the OSLO Process and Methodology [4], stakeholders embarked on standardizing data across the mobility domain, with the initial focus on traffic measurements. By creating data standards using this standardized approach, local governments and public organizations in Flanders can enforce the conformance of a data standard from their suppliers in their public tenders.

The OSLO program is tasked within VSDS with overseeing the semantic agreement between stakeholders, ensuring a common semantic foundation across the ecosystem. This involves multiple workshops spanning in-depth discussions about semantics and reaching consensus, followed by a public review phase, allowing data publishers to integrate the data model into their applications.

The result in this case is the OSLO Traffic Measurement model and its application profile. The data model builds on the Observations and Measurements OSLO model, based on the ISO 19156 standard, and the Sensors and Sampling OSLO model, based on the W3C SSN/SOSA standard. There are also references to INSPIRE's Transport Network model and the ISO 19157 Data Quality standard.

The main focus concept is the Traffic Measurement class, which indicates which traffic characteristic of which traffic object was measured and what the result was. Traffic is understood as the movement of objects such as people or goods over a transport link such as a road, waterway, railway, and so on. In this model the focus is on road traffic. Objects related to traffic are the objects that move or are moved, such as vehicles and road users such as drivers, passengers and pedestrians. Of equal importance are the objects over which traffic takes place. In order to observe traffic characteristics of the objects that are moved, classes such as Vehicle and Traffic Participant are necessary. Examples of characteristics in that case are speed, license plate and the like. However, as the domain of traffic measurements is very broad, including cross section counts, intersection counts, origin-destination counts, and so on, it was decided by the ecosystem to focus on cross section counts and thus further specify the application profile into an implementation model, while leaving the option open for the other aspects of traffic measurements.

Further details about the data model and implementation of the Traffic Measurements Data Space are also available online through the OSLO and VSDS documentation.

Data Sources

In total 26 organizations took part in the OSLO workshops to establish a common foundation on how to exchange data about traffic measurements. Furthermore, five were willing to publish their data as an LDES and based on the OSLO Traffic Measurements data model:

• Telraam captures passengers, bicycles, cars, and trucks with around 2000 sensors. The published LDES is available at telraam-api.net/ldes/observations/by-page.
• Geomobility captures data about bicycles, cars, and trucks using pneumatic tubes. The published LDES is available at brugge-ldes.geomobility.eu/observations.
• Roads and Traffic Agency in Flanders captures data about cars, trucks, and bicycles. For the main roads in Flanders induction loops are used, while pneumatic tubes are used for temporary counting campaigns on the underlying road network.
• Signco captures data about bicycles, cars, and trucks using both induction loops and pneumatic tubes, depending on the location.
• Krycer captures cars and trucks using radar for speed measurements.

Each of these data publishers agreed to make their traffic measurements within a timespan of 15 minutes available through an LDES.

4. Solution

With OSLO being responsible for the semantic interoperability, VSDS focuses on the technical layer, particularly through the adoption and promotion of the LDES specification. To facilitate the adoption of LDES and stimulate ecosystem engagement, VSDS has embarked on the development of a suite of open-source and reusable building blocks. These tools are designed to serve every participant in the ecosystem, going from data publishers to data consumers, and aiming to build a community that innovates and stimulates the growth of the data economy. The introduction of these building blocks represents the effort by VSDS to lower the initial hurdles associated with implementing the LDES specification, thereby encouraging broader participation in the ecosystem.

The suite of building blocks includes three main components:

• LDES Server, serving as the backbone for publishing datasets as LDESs and implementing fragmentation strategies and retention policies.
• LDES Client, tasked with replication and synchronization of LDESs from various sources.
• LDES Data Pipeline components, specialized configurable components for transformation, filtering, and writing data into databases or other storage solutions.

All these components are open-source, written in Java, and together they form a comprehensive toolkit that supports both the publication and consumption of LDESs, enhancing interoperability. Concurrently, VSDS continues to develop an onboarding process to streamline the onboarding of organizations into VSDS. By providing tailored support for both data publishers and consumers, VSDS is laying the groundwork for a more integrated and interoperable data ecosystem.

The onboarding process consists of five steps:

1. Intake, where needs, stakeholders, and datasets are mapped out collaboratively.
2. Pilot, where a pilot version is developed to meet the specific needs of the stakeholders.
3. Setup, where detailed planning follows from the pilot insights.
4. Development, focused on the requirements defined during intake and pilot.
5. Activation, where stakeholders are supported while implementing the solutions in their own architecture.

All of the stakeholders underwent the described onboarding process, leading to the launch of the Traffic Measurements Data Space in 2023. An example of the final technical solution designed for Geomobility to create and publish its LDES is shown in Figure 3.

The light blue blocks represent the data pipeline that is needed to transform the original data of Geomobility to the OSLO model and ingest it into the LDES Server. The pipeline is as follows:

1. HTTP Listener, listening for HTTP messages sent by the Idaso backend system.
2. HTTP Listener Adapter, adding a JSON-LD context to transform received JSON to Linked Data.
3. WKT Converter, converting GeoJSON geometry objects into WKT.
4. Site and data converter plus OSLO Converter, transforming the data to make it compliant with OSLO.
5. Version Creator, preparing versioned immutable LDES members.
6. Version Sender, ingesting the LDES member into the LDES Server with an HTTP POST request.

Consuming LDESs

To illustrate the effectiveness of VSDS in addressing data integration challenges, a demo application was developed that visualizes traffic measurements on a map. This application highlights the streamlined data integration process enabled by LDES and the VSDS building blocks.

The initial step consisted of setting up a SPARQL quadstore, in this case an Ontotext GraphDB instance, and a data pipeline to replicate and sync LDES data into the database. The choice for an Ontotext GraphDB is based on the stakeholders' knowledge about quadstores. The pipeline supports any quadstore that implements the RDF4J API. Adding a new dataset is done through configuration, as demonstrated with the Telraam LDES integration in Listing 1.1. This simplicity underlines the shift towards a configuration-driven approach in data integration, reducing the complexity traditionally associated with it.

Listing 1.1: Pipeline configuration for replicating an LDES into a SPARQL quadstore

server:
  port: 8080
orchestrator:
  pipelines:
    - name: Telraam - LDES
      input:
        name: be.vlaanderen.informatievlaanderen.ldes.ldi.client.LdioLdesClient
        config:
          url: https://telraam-api.net/ldes/observations/by-page?pageNumber=1
          sourceFormat: text/turtle
      outputs:
        - name: be.vlaanderen.informatievlaanderen.ldes.ldi.RepositoryMaterialiser
          config:
            sparql-host: http://host.docker.internal:7200
            repository-id: LDES
            named-graph: http://telraam.be/ldes

With the data integration challenges reduced to a matter of configuration, efforts were able to be redirected towards the development of the application. First, an OpenAI assistant was created capable of generating SPARQL queries aligned with the OSLO data model based on user natural language inputs. These queries facilitated the retrieval of data from GraphDB which was then visualized on a map, where the question asked was "Show me all measurements for cars between 15h and 16h for which the count is higher than 70". The result of this query was a list of data points, namely traffic measurements with a coordinate, that have a count higher than 70, regardless of the vehicle, between the 15-16h period of any day. A second view shows these traffic measurements in a graph for a specific location.

This demo application validates the LDES specification and the VSDS building blocks' capability to simplify data integration, while highlighting its potential to enhance the development of data-driven applications.

5. Discussion

The VSDS initiative, through its comprehensive approach to data integration, leveraging the LDES specification, and the development of ecosystem-supporting building blocks, demonstrates potential in addressing the complexities inherent in data sharing and interoperability.

However, despite the progress, the initiative faces challenges. The reliance on stakeholders' willingness to adopt and integrate new standards and technologies, such as LDES and OSLO data models, can vary. The technical barriers, while lowered, still require a certain amount of technical knowledge that may not be uniformly present across all data publishers and consumers. Moreover, the initial aim of LDES was to redistribute the costs across all stakeholders. The general experience during the onboarding process of data publishers was that LDES leads to additional costs in the first phase. This is due to some data publishers putting the LDES Server next to their existing APIs, which generates an extra cost. The noted benefits of LDES are lower integration costs and larger data availability at consumer side.

The iterative and collaborative approach adopted in the development and implementation of the Flanders Smart Data Space, evidenced by the structured onboarding process, highlights the initiative's methodology and openness towards its ecosystem. Continuous engagement and feedback from a broader spectrum of stakeholders are essential for refining both the onboarding process and the building blocks, which will enhance the applicability and effectiveness.

6. Conclusion

In the scattered landscape of data, the VSDS initiative means a step forward in the quest for seamless data integration and interoperability within the region of Flanders. By addressing both the semantic and technical layers of data integration, VSDS enables the enhancement of data accessibility and usability, but also paves the way for innovative data-driven solutions, as demonstrated by the traffic measurements application.

The key contributions lie in the holistic approach to data integration, the development of a suite of tools that simplify the data sharing process, and the potential to serve as a blueprint for similar initiatives. By working towards configuration-only instead of custom code in data integration processes, it stands out as an innovative aspect, offering a scalable and efficient pathway to integrating various data sources.

Looking forward, the continuous enhancement of the VSDS framework and its building blocks, to enclose additional domains beyond traffic measurements, and its alignment with international Data Space initiatives, present new opportunities for further research and development. The ongoing evolution of Semantic Web technologies and data standards will play a critical role in this endeavor. Moreover, the VSDS's ability to foster an ecosystem of collaboration and shared standards across different sectors and regions will be very much needed in realizing the full potential of interoperable Data Spaces.

To make the ongoing efforts on the Flemish level more sustainable on a European and international level, VSDS is undertaking several actions. First, VSDS is positioning the LDES specification in international ecosystems and architectures such as FIWARE. Preliminary analysis has indicated that NGSI-LD and LDES could be compatible. A document is being worked on that will be standardized by ETSI to make this official, but at the time of writing this had not been done yet. VSDS will continue to monitor and work towards compatibility with other Data Space initiatives such as the Eclipse Data Space Connector, positioning LDES within IDSA and Gaia-X ecosystems.

References

1. Ahmed Abid, Alexey Medvedev, Ali Hassani, Franck Le Gall, Giuseppe Tropea, Juan Antonio Martinez, Lindsay Frost, and Martin Bauer. Guidelines for Modelling with NGSI-LD. 2021.
2. Álvaro Alonso, Alejandro Pozo, José Manuel Cantera, Francisco De la Vega, and Juan José Hierro. Industrial data space architecture implementation using FIWARE. Sensors, 18(7), 2018.
3. Arnaud Braud, Gaël Fromentoux, Benoit Radier, and Olivier Le Grand. The road to European digital sovereignty with Gaia-X and IDSA. IEEE Network, 35(2):4-5, 2021.
4. Raf Buyle, Laurens De Vocht, Mathias Van Compernolle, Dieter De Paepe, Ruben Verborgh, Ziggy Vanlishout, Bjórn De Vidts, Peter Mechant, and Erik Mannens. OSLO: Open standards for linked organizations. In Proceedings of the International Conference on Electronic Governance and Open Society: Challenges in Eurasia, pages 126-134, 2016.
5. Pieter Colpaert. Building materializable querying interfaces with the TREE hypermedia specification. In Damien Graux, Fabrizio Orlandi, Emetis Niazmand, Gabriela Ydler, and Maria-Esther Vidal, editors, Proceedings of the 8th Workshop on Managing the Evolution and Preservation of the Data Web (MEPDaW), CEUR Workshop Proceedings, pages 8-18, 2022.
6. Edward Curry, Simon Scerri, and Tuomo Tuikka. Data Spaces: Design, Deployment and Future Directions. Springer Nature, 2022.
7. Britt Lonneville, Harm Delva, Marc Portier, Laurian Van Maldeghem, Lennert Schepers, Dias Bakeev, Bart Vanhoorne, Lennert Tyberghein, and Pieter Colpaert. Publishing the Marine Regions gazetteer as a linked data event stream. In JOWO, 2021.
8. Lars Nagel and Douwe Lycklama. Design principles for data spaces - position paper. Technical report, 2021.
9. Boris Otto. The evolution of data spaces. In Designing Data Spaces: The Ecosystem Approach to Competitive Advantage, pages 3-15. Springer International Publishing, 2022.
10. Boris Otto, Alina Rubina, Andreas Eiter, Andreas Teuscher, Anna Maria Schleimer, Christoph Lange, Dominik Stingl, Evgueni Loukipoudis, Gerd Brost, and Gernot Boge. Gaia-X and IDS, November 2021.
11. Gürkan Solmaz, Flavio Cirillo, Jonathan Fürst, Tobias Jacobs, Martin Bauer, Ernö Kovacs, Juan Ramón Santana, and Luis Sánchez. Enabling data spaces: existing developments and challenges. In Proceedings of the 1st International Workshop on Data Economy, pages 42-48, 2022.
12. Brecht Van de Vyvere, Olivier Van D'Huynslager, Achraf Atauil, Maarten Segers, Leen Van Campe, Niels Vandekeybus, Sofie Teugels, Alina Saenko, Pieter-Jan Pauwels, and Pieter Colpaert. Publishing cultural heritage collections of Ghent with linked data event streams. In Research Conference on Metadata and Semantics Research, pages 357-369. Springer, 2021.
13. Dwight Van Lancker, Pieter Colpaert, Harm Delva, Brecht Van de Vyvere, Julián Rojas Meléndez, Ruben Dedecker, Philippe Michiels, Raf Buyle, Annelies De Craene, and Ruben Verborgh. Publishing base registries as linked data event streams. In International Conference on Web Engineering, pages 28-36. Springer, 2021.
14. Ruben Verborgh, Miel Vander Sande, Olaf Hartig, Joachim Van Herwegen, Laurens De Vocht, Ben De Meester, Gerald Haesendonck, and Pieter Colpaert. Triple pattern fragments: A low-cost knowledge graph interface for the web. Journal of Web Semantics, 37-38:184-206, 2016.
15. Tom Windels, Wout Slabbinck, Pieter Bonte, Stijn Verstichel, Pieter Colpaert, Sofie Van Hoecke, and Femke Ongenae. LDESTS: enabling efficient storage and querying of large volumes of time series data on solid pods. In Posters, Demos, and Industry Tracks at ISWC 2023, CEUR, 2023.