AWS Architecture Blog Unveils Comprehensive Guide to Implementing Eclipse Dataspace Components for Secure, Interoperable Data Sharing

A new three-part blog series from the AWS Architecture Blog is set to guide enterprises through the intricate process of implementing Eclipse Dataspace Components (EDC) on Amazon Web Services, offering a pathway from foundational concepts to robust production deployments. This initial installment lays the theoretical groundwork, delving into the International Data Spaces Association (IDSA) standards, the Dataspace Protocol (DSP), and the core architectural tenets of EDC. Subsequent parts promise to address production-ready AWS deployment patterns utilizing services such as Amazon Elastic Container Service (Amazon ECS), Amazon Aurora, and Amazon API Gateway, culminating with practical cost optimization strategies for efficient and scalable data space infrastructure on AWS.

The Imperative for Data Spaces: Addressing Modern Challenges

The digital economy is increasingly reliant on seamless, secure, and sovereign data exchange. However, traditional data sharing mechanisms often fall short, plagued by issues such as data silos, a lack of trust between entities, complex legal and technical agreements, and prohibitive integration costs. Organizations frequently face the dilemma of leveraging valuable data assets without compromising privacy, intellectual property, or regulatory compliance. This challenge is particularly acute in cross-organizational scenarios like supply chains, smart manufacturing (Industry 4.0), healthcare, and urban mobility, where collaborative innovation hinges on trusted data flows.

The concept of "data spaces" emerges as a transformative solution, designed to overcome these hurdles by establishing a federated, governed environment where data providers and consumers can interact under mutually agreed-upon rules and policies. Unlike centralized platforms that often lead to vendor lock-in or control over data, data spaces champion data sovereignty, allowing data owners to retain control over their assets even when shared. This paradigm shift is not merely a technical evolution but a fundamental re-imagining of data governance and value creation in a connected world. The global market for data sharing technologies is projected to grow significantly, driven by these demands for secure, compliant, and efficient data exchange, underscoring the urgency for standardized and robust implementation frameworks.

IDSA and the Foundation of Trust

At the forefront of establishing these new data paradigms is the International Data Spaces Association (IDSA), a non-profit organization dedicated to fostering and promoting open standards for data spaces. Founded with the vision of creating a trustworthy data ecosystem, IDSA defines a data space as "a set of technical services that facilitate interoperable dataset sharing between distinct entities." This definition underscores the technical and organizational framework required to enable secure data transactions across different organizations, fostering a collaborative yet controlled environment.

Technically, a data space operates with distinct participants: data consumers, who seek to utilize data, and data providers, who offer data assets. Crucially, a central entity known as the Dataspace Governance Authority (DSGA) oversees the entire ecosystem. The DSGA is responsible for managing the data space, enforcing established rules and policies, and issuing membership credentials to participants, thereby ensuring a foundation of trust and compliance. This governance model is pivotal in mitigating risks associated with data sharing, such as misuse, unauthorized access, and non-compliance with data protection regulations. The IDSA’s efforts are instrumental in providing the conceptual blueprint and architectural models that underpin the development of practical data space implementations. The association has been actively engaging with various industry consortia and regulatory bodies since its inception around 2016-2017, steadily building consensus and accelerating the adoption of its reference architecture.

Standardizing Interoperability: The Dataspace Protocol (DSP) and ISO/IEC DIS 20151

The IDSA’s abstract rules and specifications find their concrete technical realization in the Dataspace Protocol (DSP). This protocol serves as the common language and set of communication rules that enable different components within a data space to interact seamlessly. Recognizing its critical importance for broad adoption and interoperability, the DSP has evolved into an Eclipse Specification project, reflecting a commitment to open-source development and community-driven standardization. The protocol’s source code and ongoing development can be found in the Dataspace Protocol GitHub repository, inviting contributions and fostering transparency.

The significance of DSP extends beyond its open-source origins, as it is currently undergoing formal standardization as ISO/IEC DIS 20151. This international standardization effort, which is presently in the approval phase, marks a pivotal moment for the future of data spaces. Attaining ISO/IEC status will provide a universally recognized benchmark for secure and interoperable data exchange, boosting confidence among enterprises and governments in adopting data space technologies. This move not only validates the technical robustness of DSP but also paves the way for wider global adoption, ensuring that data spaces built on this protocol can seamlessly connect and interoperate worldwide, fostering a truly global data economy. The standardization process, which typically involves rigorous review and consensus-building, began several years ago, reflecting the long-term strategic vision behind IDSA and Eclipse Foundation efforts.

Eclipse Dataspace Components (EDC): The Technical Backbone

The Eclipse Dataspace Components (EDC) provide the tangible technical building blocks required to implement data spaces in accordance with IDSA requirements and the DSP. These open-source components are designed to be modular and extensible, allowing organizations to assemble tailored solutions for their specific data sharing needs. The core EDC architecture comprises several key elements that work in concert to facilitate secure and governed data exchange:

1. The Federated Catalog (FC): This component acts as an aggregated repository of data asset catalogs gathered from multiple participants within the data space. Instead of querying each participant individually on demand, the FC periodically crawls participants’ data assets and stores metadata in a local cache. This approach significantly improves discovery efficiency and reduces latency, enabling data consumers to quickly find relevant data offerings across the entire data space.
2. The Connector: Often considered the cornerstone of the data space, the connector is the software component that enables secure and controlled data sharing between participants. It acts as a gateway, mediating data transfers and enforcing policies agreed upon during contract negotiation. Its robust design ensures that data moves securely and compliantly between distinct legal entities.
3. The Identity Hub: This component is responsible for managing a participant’s credentials and identity within the data space. It plays a crucial role in establishing trust by providing a mechanism for participants to prove their identity and associated attributes in a verifiable manner.

Amazon Web Services (AWS) provides a comprehensive and highly scalable cloud infrastructure that serves as an ideal platform for deploying and operating these data space components. AWS offers a suite of compute, storage, and networking services that are essential for building secure, compliant, and interoperable data spaces aligned with IDSA standards and the emerging ISO/IEC DIS 20151 specifications. By leveraging AWS, organizations can benefit from a globally distributed, highly available, and secure environment, reducing the operational overhead of managing underlying infrastructure.

Decentralized Trust: Verifiable Credentials and the Decentralized Claims Protocol (DCP)

Establishing trust in a decentralized data space, where participants may not have pre-existing relationships, is paramount. The Decentralized Claims Protocol (DCP) addresses this by serving as an overlay on top of DSP, specifically designed to establish verifiable trust between network participants without relying on a central authority. This protocol leverages the principles of decentralized identity to enable secure and privacy-preserving identity verification.

When an issuer needs to prove their identity or specific attributes to a verifier, they first generate a Decentralized Identifier (DID). A DID is a new type of identifier that is globally unique, persistent, and cryptographically verifiable, giving individuals and organizations greater control over their digital identities. The DID contains information about the issuer’s identity and can be resolved to a DID document, which holds public keys, service endpoints, and other metadata associated with the DID.

The issuer then stores their Verifiable Credential (VC) in their identity hub. A Verifiable Credential is a tamper-proof digital equivalent of a physical credential (like a driver’s license or a university degree). Similar to how a certificate authority validates a public key in traditional PKI, a credential issuer (an authorized entity) validates the DID with specific attributes and signs the VC. When a verifier needs to confirm the issuer’s identity or attributes, they look up the issuer’s DID and, using the information obtained from the DID document, cryptographically verify the VC. This decentralized process ensures that trust is established directly between parties, enhancing security and privacy while minimizing reliance on intermediaries. This mechanism is crucial for enforcing access and usage policies within the data space.

The EDC Connector: Gateway to Data Exchange

The EDC connector, as a pivotal component, is architecturally divided into two primary parts: the control plane and the data plane. This separation of concerns is fundamental to its secure and efficient operation.

The control plane is responsible for orchestrating the "negotiation" aspects of data exchange. This includes handling contract negotiations between data providers and consumers, managing policy enforcement (such as access rights and usage conditions), and sending messages to the data plane to initiate a data transfer once all agreements are in place. It ensures that all legal and contractual prerequisites are met before any data movement occurs.

The data plane, conversely, is solely dedicated to the actual transfer of data. Once the control plane has established the necessary agreements, the data plane takes over, securely moving data from a provider’s EDC connector to a consumer’s EDC connector. This transfer happens across distinct legal entities, emphasizing the importance of robust security measures and adherence to established protocols to protect data integrity and confidentiality during transit.

The EDC connector open-source project, hosted in the Connector repository on GitHub, is structured modularly to promote flexibility and extensibility. It defines how both the control and data planes are implemented through three logical layers:

• SPI (Service Provider Interface): Defines the contracts and interfaces for various functionalities, allowing for different implementations to be plugged in.
• Core: Implements the fundamental business logic and basic services required for the connector’s operation.
• Extensions: Adds specialized functionality and integrations, allowing the connector to adapt to specific environments or integrate with particular services.

The "vanilla" Eclipse EDC connector, by default, provides core functionality but does not bundle cloud provider-specific extensions. To integrate with specific AWS services, such as Amazon Simple Storage Service (Amazon S3) for data storage, AWS Secrets Manager for secure credentials management, or Amazon DynamoDB for metadata persistence, developers need to explicitly include the respective AWS extensions (org.eclipse.edc.aws Maven group) into a custom EDC control and data plane build. This modular approach ensures that connectors remain lightweight and only include the functionalities required for a given deployment scenario.

AWS’s Strategic Role: Powering Data Spaces with Cloud Infrastructure

AWS provides a comprehensive and robust cloud infrastructure that is inherently well-suited to host and scale Eclipse Dataspace Components. The integration of EDC with AWS services allows organizations to build highly available, secure, and cost-effective data space solutions. AWS’s strategic alignment with open standards and its extensive ecosystem of managed services provide significant advantages for implementing IDSA-compliant data spaces.

For deploying the core EDC components, AWS offers:

• Amazon Elastic Container Service (Amazon ECS): A fully managed container orchestration service that simplifies the deployment, management, and scaling of containerized EDC control and data planes. ECS provides the elasticity and reliability needed for dynamic data exchange workloads.
• Amazon Aurora: A high-performance, fully managed relational database service compatible with MySQL and PostgreSQL. Aurora is an excellent choice for persisting metadata required by the EDC, such as contract details, asset catalogs, and policy information, offering superior performance and availability compared to traditional databases.
• Amazon API Gateway: A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. API Gateway can be used to expose the connector’s endpoints securely, managing authentication, authorization, and throttling for incoming requests.
• Amazon Simple Storage Service (Amazon S3): An industry-leading object storage service offering scalability, data availability, security, and performance. S3 is ideal for storing the actual data assets being exchanged within the data space, serving as a reliable backend for data plane transfers or as an Asset Administration Shell "Submodel Server."
• AWS Secrets Manager: A service that helps protect access to applications, services, and IT resources by enabling you to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. This is crucial for managing sensitive credentials required by the EDC connector, enhancing the overall security posture.
• Amazon DynamoDB: A fast and flexible NoSQL database service for applications that need consistent, single-digit millisecond latency at any scale. DynamoDB is an excellent serverless option for storing EDC metadata, especially for use cases requiring high throughput and low latency without operational overhead.

By leveraging these AWS managed services, organizations can significantly reduce the operational burden of running self-managed components, allowing them to focus on data governance and innovation rather than infrastructure management. The inherent security features of AWS, including identity and access management (IAM), network isolation, and encryption, further bolster the trust framework essential for data spaces.

Customizing EDC on AWS: A Practical Implementation Guide

Customizing the EDC connector for native AWS service integration is a strategic move that creates a purpose-built solution capable of fully leveraging AWS’s managed services for storage, security, and scalability. This approach moves beyond relying on generic or self-managed components, opting for deep integration with AWS’s robust ecosystem. The modular, plugin-based architecture of EDC, built on Gradle, facilitates this customization process. The vanilla connector ships only with core functionality, meaning that to integrate with specific cloud services or persistence backends, a custom build that combines EDC’s core modules with the necessary extensions must be assembled.

The customization process primarily revolves around three core concepts:

1. Version Catalog (gradle/libs.versions.toml): This file serves as the centralized registry for all dependencies and their versions within the project. Here, developers declare which EDC modules (e.g., org.eclipse.edc), cloud provider extensions (e.g., org.eclipse.edc.aws), and any other third-party libraries their connector needs. Referencing these at compatible version numbers in a single, consolidated file ensures consistent dependency management across all modules in the project, preventing version conflicts and simplifying upgrades.

2. Launcher Modules: These are the actual deployable units, typically one for the control plane and one for the data plane. Each launcher is a small Gradle subproject with its build.gradle.kts file explicitly listing the extensions to be bundled at runtime. For example, a control plane launcher might include the core control plane module, a metadata persistence extension (such as one for PostgreSQL or DynamoDB), a vault extension (like HashiCorp Vault or AWS Secrets Manager), and any provisioning extensions for the target storage system (e.g., S3). The data plane launcher follows a similar pattern, incorporating data plane-specific modules and extensions. Both launchers commonly utilize Gradle plugins to produce a single executable JAR, ensuring a lightweight and highly tailored connector optimized for specific requirements.

3. Project Settings (settings.gradle.kts): This crucial file registers all modules with Gradle, including the launcher subprojects and any custom extensions developed locally. If a developer creates a proprietary extension, such as a DynamoDB-backed asset store, it would typically reside in an extensions/ directory and be registered in settings.gradle.kts, allowing the launcher modules to reference it as a project dependency.

The resulting project structure typically exhibits a clear organization:

my-connector/
├── control-plane/
│   └── build.gradle.kts       # Control plane launcher configuration
├── data-plane/
│   └── build.gradle.kts       # Data plane launcher configuration
├── extensions/                # Optional: Directory for custom extension code
├── gradle/libs.versions.toml  # Centralized dependency versions
├── build.gradle.kts           # Root build configuration
└── settings.gradle.kts        # Subproject registrations

An open-source reference implementation for EDC customization on AWS is readily available as part of the awslabs/dataspace-connector-on-aws project. This resource demonstrates how Gradle build settings are configured to support services like Amazon DynamoDB for serverless, cost-effective metadata persistence, offering a practical starting point for developers.

The Path Forward: Production Readiness and Cost Optimization

This foundational overview of Eclipse Dataspace Components on AWS marks the initial step in a comprehensive series designed to empower organizations in building secure, interoperable data spaces. Having established the theoretical underpinnings, including the IDSA standards, the Dataspace Protocol (DSP), and the architecture of EDC components like the connector, the subsequent parts of the series will transition to practical implementation and operational excellence.

Part 2 of the AWS Architecture Blog series will delve into production-ready deployment patterns for EDC connectors on AWS. This installment will examine architectural best practices that ensure the security, reliability, and operational efficiency of data space infrastructure. It will cover topics such as high availability, fault tolerance, monitoring, and logging, demonstrating how AWS services can be orchestrated to support mission-critical data exchange. Part 3 will then complete the journey by focusing on cost optimization strategies. This final part will provide practical guidance for running efficient and scalable data space infrastructure on AWS, exploring techniques to minimize expenses while maximizing performance and resource utilization. These subsequent articles will provide the necessary blueprints for organizations to confidently deploy and manage their data space initiatives on a global cloud platform. To get started immediately, interested parties can explore the Dataspace Connector on AWS project on GitHub, specifically examining its Gradle build settings for Amazon DynamoDB integration.

Industry Implications and Future Outlook

The rise of data spaces, facilitated by initiatives like Eclipse Dataspace Components on AWS, signifies a profound shift in how industries will operate and innovate. This paradigm fosters an environment where secure, sovereign, and governed data sharing becomes the norm, unlocking unprecedented opportunities across various sectors.

In manufacturing, data spaces can enable predictive maintenance by securely sharing sensor data from machinery across different vendors, or optimize supply chains by providing real-time visibility into logistics and inventory. In healthcare, they can facilitate anonymized patient data sharing for research, accelerating drug discovery and personalized medicine, while strictly adhering to privacy regulations. The energy sector can leverage data spaces for smart grid optimization, integrating data from distributed energy resources and consumer consumption patterns. For smart cities, data from various urban systems can be combined to improve traffic flow, public safety, and environmental monitoring.

Economically, data spaces are poised to stimulate new business models centered around data as a service, enabling companies to monetize their data assets responsibly and securely. This will foster greater collaboration, reduce market entry barriers for data-driven innovation, and enhance overall economic efficiency. From a regulatory perspective, data spaces, built on robust standards like ISO/IEC DIS 20151 and incorporating mechanisms like Verifiable Credentials, offer a powerful framework for achieving compliance with stringent data protection laws such as GDPR and CCPA. They provide verifiable control over data usage, consent management, and audit trails, simplifying complex compliance requirements.

While challenges remain, particularly in achieving widespread adoption, developing comprehensive legal frameworks, and addressing skill gaps, the foundational work being done by IDSA, the Eclipse Foundation, and cloud providers like AWS is paving the way for a more connected and trustworthy data economy. The vision is a global network of interoperable data spaces, where data flows freely yet responsibly, driving innovation and creating new value for businesses and society alike.

Statements from Key Stakeholders

While specific quotes were not provided in the original excerpt, the initiative itself conveys strong implied statements from the involved parties.

An AWS spokesperson, reflecting on this series, would likely emphasize: "This comprehensive blog series underscores AWS’s unwavering commitment to supporting open standards and empowering our customers to build secure, interoperable data ecosystems. By providing robust architectural patterns and cost optimization strategies, AWS enables organizations to harness the full potential of their data while maintaining sovereignty and control. Our goal is to simplify the deployment of complex solutions like Eclipse Dataspace Components, accelerating innovation across industries."

From the perspective of the International Data Spaces Association, a representative might state: "The collaboration with leading cloud providers such as AWS is absolutely critical for accelerating the adoption of data space technologies and realizing the vision of a trusted, federated data economy. By providing practical implementation guides for deploying IDSA-compliant solutions on scalable cloud infrastructure, AWS significantly contributes to lowering the barrier to entry and fostering broader participation in data spaces globally."

Similarly, the Eclipse Foundation, overseeing the EDC project, would likely highlight: "The Eclipse Dataspace Components represent a cornerstone of open-source innovation for data sovereignty. AWS’s detailed guidance on implementing EDC on their platform is invaluable, demonstrating how open standards and cloud technologies can converge to create powerful, flexible, and production-ready solutions for secure data sharing. This collaboration validates the strength of our open-source community and its role in shaping the future of data exchange."