Empowering Open Data and Beyond: Major Initiative to Boost CKAN Development

Context: 2023 was a landmark year for CKAN, marked by global recognition, innovative projects, pivotal developments, collaborations, and initiatives that have collectively advanced CKAN's mission of fostering open data and empowering communities worldwide. As many of our readers are already familiar with, the CKAN 3.0 Taskforce team is diligently working on CKAN 3.0, aiming to bring significant enhancements to the platform. Here’s a glimpse of the CKAN Taskforce's achievements in 2023: CKAN 2023 - A year of milestones. Now, building upon this momentum, a new initiative has been launched to further enhance CKAN's capabilities. Funded by co-steward Steven De Costa and driven by key developers, this endeavor aims to bolster CKAN's infrastructure, refine core functionalities, and broaden its applicability to diverse user needs. Emphasizing openness and community collaboration, the initiative is committed to ensuring CKAN's enduring status as a pivotal Digital Public Good. Continue reading to delve into the details about the key contributors and focus areas propelling this impactful initiative.

Intro

In a significant move for the CKAN community of users and contributors, Link Digital has announced a new initiative that promises to enhance the capabilities and reach of CKAN, the leading open-source data management platform. Funded directly by Steven De Costa (Link Digital's Founder and Executive Director), this initiative is a robust response to the growing demand for advanced data management and cataloging solutions across government, non-profit sectors, and enterprises.

Investing in openness

Understanding the pivotal role that CKAN plays in open data specifically and data management more generally, Steven De Costa has committed to funding key roles targeting the improvement of the CKAN core and its primary extensions. This investment will support the dedicated efforts of notable CKAN developers Ian Ward and Adrià Mercader over the next 12 months. Additionally, Sergey Motornyuk, Tech Lead at Link Digital and a core contributor to CKAN, will also play a crucial role in this initiative. Their focus will be on refining CKAN to ensure it remains the top choice for open data portals and expands its utility for a variety of data management tasks.

This strategic decision is not just about enhancing the software but is aimed at fortifying the infrastructure for both developers and end users, and doubling down on CKAN as a critical Digital Public Good. By improving the core functionalities and extensions, the initiative seeks to make CKAN more robust, adaptable, and user-friendly, catering to the diverse needs of its global user base.

Paul Walsh, CTO of Link Digital and open data veteran, will coordinate the initiative, aligning it with other ongoing CKAN projects, including the CKAN v3 roadmap. His role will ensure that the efforts of Ian, Adrià, and Sergey are synergistic with the broader goals of the CKAN community, particularly in terms of technical advancements and community engagement.

With the community

The team will continue engaging through existing channels and will introduce a monthly "what we did" discussion on GitHub. This will not only keep the community updated but also allow for broader participation and feedback, ensuring that the developments reflect the community's needs and expectations. Work will feed into the wider CKAN Tech Team, and work of CKAN Product Manager Alex Gostev.

Focus areas

Every few months, the team will outline key focus areas for the coming period. For the first tranche of work, those will be (i) DCAT v3 support, (ii) developer and maintainer experience, and (iii) file management in CKAN.

DCAT v3 support

With DCAT v3 in candidate mode and application profiles being developed in the US and the EU, frictionless support in CKAN is crucial to interoperability in open data ecosystems. We are planning out the implementation and starting work now. This is a key area and we will follow up with further announcements to gather further stakeholder input. Issues for DCAT v3 support will be tracked here.

Developer and maintainer experience

We are targeting specific improvements and new tools around the main CKAN repo and the Docker images to provide a better experience when working on CKAN, both from a developer point of view (like enabling VSCode remote debugging) and from the maintainers one (like automating backports). Issues for developer and maintainer experience will be tracked here.

Enhanced file management

There is much room for improvement in how files are managed in CKAN. Key aspects include better support for large files, separating file management from data resource management, reducing CKAN load when accessing data, and features like resumable uploads and eager data validation. Some great initial work in this direction has been done by Sergey Motornyuk, Oleksandr Cherniavskyi, and Ian Ward. Issues for enhanced file management will be tracked here.

See the associated project boards for each focus area for an idea of the scope of work, and feel free to jump in on Github or Gitter to contribute ideas and feedback.

Looking ahead

This funding initiative is a testament to the power of strategic investment and community collaboration in the open-source ecosystem. By supporting key developers and aligning efforts with the CKAN roadmap, co-stewards Link Digital are setting the stage for substantial improvements to CKAN that will benefit users worldwide.

As we look forward to the innovations and enhancements that this initiative will bring, it's clear that the future of open data is bright, and needed more now than ever.