This move to decentralise is far more than only structural reshuffling. It is a strategic imperative to bring data closer to those who need it most. The approach fosters enhanced collaboration and cultivates a fundamentally deeper understanding of business needs and opportunities. This directly empowers data professionals to build more impactful solutions and enables business stakeholders to leverage data more strategically, ultimately driving a higher level of lasting impact.
Decentralisation isn’t about complete independence but about controlled autonomy within a well-defined strategic and architectural framework. While various methodologies have influenced this evolution, the principles of ‘data as a product’ and distributed responsibility, notably championed by concepts like data mesh, have profoundly shaped our current thinking about data management.
This blog post examines the practical lessons learned from operating within these decentralised data structures, focusing on the dynamics of domain teams, the critical role of data maturity, and the pursuit of strategic impact, all drawn from real-world experience.
Data domains and data domain teams
A data domain is fundamentally a logical grouping of data tied to a specific business area or concept, defined by its ownership, usage, and inherent business meaning. Ideally, these data domains align one-to-one with core business functions (often referred to as business domains in a data context, e.g., production, sales, finance, HR, and logistics). Direct alignment with functional business domains offers significant benefits, fostering clarity and speed by minimising the need to cross organisational boundaries for subject matter expertise, guidance, and prioritisation.
However, this straightforward alignment isn’t always achievable. Concepts like ‘customer’ or ‘product’, for instance, often span multiple business functions. Furthermore, difficulties arise because functional business structures don’t always align perfectly with business responsibility, especially in organisations with distinct business units, local subsidiaries, or strong process ownership designed to break down functional walls.
Given these complexities, while a logical data domain might be cross-functional, the decentralised domain team responsible for it often finds it more practical to operate largely within an existing organisational silo. This is due to the practicalities of leadership, target setting, and funding, which are typically anchored within these silos (e.g., specific business areas or units), making cross-silo accountability and shared funding notoriously complex to manage. Regardless of a domain team’s organisational placement, data connections and shared value areas between different data domains will always exist.
For a dynamic data ecosystem, it is essential to have clear definitions for both what constitutes a data domain and the domain team stewarding it. The responsibilities of these teams, and the processes for their formation and dissolution, must be well-defined and commonly understood. An uncontrolled increase in the number of domain teams, or new ones forming without clear oversight, can lead to a fragmented landscape that complicates collaboration and negatively impacts the overall quality of data engineering and architecture. To counteract this and ensure strategic alignment across the organisation, effective ecosystem-level leadership, often guided by a strong enterprise architecture function, is indispensable. No matter how autonomous individual domain teams are, they must operate within these broader architectural guardrails.
Data domain team skills and composition
The expertise and roles needed are closely related to what the team is supposed to do. Therefore, it is often useful to define a vision that, at a minimum, includes business targets, communicates how the domain team will create and deliver value, and provides a roadmap for how the domain team is expected to develop within the next 2-3 years.
A domain team is hardly ever so autonomous that it could shield itself from organisational steering, portfolio management, and governance practices. At a minimum, the budget will come from somewhere and someone (who is probably not a data professional).
Must-have roles in all domain teams:
- Data architect
- Data engineer
- Domain team lead
Data architects and engineers are central to any team. While data engineering is often primary, a dedicated architect is crucial in every domain team to ensure solution quality, data reusability, and effective collaboration across the broader data ecosystem. Similarly, a domain team lead is essential for representing the team in company forums, providing a clear stakeholder contact, and managing external team members, even if the team operates as a group of equals. Without these roles, issues with quality and coordination will inevitably arise.
Additional roles based on team type
Data-as-a-product teams require data owners (for accountability and data quality), domain experts, and possibly data scientists to ensure deep data understanding and purposeful productisation.
Data development teams need to consider even more how they interact with stakeholders and manage the team’s backlog. Engaging many people with varied data understanding is common, since development initiatives are likely going to have their designated stakeholders (e.g. subject matter experts, business users, business analysts, decision-makers, and other internal customers who depend on or are impacted by the outcomes).
It requires strong non-technical skills to manage effectively and a professional understanding of data development to avoid unnecessary delays for the technical work. A scrum master can support the domain team lead, whose focus remains on team leadership, managing the development portfolio, and effective collaboration with business stakeholders.
Specialised roles for impact and scale:
- Transformation designer: Crucial for educating stakeholders/SMEs and accelerating value capture through effective solution implementation.
- BI developers: Necessary for building analytical data products (reports, dashboards)
- Data scientist: For advanced analytics or complex datasets.
- Product owner: Consider for teams nearing 10 people, to support the data domain lead. Essential for scoping and specifying development initiatives together with stakeholders. A conceptual understanding of data, together with process expertise, helps bridge potential gaps between designers and data architects and engineers.
Key considerations for domain team leaders:
- Minimum size: Determine the critical minimum team size relative to its purpose to ensure flexibility and efficient skill utilisation.
- Agility & focus: Decentralisation’s one benefit lies in focused and agile teams. If a team becomes too large, it may lose this purpose.
- Document business logic: Ensure business logic is thoroughly documented and internally understood, avoiding too high reliance on external consultants.
Learnings from decentralised data organisations
I wanted to write this blog post because centralisation is often seen as ‘bad’ and representing the past. While this might be true, it may not have as strong an implication on the path to action as we might think. Nothing is inherently bad or good. It can be very problematic and difficult to be successful if we pursue one thing because we dislike the alternative.
We also need to consider that our opinions of data professionals are influenced more by other data professionals than by people from other fields. Our practical and solution-oriented approach can lead us to jump to conclusions, especially if there is a promise of getting rid of things that cause us headaches. Unless we enable something that wasn’t possible before, we are probably just trading old problems for new ones.
To be unequivocally clear, this post advocates strongly for decentralised data organisations as the most effective path forward for modern enterprises. The subsequent discussions of challenges aren’t meant to deter, but to provide an honest assessment of the complexities involved and share insights for navigating them successfully.
Before deciding to proceed with your data organisation, you need to truly consider your data maturity and how data-driven the organisation genuinely wants to be. Not the idealised version where you want more of everything that is good and nice, but the one where you need to ‘walk the talk’ more than a few extra miles. Decentralising your data organisation will bring data work to new people and new tables.
Cultivating domain expertise is a key benefit of a decentralised data organisation
The primary rationale for decentralising the data organisation in the first place is to bring data development closer to the people who need the data and are using the solutions being built. This means getting closer to business needs, solving the right problems, and better understanding the users. It brings domain expertise close to data engineering and development.
From the perspective of the domain team, there are two additional benefits related to skill and speed of delivery. Firstly, when you have a domain-specific team, you can recruit people who have previous experience and strong expertise relevant to that team. For a finance domain team, this means developers already understand terms like balance sheet or EBITDA, significantly smoothing collaboration and saving stakeholders from repeated, lengthy explanations of complex concepts. It will save valuable time and make stakeholders feel they are being understood.
Secondly, a dedicated domain focus fosters deep knowledge of specific data, systems, and tools. In complex corporate environments, developers can take up to six months to reach full productivity in a new domain. Constant switching across contexts could lead to never reaching a level of high productivity.
Imagine a conversation where one person cannot explain what they need, and the other person cannot understand the context of what the first person cannot explain. Cultivating domain expertise is critical. Its absence results in costly misunderstandings, prolonged scoping, and ultimately challenges the validity of your business cases.
Cooperation between domain teams isn’t as easy as you might think, but essential for success
When you are working in a domain team, you are focused on the work you are doing, be it building data products, dashboards, collecting user insight, or something else. Your primary focus won’t be looking around at what other domain teams are doing. However, connections will exist in the data and between the data solutions that the different teams are building. For example:
- Using the same data from a slightly different perspective or for another solution
- Needing data that is part of another domain team’s area of responsibility and not yet available on the data platform
- Multiple stakeholders from different domains sharing a business need
This interconnectedness underscores the need for a dedicated data architect within each domain team. Without someone overseeing data-level collaboration, organisations risk duplicated effort, conflicting timelines, and costly refactoring when data is designed from a single perspective. While non-technical roles like the domain team lead, product owners, and scrum masters are crucial for general collaboration, their expertise and capacity are typically insufficient for architectural topics.
Crucially, not all data consumers have the same requirements. This impacts the necessary characteristics and governance of the data. For instance, enterprise-level reporting, regulatory compliance, or foundational master data demand highly stable, rigorously defined, and formally governed data structures and policies. This ensures consistency, auditability, and long-term reliability. In stark contrast, data for locally agile domain team exploratory analytics or rapid prototyping thrives with more flexible, iterative data models and less stringent governance. Here, speed and adaptability are prioritised. The domain architect is therefore instrumental. They must distinguish between these varying consumption patterns and design data solutions with appropriate levels of rigour and flexibility. This ensures both agility, where needed and the requisite stability for critical data assets.
When multiple stakeholders from different domains connect to the same business need, it can become a question of who pays for what, but also one of conflicting interests and priorities. Significant efficiency and value loss can occur if the ability to fund data initiatives holistically is lacking. This can manifest as fragmented data solutions addressing the same business need but catering only to a single domain’s stakeholders, where collaboration and common understanding get sidelined by an ‘I need this now’ mentality.
These collaborative challenges aren’t arguments against decentralisation itself. Rather, they are critical design considerations for its successful implementation. The benefits related to domain-specific expertise, increased agility, and direct business alignment are substantial. The key lies in proactive design and effective mechanisms to manage these cross-domain interactions, ensuring collaboration enhances rather than hinders progress in a decentralised data ecosystem.
To ensure effective cross-domain cooperation, each team should:
- Share its roadmap/portfolio within the data ecosystem. The roadmap should include all active initiatives, not only those in active development. If only active developments are considered, you are communicating things too late from a planning and portfolio management perspective. A significant difference exists between identifying dependencies on the go versus having them as an input in schedule and resource planning.
- Describe the business rationale (what is being done and why) for each data initiative. It is much too hard to spot dependencies if the context around them isn’t understood.
- Describe what data and source systems an initiative touches, especially for new data not yet on the platform. In the pre-development phases, you might not yet be able to, e.g., list data tables, but already describing what kind of data you are working with can help others spot dependencies towards you.
- Carry architectural responsibilities proactively when it comes to data, information, and solutions. A domain team should align with data and enterprise architectural guidelines and frameworks, not invent its own practices. The value of data architecture isn’t always visible to stakeholders and decision makers, which can cause eagerness to take shortcuts. Be prepared to explain the value and importance of this in a way that everyone can understand.
- Have a clear understanding of responsibilities between the domain teams themselves and in relation to other teams, e.g. data platform team and IT. This will be helpful guidance for both the domain team and decision makers supporting and steering it.
- Work with a dedicated data ecosystem coach who facilitates cooperation among all data teams. The coach should be highly proactive and easily available for short, informal discussions.
Decentralisation and the reality of data maturity
Transitioning to a decentralised data organisation almost always means shifting from a centralised one. This inevitably confronts the well-known problem of the data relationship between business and IT. Fundamentally, data is and should be a business topic, not merely an IT concern. Historically, business leaders have often delegated ‘data stuff’ to IT. This reality highlights why traditional organisational data governance frameworks are often abandoned and considered too bureaucratic or theoretical for enabling true value creation. Instead, it makes far more sense to adopt a principle-based, locally flexible federated governance model. This approach delivers governance where it is truly needed for practical efficiency, making it more non-invasive and ‘data-meshy’ by focusing on what governance needs to achieve, rather than simply creating new organisational roles.
Decentralisation pushes data development closer to subject-matter expertise and, critically, compels business domains to take unprecedented responsibility and accountability for data. This fundamental shift, a ‘Trojan horse of business impact,’ transforms data into a central part of business discussions, driving strategic value creation beyond the ‘ticket-like’ needs often served by centralised, IT-driven models focused on loudest voices rather than holistic value.
However, it is crucial to understand that data maturity isn’t a singular, monolithic organisational state. Different departments, business units, or even specific data domains will possess varying levels of capability, readiness, and appetite for data ownership. Therefore, a successful decentralised model doesn’t imply an immediate, uniform shift to full autonomy for everyone. Instead, effective decentralisation often involves a hybrid approach, where certain foundational data services or areas of lower domain maturity might temporarily retain more centralised support, while others rapidly embrace full domain ownership.
The real peril isn’t necessarily ‘low’ data maturity in isolation, but rather the failure to acknowledge and adapt to the diverse maturity landscape within an organisation when pursuing decentralisation. Attempting a ‘one-size-fits-all’ decentralisation can lead to unrealistic expectations and overburden unprepared domains. It risks organisations focusing on low-value ‘low-hanging fruit’ and becoming trapped in unproductive initiatives, especially if data professionals and consultants, while highly skilled, lack a deep, business-minded view of the domain’s ‘big picture’. This creates a critical demand for leaders who can shape the strategic data vision within each domain, a significant challenge for every decentralised data organisation, as these people aren’t always readily available.
Maintaining stability and impact is crucial for domain teams to focus effectively on development. While decentralisation aims to grant them autonomy for efficient functioning, a significant risk of ‘overcorrection’ can emerge. This often stems from a combination of decision-makers’ potential inexperience in leading data development, and IT’s typically strong, controlling nature. An example of this ‘overcorrection’ is when IT rolls out excessive non-technical requirements and responsibilities that conflict with the domain team’s goals. Such actions can significantly slow down domain teams and pressure steering members toward micromanagement, undermining the very efficiency and autonomy decentralisation seeks.
Proactive educating data decision-makers is crucial for success. Without it, there is a high risk of ‘pouring water on too big a surface’, creating fragmented, locally valuable solutions that lack broader strategic impact. This will become a problem for the domain team, since demonstrable strategic impact is often required to sustain funding. Incremental improvements, even if measurable, may be too diluted to justify investment. Value from data isn’t always as visible as we would hope, and it is often about loss of opportunity, and it hurts less to lose money you never had. In the long run, all data teams, no matter how they are organised, must prove, beyond doubt, that data delivers significant strategic added value.
Key considerations for data domain team leaders to ensure success and strategic alignment:
- Align the vision: Develop a strategic data vision for your domain. This vision should articulate the desired future state and tangible business impact that you will enable, directly aligning with organisational goals and metrics. To foster supportive engagement, it needs to clearly communicate the why (strategic purpose and value), how (approach and required capabilities), and what (key outcomes and deliverables) of the data work. Ensuring shared understanding of the value proposition and long-term direction.
- Iterate the vision: Be prepared to revise the data vision as maturity evolves and communicate the likelihood of this happening transparently to stakeholders and steering forums. You should avoid at all costs getting stuck with a vision that doesn’t create the best path forward.
- Portfolio prioritisation: Prioritise domain team work at the portfolio level, not individual initiatives. When the domain team’s vision is reiterated, it should have an immediate impact on your development portfolio and its assigned priorities. For young domain teams (under two years), favour shorter commitments to allow for easier pivots (less resistance from stakeholders) to higher-value work as maturity grows
- Strategic stakeholder management: Recognise that there will always be an abundance of requests, and navigating company politics is to some extent inevitable. To safeguard the team’s focus and portfolio integrity, it is crucial to manage interactions with powerful stakeholders strategically. Rather than having the domain team leader unilaterally deny requests, leverage higher-level authorities to prioritise competing demands. This approach protects the domain team’s capacity, maintains the ability to drive a cohesive portfolio, and prevents getting drawn into unproductive (=lost) political battles. Over-committing to too many items in your portfolio is a clear path to failure.
- Mechanisms for value: Implement processes within domain teams to identify, refine, and deliver valuable data initiatives. Always ensure a clear organisational business need and consider increasing the impact of your data solutions by offering business implementation and change management support.
- Manage expectations effectively: Balance ambition with realistic delivery. Avoid promising too little (perceived incompetence) or too much (struggle to deliver). To prove yourself, you need to show people something of strategic importance that wasn’t possible before.
Want to learn more? Read how to make data work for your business!