Introduction
I have previously blogged about the documentation as code principle and how to render Markdown files with Zensical and host them on GH Pages. Those Markdown files are typically geared towards technical stakeholders, e.g., tech leads, architects, and developers. This blog post looks at other documentation sources and how they fit into a cohesive whole.
Documentation sources
In a typical software project, many stakeholders want insight. They range from project managers and business stakeholders who report to the board, to developers who implement, and architects who lay out the direction for developers to follow. Each views the project from a different level of abstraction and complexity. My recommendation is typically to have 3 sources of documentation, each intended for one, and only one, abstraction level:
- Highest level, directed towards upper management who are not very technical, business analysts, and product owners. They use the documentation to get a high-level overview they can use to coordinate with leadership. Examples of sources include Confluence or any other WYSIWYG editor.
- Technical high-level documentation. Typically, tech leads, architects, and developers use this to get an overview of systems, high-level flow, integrations, and domains/sub-domains. The Zensical setup I previously blogged about is such an example.
- Low-level application-specific documentation. This includes README.md files and code comments, though we want to limit comments that just repeat the code. The documentation should make it easy to get going for new developers, e.g., how to set up and run the application locally.
A stakeholder can follow the documentation from the highest level down to individual code lines. Lower-level documentation may change more often. All sources are dynamic and change throughout the service's lifetime, influencing each other bidirectionally.
Example
The board of a power grid company decides to start a new service for charging electric bikes and names it VoltGo. The CTO is in the boardroom and coordinates which systems/sub-domains they can reuse and what they are missing.
Here, the CTO typically works in Confluence, adds a new sub-domain for VoltGo, writes up stakeholders and the mission statement, and links to SharePoint Excel sheets, PowerPoint presentations, and very high-level functionality flows.
Next, the CTO works with various architects and tech leads to map out the functionality, architectural characteristics, non-functional requirements, missing infrastructure, which applications to build, ADRs, and principles. They also reference Confluence where necessary, making it easy to navigate between the two abstraction layers. Let´s say that VoltGo needs to integrate with an existing payment system. This central documentation repo maps out the integration and provides links. We typically want to answer the questions of what and why while providing the boundaries for potential hows/implementations. All of this goes into the central GitHub/GitLab repository. We define the Ubiquitous Language and work with business flows in code together with domain experts.
Developers take the design documentation and start implementing. The individual application repository documents the API contract, configuration, and implementation details. During implementation, we are bound to find new insights and information that will affect the progress or architecture. Maybe our VoltGo implementation needs changes to our CRM solution, for example. That information influences higher-level documentation.
Avoiding drift
I think we've all worked on projects where the documentation is so outdated we simply ignore it. With the 3 abstraction layers, we should notice that the parts that change frequently have less documentation (e.g., code). The code should be self-explanatory, and the documentation should provide context at both the highest level (Business mission statement, for example) and lower levels (architectural characteristics, for example), rather than explaining the code in detail.
I usually interpret a statement like “then we need to remember to keep X and Y in sync” as a warning sign. Those are likely becoming out of sync. Instead, document at a level that stays accurate through frequent, low-level code changes. For example, we do not want to document third-party API details that are already part of their OpenAPI spec. Instead, provide a link. However, one exception to the “sync rule” is to keep the different levels of abstraction in sync. When the business changes an integration or when the implementation finds a breaking change, for example, we need to determine what parts of the documentation are affected and update those. In this case, the product owner and the tech lead need a shared review to synchronize. However, in general the different abstraction layers should be able to change without constantly keeping the other layers in sync. For example, changing the payment provider might affect integration documentation while leaving the business mission unchanged.
Conclusion
With appropriate, explicit documentation levels, we enable different stakeholders to contribute throughout our projects while keeping them informed. They can contribute at their level, bringing insights to other types of documentation and ultimately improving overall software quality.