Minimum Viable Paperwork (part 2)

There’s relatively little overlap between the things governance boards consider important, and the documentation that developers and technical architects need to do their jobs

If you’ve followed the steps outlined in the previous post in this series, you’ve probably done almost everything you need to satisfy your minimum documentation requirements. A little light cutting-and-pasting, some minor reformatting, and your project will be ready for a glowing review by the governance board. It’s more than likely  that this is all that’s strictly demanded of you by public sector managers.

Stopping here, though, would be a mistake.

But why?

If you’re coming from a software engineering background, the answer will seem obvious: What’s missing from the documentation you’ve assembled so far is, quite simply, all the important stuff. There’s relatively little overlap between the kinds of things governance boards consider important, and the documentation that developers and technical architects need in order to do their jobs. Where are the UML and ER diagrams for the database? Where are the class diagrams? What about READMEs and SwaggerDocs? 

And all these questions become more urgent in the kind of high churn, low continuity environment that characterises much of the Civil Service in recent years. 

Fortunately, having just completed all the governance paperwork, there’s no better time – and, almost certainly, no-one’s in a better position – to start answering them. Completing governance paperwork involves a lot of research: talking to stakeholders, wading through SharePoint, and scanning through workshop notes. Most of what you find won’t end up in the final governance reports. 

But it doesn’t mean all that research time was wasted – or at least, it won’t be if you now make the effort to ensure it’s available to the audiences who need it. And that effort is going to be a lot less (and probably much more satisfying) than what you expended on the often bureaucratic task of satisfying governance.

Create a single site of truth

The first decision you have to make is, simply, where you’re going to keep everything. In the end, you want to have a single location – whether it’s a wiki, a Google Drive directory, or a GitHub repository – where staff can go to look for information. And ideally they should have enough confidence in that location that:

  1. If they don’t find needed documentation there, they know it doesn’t exist and feel empowered to create it themselves in that shared space
  2. If they’re updating a document, they do it there, where everyone can see it, rather than off in isolated environment with limited visibility

Having this single location doesn’t mean that every relevant document has to be in it. Moving a file from a private or restricted location to a shared one will often be the simplest path to making it more widely available. But it’s also useful and acceptable to create links from your shared workspace to more specialised locations, as long as the linked documents are visible to anyone in the organisation who might need them. The point is that everyone knows where to start looking for information, not to ensure that the resource itself is entirely self contained.

Zero-tolerance deprecation 

Almost any project of any age will be accompanied by a litter of electronic files that are, in terms of informational value, useless or worse than useless: 

At best, such documents are simply redundant. At worst, they can be actively misleading. If you can’t clarify their gaps or rationale with existing document owners, it’s usually best simply to delete them. If you’re prevented from doing that by an abundance of caution or departmental policy, at the very least move them into a directory with a name like ‘DEPRECATED – DO NOT USE’ to ensure they don’t lead staff astray. Then make sure you add links back to the single site of truth you established above, so that people know where they should be looking for the information they need.

Mind the gaps

Now that you’ve got a location and populated it with some content, you need to perform a couple of analytical tasks to ensure it’s useful and that it stays updated.

The first is a gap analysis: what’s missing from the documentation you’ve got? If you’re coming from outside the organisation, this will usually be easy to assess, as you’re probably coming to the project with a lot of unanswered questions to begin with. Another simple approach is to ask yourself what a new developer would need to know to be able to hit the ground running with the project. In most cases, this basic thought experiment will help quickly identify at least the most glaring holes in your documentation.

Watch your speed

The next question you need to answer is how fast all the documents – both those which you have and the ones that need to be created – are likely to change. Generally speaking, the closer you get to the implementation layer, the more volatile a document is going to be. Code comments will likely change daily, SwaggerDocs every sprint, architecture diagrams once or twice a year, and so on.

Establish ownership

Once you’ve completed both your gap- and pace-analysis, you’re in a good position to assign ownership of each document – both the ones which already exist, and those which you’ve identified as needing to be created – to the appropriate members of your team. Again, you should be guided here by closeness to implementation and the volatility of the documents.Low level documents liable to rapid alteration should be the responsibility of people who are relatively close to the code, while higher level design documents should be assigned to project managers, business owners, technical architects, and the like.

Seeing the forest, as well as the trees

Past this point, it’s hard to make hard and fast recommendations about what to do next. The kind of additional documentation you provide, and how you do that, will depend on the conclusions of your gap- and pace-analyses.

One anti-pattern commonly encountered in large organisations, however, is that detailed technical areas are documented in some detail, while wider questions of technical architecture and context are not. This can be an extremely costly oversight, leading to an inability to revisit technical decisions, teams operating blind, and cloud components left running because, with the departure of senior people, no-one is aware of either their existence or purpose.

Taking the time to provide these overarching documents is also valuable because they provide a boundary around the problem space, and help clarify the relationships among all the other documents. Providing broad overviews, then, also increases the value of all the other work you’ve done.

If they don’t already exist, then, it’s probably a good idea where possible to provide:

  1. A user manual. It’s astonishing how many public sector software systems don’t have these. In addition to helping end users, software manuals act as a kind of lightweight functional specification for developers – and can provide vital context for coders, product owners and stakeholders. In addition, their very nature helps keep the team user focused – almost always a good thing.
  2. The top 3 levels of a C4 diagram. Trying to complete a C4 diagram is a great way to uncover areas of vagueness or uncertainty in your and the organisation’s understanding. In addition, they degrade gracefully. If you have a really deep understanding of a given area they cover, they allow you to communicate this effectively. And if you don’t – well, you can communicate that, too. In particular, leaving the fourth (‘code’) level of a C4 diagram incomplete gives a useful point of attachment for people close to implementation details to contribute what they know.

In the early stages of assembling documentation, be prepared to accept that the person best placed to create this kind of overview document might well be you. After all, you’re the person who’s carried out tasks like most recently reviewing the existing documents and spotting the gaps. So it’s probably going to be simpler for you simply to create a document yourself than to try to transfer all the information from your head into somebody else’s so that they can write it. After all, ‘information transfer’ is precisely why you’re creating the document in the first place.

But once you’ve created your initial drafts, it’s important to hand their ownership over to the people working closest to the domain they describe. 

Keeping it fresh

Handover can be hard: given that you’ve probably assembled a lot of your governance and technical documentation from a shared drive graveyard of abandoned and forgotten files, how do you know your own laboriously created ‘single site of truth’ won’t suffer the same fate? And how do you guarantee that document owners will actually take responsibility for them?

The simplest way to make sure documentation continues to be updated is to ensure that it’s used. And the simplest way to do that is to embed the documentation you’ve created into the onboarding process. This is particularly important in high churn environments such as the Civil Service, where staff often rotate quickly through several departments. The documentation you’ve created can help ensure new team members get up to speed quickly in their projects. And their feedback can be used to:

As for how you ensure that documentation review and feedback becomes part of the onboarding process – you can apply some of the knowledge picked up during the governance phase of documentation work with your organisation’s stakeholders to get this baked into policy. With a little effort, a virtuous circle can be achieved – with technical documentation adding value to the team, and the team adding value to the documents.

Recap

There are stages to ensuring public sector teams have the documentation they need to understand and change their own systems quickly and capably:

  1. Create a single site of truth, into which you move, or from which you link to, all existing necessary documents
  2. Delete or deprecate alternative sites
  3. Perform a gap analysis, identifying what’s missing from your documentation
  4. Perform a pace analysis, and assign document ownership accordingly
  5. Create a user manual and C4 diagram for any application(s) that lack them
  6. Use your ‘single site of truth’ for onboarding
  7. Ensure feedback from the onboarding process is used to improve documentation
  8. Get steps 6 and 7 encoded as matters of policy

The above might look like a lot of effort. But remember, most of the hard work – digging through documents, understanding what’s relevant and what’s not, understanding the institutional and functional context of the application or service you’re working on – has already been done as a necessary part of satisfying governance requirements. All this is simply rationalising that work to make it useful for a wider audience over the long term. Your team and organisation will thank you for it. And the next time governance requirements need to be satisfied, the task should be a simple one.

In the public sector, that time’s likely to come around sooner than you think. The time you save may be your own.

Featured image by Wesley Tingey on Unsplash