Documentation... In an AI World

As I wrote about a few weeks ago, I expect that AI-powered software development is here to stay. While I sincerely hope it will be able to move to local models, one way or another, LLMs are going to contribute to the work done by many, many developers… but not all developers.

If you are making SDKs or other types of libraries, you will need to consider how natural and artificial developers will work with those libraries. A key aspect of that is documentation. So, what should documentation look like in an AI world?

Your Documentation Is… Markdown

One way or another, LLMs can process a lot of different types of input. I have been vibe-coding some Pebble Time 2 apps, and LLMs’ ability to read the contents of screenshots is genuinely impressive.

That said, especially if you want to help developers use local models or other less-intensive AI agents, format matters.

• It will take more tokens for an LLM to process HTML generated from Markdown than the original Markdown, simply because that HTML will be more verbose

• It will take many more tokens for an LLM to process a PDF, a Word document, or other complex binary formats… assuming that the LLM does not invoke some tool to convert those formats into something more text-based

• Don’t get me started on having agents read stuff out of PowerPoint slides, videos, etc.

You still can have videos, PDFs, and the like. However, that should be for human consumption, with agents having access to the same material in text-based formats.

Your Documentation Is… Bite-Size

LLMs can wade through long documents. Among the many non-programming uses of LLMs is summarizing long documents, extracting data baked into long documents, and so on.

But, again, if you want efficient use by LLMs, to reduce token usage and therefore make local models more realistic, forcing agents to read long documents is the wrong answer. Ideally, documentation is in smaller chunks, focused on specific topics, and readily identifiable (e.g., by a well-constructed table of contents, by cross-document links).

This is not necessarily a bad structure for humans. But people are in better position to deal with longer works when that’s the most convenient distribution mechanism.

To put it another way, a 4,000+ page PDF is a dreadful choice for LLMs. 200+ Markdown files (1 per chapter) would be much better. A few thousand Markdown files (1 per section) would better yet.

Your Documentation Is… Well-Indexed

Best is a few thousand Markdown files that can be searched.

Right now, there is a cottage industry for tools to help LLMs navigate large projects, and much of that focuses on building up suitable searchable indices for project artifacts like source code.

Rather than expect consumers of your library to wrangle your stuff into those sorts of tools, make sure that you offer some means of agents being able to search the documentation for topics of relevance. That might come “for free” from the CMS or however you publish your documentation. Perhaps you use third-party search services like Algolia, which powers the search for my Jetpack Compose newsletter archives. Perhaps you add a search plugin to your static site generator. There are tons of options, free and paid, for offering search. Pick one!

The good news is that this can benefit ordinary developers, not just agents.

Your Documentation Is… LLM-Shaped

That said, you are going to want to consider having some options that are specifically aimed at agents.

For modest-sized documentation, consider offering it packaged as a downloadable skill that can be plugged into most major coding agent harnesses. That just requires a top-level SKILL.md file, plus optional supplemental material, in a directory. Outside of some YAML front-matter for that SKILL.md file, the content is free-form and up to you to decide. Dealing with versioning is a bit of a challenge nowadays, often requiring plugins, but for documentation that does not change much, skills are still reasonable options.

Or, consider offering an MCP server. The Model Context Protocol (MCP) is mostly thought of in terms of making it easier for agent harnesses to be extended with new tools. However, those tools can be ways of delivering documentation. For example, Alex Styl’s Composables UI library packages documentation in MCP form, with the CLI also serving double-duty as a way to bootstrap projects to use those libraries. Since MCP servers can be hosted, you could create a simple MCP front-end to your searchable documentation and have it hosted in the same place that your documentation itself is. For example, I need to make a note to see if Algolia Public MCP is something that I can use with those newsletter archives. 🤔

Your Documentation Is… More Than Just Your Documentation

What you and I might consider to be “documentation” is a subset of what agents might consider to be documentation.

For example, consider Kotlin’s binary compatibility validation. You might use those ABI dumps to determine where you make backwards-incompatible changes. Agents can use those ABI dumps to rapidly absorb the shape of your API.

Review your project artifacts and see which among them might be useful inputs for agents, even if they are not really something that ordinary developers might use as documentation directly, or even indirectly.

Documentation is not an “either-or” proposition, where you can support people or agents but not both. A lot of making documentation agent-ready is in packaging, leveraging the same content that you create for human consumption but making it easier for agents to use cheaply.