Introduction

hyperlit is a system for embedding documentation in code, thus making it easy to write and maintain documentation for a software project.

Rather than writing documentation in a separate file, you can embed it directly into your code:

Example:

#![allow(unused)]
fn main() {
/*📖
title: Adding new widgets

Here's how to add a new widgets....
*/
}

Advantages

This has the following advantages:

• It's easy to add documentation to existing code
• A higher chance of keeping documentation up to date, since it's part of the code
• It removes the cognitive load of deciding where to put documentation
• Removing source files automatically removes the corresponding documentation, thereby reducing confusion caused by stale documentation

Goals

The primary goals of hyperlit are:

1. Make it easy to add documentation
2. Support for multiple languages
3. Support for multiple documentation formats

Values

The guiding values of hyperlit are:

1. User-friendliness: Tools should be a joy to use
2. Productivity: Documentation should make us more productive, not be a chore
3. Simplicity: Prefer simple tools over complex ones

Observations and conclusions from my personal documentation experience

1. Documentation requires a great search mechanism to be useful

   If your documentation search is great, it does not matter where you put it: flat search beats deeply nested hierarchies

2. The easier it is to write documentation, the more likely it is going to be done

3. The more obvious existing documentation is, the more likely it is going to be kept in sync with the code

4. Rather than putting documentation in its own cupboard, let's put it in the code

Adding a new output backend

#backend #howto Manuel Woelker 2025-06-22T14:21:06+00:00

To add a new output backend, you need to implement the Backend trait.

See mdbook_backend.rs for an example.

backend/src/backend.rs:5

hyperlit.toml configuration file

#config #howto Manuel Woelker 2025-06-22T14:21:06+00:00

The hyperlit.toml configuration file is used to configure the document generation process. It contains the following fields:

• src_directory: root path to source code. This may be the repository root (i.e., ".") to scan all directories in the repository

• src_globs: globs to use when searching for source files, these may be prefixed with "!" to exclude files or directories

• docs_directory: path to the docs directory, this should contain the documentation files

• doc_globs: globs to use when searching for documentation files, may be "*" to include all files

• build_directory: path to a build directory used for temporary files

• output_directory: directory to write the complete documentation output to

• doc_markers: list of marker strings used to identify documentation segments to extract from the source code, defaults to ["📖", "DOC"]

• source_link_template: Template used to generate links to source code (e.g. on github, etc.), placeholders ${path} and ${line} will be replaced

core/src/config.rs:7

Decision records

This section contains decision records for the project.

The purpose of these records is to document decisions that have been made about the project.

DR-0000 Use decision records to document decisions

#decision

Status: Approved
Date: 2025-06-08

Decision

To document the values, structure, architecture and of the project, we will use decision records to document why things are the way they are, and what alternatives have been considered.

Context

Source code mostly conveys the How of a solution, but is often insufficient to explain the Why behind it, and usually never the alternatives that were considered, but ultimately rejected.

A lack of understanding of these "Why's" can lead to incorrect assumptions, duplicate effort, inconsistent structures and behaviors, as well as an overall lack of vision.

Consequences

For all major decisions, we will use decision records to document them.

These decision records will be checked in to the source control repository.

The decision records should be written in a way that is clear, concise and easy to understand.

The format should be:

Title: DR-{sequential number} {title}
Status: {Decisions status, one of: In Progress, Approved, Rejected, Superseded}
Date: {Date of last update in YYYY-MM-DD format}

Decision

{What is the decision?}

Context

{What question does this decision answer?}

Consequences

{What are the consequences of this decision?}

Considered Alternatives

{What alternatives were considered? Why were they rejected?}

Considered Alternatives

Not documenting decisions

Not documenting decisions seemed like a bad idea for a documentation tool:

The reasoning and considerations behind these issues and decisions would not be published a available to other interested parties.

Using architecture decision records (ADRs)

These decision records were originally intended for architecture decisions.

However, there seemed no good reason to restrict this process to architecture decisions only.

This would mean excluding other relevant decisions, such as those related to the development process, or even non-technical decisions.

DR-0001 Use mdbook as first backend to generate documentation

#decision #backend Manuel Woelker 2025-06-22T14:21:06+00:00

Status: Approved
Date: 2025-06-08

Decision

To generate HTML documentation, we will use mdbook as the first backend.

Context

The goal of hyperlit is to create documentation artifacts that are straightforward to read and understand.

The requirements for the initial backend were:

1. Easy to integrate
2. Support for Markdown
3. HTML output
4. Good search capabilities

Consequences

mdbook is the "default" backend for hyperlit.

This backend should be maintained and supported in the future.

Considered Alternatives

Custom backend

A custom backend could be implemented, but it would be more complex to create, maintain and update.

Such a backend may make sense in the future to better support more complex use cases.

typst backend

typst might also work as a backend.

However, it is currently not well established. Its HTML export functionality is still a work in progress.

backend-mdbook/src/mdbook_backend.rs:1

DR-0002 Use syntect to extract doc comments from code

#decision #extractor Manuel Woelker 2025-06-22T14:21:06+00:00

Status: Approved
Date: 2025-06-19

Decision

To extract doc comments from source code, we will use the syntect crate.

Context

To extract doc comments from code, we need to find all the comments in the code, for various languages.

The requirements for this extractor were:

1. Wide support for various programming language formats
2. Robustness against invalid code/syntax
3. Good performance

Consequences

syntect is used to extract doc comments from the code.

To support as many languages as possible, the two-face crate is used.

Considered Alternatives

Custom lexer

A custom lexer could be implemented to find comments. Due to the number of languages and the complexity of handling different syntaxes, this might not be a good idea. Especially handling "comment-like" syntax in strings would potentially mean having a custom lexer for each language.

tree-sitter

tree-sitter parsers could be used to extract the comments from source files.

The drawback is that these parsers need to be curated, are platform-specific and are relatively heavyweight.

inkjet

inkjet bundles ~70 tree-sitter parsers for various languages.

The downside of this approach is that all these parsers need to be compiled (making the compilation much slower) and bundled in the binary (making the binary much larger)

extractor/src/extractor.rs:1