I’ve been working on several projects lately and writing a bit about them on my blog – especially, specific technical problems. This very blog entry is on a different level though: it’s about our tooling in Rust.
See, most tools in Rust are wonderful:
rustc
, the official compiler, is pretty easy to use and even though it’s an advanced piece of
software, it’s pretty unlikely you’ll have to use it directly (unless you’re doing something
very exotic, or working on it directly).cargo
, by far the most appealing tool in the Rust ecosystem to me, is a Rust build system and
package manager. It downloads and checks your projects’ dependencies, using https://crates.io as
a default mirror for your crates (but you can
use other mirrors if you want and are brave enough,
even yours), (cross) compiles, runs library, binary and even documentation tests, etc.cargo
also comes with a plugin system, allowing people to write cargo subcommands that can
be installed like with cargo install cargo-tree
and run with cargo tree
. Most common plugins
are:
bloat
.watch
.tree
.clippy
.fuzz
.expand
.outdated
.rustup
, used to select, switch, install and update Rust compilers, toolchains, support
cross-compilation and select targets, etc.Clearly, you can see we have lots of wonderful tools made by talented people and you can clearly feel how easy it is to start writing a crate, test it and publish it along with its documentation. It feels seamless. The documentation is a very important thing in any tech ecosystem for several reasons. And this blog is going to be about doc tooling and some specific parts of Rust.
Whatever the project you work on, you ~should~ must document your code. There are several situations – let’s call this the First Hypothesis:
If you write code that is intended to be released publicly, you must document every public symbols:
impl Default
can be!).Now, if you are writing something that is private to a crate that you plan to release, think about people who will want to contribute to help you with the crate. They will have to get to read your code. They will have to go through the (sometimes painful) process of adapting to someone else’s way to write code. So do them a favor: document your code.
Finally, what about things you don’t plan to release? Like, for instance, a binary? The same rule actually applies: what happens if someone tries to contribute? What if you save the project aside for some weeks and get back to it afterwards?
Those three paragraphs just show you that the First Hypothesis was wrong: there are not several
situations. Only one: as soon as you write code, whatever whom it’s aimed at, just document it.
You’ll be thanked and you might even thank yourself in a near future for doing so. Documenting your
code enables you to run cargo doc --open
and immediately start exploring. Exploring is when you
need to catch up with a project’s ideas, concepts, or whenever you join a team as a new job and need
to get used to the codebase.
I want to share my experience here: most people are bad at this – it’s not necessarily their faults, don’t blame them. Most teams I worked in were under high pressure, with business deadlines to meet – I’ve been there and still am. This is more about a political discussion to have with “the ones who give you such deadlines” but really, developing a project doesn’t stop at writing code and testing. Onboarding and discoverability should be considered of a massive importance, because it helps preventing the project from burying itself and getting too bloated for newcomers or even long-running team members to understand the actual heck is happening there. When someone new joins your team, you should help them to get accustomed to the codebase… but you also have to work. They should have leads or hints to get their feet wet. Several solutions exist to document that in kind of standardized ways:
README.md
in all the projects your team has. This should be mandatory. Describe what
the project is about, how to build it as a developer, how to run it as a dev-ops. Describe the
architecture, etc.CONTRIBUTING.md
to help onboarding newcomers with your team guidelines, conventions,
engineering processes, etc.README.md
: write it!README.md
of your team. This is a real plus: people
will just go to that documentation link to search for things “that have already been done by
the team” instead of reinventing the wheels – trust me, even the ones who wrote the features
might have forgotten writing them already.We’re now reaching the point of this blog entry: documentation can (should?) be used to explore what a project is about, how to use it, what are the public variants and invariants, etc. But there is however a problem in the current Rust ecosystem, preventing us from completely have a comfortable exploration of a crate or set of crates. A very, very important kind of public symbol is missing from the list of documented Rust symbols. Do you think you can figure out which one?
Have you ever worked with a customized crate? If you don’t know what I’m referring to, I’m thinking about crates that can be configured with features. For instance, as a good example of this, I’ll talk about a crate of mine: splines. The crate has several features you can set to achieve different effects:
"serialization"
: this feature will turn on the serde serialization code (both Serialize
and Deserialize
) for all the public types exposed by splines."impl-cgmath"
: turn this on to have some cgmath’s types be usable directly within splines
by implementing the appropriate traits from splines."impl-nalgebra"
: same thing as "impl-cgmath"
but for nalgebra."std"
: automatically enabled, you can disable it with default-features = false
or
--no-default-features
. When disabled, the code behaves compiled with no_std
.All of this is currently documented in the README.md
of the project and the top-level
documentation of the project (for instance, splines-0.2.3
has this paragraph about features and
what they do).
We have two problems here:
rustc
and cargo
!You can see that the situation is not really comfortable. I looked into the list of PRs and issues about that topic on rust-lang/rfcs and didn’t find anything. So I might start to write some ideas here and maybe, depending on the feedback, write a new RFC to fix that problem.
In order to know what modification we can do without introducing breaking changes, we need to have
a look at how features are defined. This is done in a Cargo.toml
manifest, such as (from
splines):
[features]
default = ["std", "impl-cgmath"]
serialization = ["serde", "serde_derive"]
std = []
impl-cgmath = ["cgmath"]
impl-nalgebra = ["nalgebra"]
A feature is a key-value object where the key is the name of the feature and the value is a list of dependencies (crate names, crate cluster, a feature of another crate, etc. – more about that here).
We could document the features here. After all, that’s what Haskell does with cabal
and flags:
Flag Debug
Description: Enable debug support
Default: False
So why not simply do the exact same in Rust within a Cargo.toml
manifest? Like so:
[features]
default = ["std", "impl-cgmath"]
[features.serialization]
description = "Set to automatically derive Serialize and Deserialize from serde."
dependencies = ["serde", "serde_derive"]
[features.std]
description = "Compile with the standard library. Disable to compile with no_std."
dependencies = []
[features.impl-cgmath]
description = "Implement splines’ traits for some cgmath public types."
dependencies = ["cgmath"]
[features.impl-nalgebra]
description = "Implement splines’ traits for some nalgebra public types."
dependencies = ["nalgebra"]
In order to introduce this feature in a retro-compatible way, using the “old” syntax would just behave the same way but without a description, making them optional.
The whole purpose of this addition would be that the rustdoc team could assume the existence of an
optional description
field on features and present them when browsing a crate’s documentation,
enriching the exploration and discovering experiences of developers. Instead of having to look at
both lib.rs
and Cargo.toml
to look for features, one could just simply go on https://docs.rs,
look for the crate they want to get features information et voila!
Some people might think of the consequences of features. Those are, after all, used for
conditional compilation. What that means is that part of the code might be hidden from the
documentation if it’s feature-gated and that the feature wasn’t set when generating the
documentation. An example with splines here.
You can see a few implementors of the Interpolate
trait (some basic types and some from cgmath
because the "impl-cgmath"
feature is enabled by default) but not the potential ones for
nalgebra, which is somehow misleading – people don’t know they can actually use nalgebra with
splines!
A workaround to this problem is to get a local copy of the documentation, generating it with the wished features. However, this will not fix the problem of the discoverability.
A solution to this could be to simply “ignore” the features while generating the documentation –
and only while generating it but annotate the documented symbols with the features. That would
enable something like this (generated with cargo doc --open --no-default-features
and retouched
to mock the idea up):
If several features are required, they would just be presented as a list above the annotated item.
That’s all for me today. Please provide your feedback about that idea. If you like it, I’ll write an RFC because I really think it’s a missing thing in the documentation world of Rust.
Keep the vibes!