Rust and features discoverability

Rust and features discoverability

rust, documentation, features

2018-10-13 23:37:00 UTC, by Dimitri Sabadie


I’ve been working on several projects lately and writing a bit about them on my blog – especially, specific technical problems. This very blog entry is on a different level though: it’s about our tooling in Rust.

See, most tools in Rust are wonderful:

Clearly, you can see we have lots of wonderful tools made by talented people and you can clearly feel how easy it is to start writing a crate, test it and publish it along with its documentation. It feels seamless. The documentation is a very important thing in any tech ecosystem for several reasons. And this blog is going to be about doc tooling and some specific parts of Rust.

Documentation is key to achieve good quality… but not only

Whatever the project you work on, you should must document your code. There are several situations – let’s call this the First Hypothesis:

If you write code that is intended to be released publicly, you must document every public symbols:

Now, if you are writing something that is private to a crate that you plan to release, think about people who will want to contribute to help you with the crate. They will have to get to read your code. They will have to go through the (sometimes painful) process of adapting to someone else’s way to write code. So do them a favor: document your code.

Finally, what about things you don’t plan to release? Like, for instance, a binary? The same rule actually applies: what happens if someone tries to contribute? What if you save the project aside for some weeks and get back to it afterwards?

Those three paragraphs just show you that the First Hypothesis was wrong: there are not several situations. Only one: as soon as you write code, whatever whom it’s aimed at, just document it. You’ll be thanked and you might even thank yourself in a near future for doing so. Documenting your code enables you to run cargo doc --open and immediately start exploring. Exploring is when you need to catch up with a project’s ideas, concepts, or whenever you join a team as a new job and need to get used to the codebase.

I want to share my experience here: most people are bad at this – it’s not necessarily their faults, don’t blame them. Most teams I worked in were under high pressure, with business deadlines to meet – I’ve been there and still am. This is more about a political discussion to have with “the ones who give you such deadlines” but really, developing a project doesn’t stop at writing code and testing. Onboarding and discoverability should be considered of a massive importance, because it helps preventing the project from burying itself and getting too bloated for newcomers or even long-running team members to understand the actual heck is happening there. When someone new joins your team, you should help them to get accustomed to the codebase… but you also have to work. They should have leads or hints to get their feet wet. Several solutions exist to document that in kind of standardized ways:

We’re now reaching the point of this blog entry: documentation can (should?) be used to explore what a project is about, how to use it, what are the public variants and invariants, etc. But there is however a problem in the current Rust ecosystem, preventing us from completely have a comfortable exploration of a crate or set of crates. A very, very important kind of public symbol is missing from the list of documented Rust symbols. Do you think you can figure out which one?

The missing piece to exploring a crate

Have you ever worked with a customized crate? If you don’t know what I’m referring to, I’m thinking about crates that can be configured with features. For instance, as a good example of this, I’ll talk about a crate of mine: splines. The crate has several features you can set to achieve different effects:

All of this is currently documented in the README.md of the project and the top-level documentation of the project (for instance, splines-0.2.3 has this paragraph about features and what they do).

We have two problems here:

You can see that the situation is not really comfortable. I looked into the list of PRs and issues about that topic on rust-lang/rfcs and didn’t find anything. So I might start to write some ideas here and maybe, depending on the feedback, write a new RFC to fix that problem.

Pre-RFC: documenting and exploring crate features

In order to know what modification we can do without introducing breaking changes, we need to have a look at how features are defined. This is done in a Cargo.toml manifest, such as (from splines):

[features]
default = ["std", "impl-cgmath"]
serialization = ["serde", "serde_derive"]
std = []
impl-cgmath = ["cgmath"]
impl-nalgebra = ["nalgebra"]

A feature is a key-value object where the key is the name of the feature and the value is a list of dependencies (crate names, crate cluster, a feature of another crate, etc. – more about that here).

We could document the features here. After all, that’s what Haskell does with cabal and flags:

Flag Debug
  Description: Enable debug support
  Default:     False

So why not simply do the exact same in Rust within a Cargo.toml manifest? Like so:

[features]
default = ["std", "impl-cgmath"]

[features.serialization]
description = "Set to automatically derive Serialize and Deserialize from serde."
dependencies = ["serde", "serde_derive"]

[features.std]
description = "Compile with the standard library. Disable to compile with no_std."
dependencies = []

[features.impl-cgmath]
description = "Implement splines’ traits for some cgmath public types."
dependencies = ["cgmath"]

[features.impl-nalgebra]
description = "Implement splines’ traits for some nalgebra public types."
dependencies = ["nalgebra"]

In order to introduce this feature in a retro-compatible way, using the “old” syntax would just behave the same way but without a description, making them optional.

The whole purpose of this addition would be that the rustdoc team could assume the existence of an optional description field on features and present them when browsing a crate’s documentation, enriching the exploration and discovering experiences of developers. Instead of having to look at both lib.rs and Cargo.toml to look for features, one could just simply go on https://docs.rs, look for the crate they want to get features information et voila!

Features consequences

Some people might think of the consequences of features. Those are, after all, used for conditional compilation. What that means is that part of the code might be hidden from the documentation if it’s feature-gated and that the feature wasn’t set when generating the documentation. An example with splines here. You can see a few implementors of the Interpolate trait (some basic types and some from cgmath because the "impl-cgmath" feature is enabled by default) but not the potential ones for nalgebra, which is somehow misleading – people don’t know they can actually use nalgebra with splines!

A workaround to this problem is to get a local copy of the documentation, generating it with the wished features. However, this will not fix the problem of the discoverability.

A solution to this could be to simply “ignore” the features while generating the documentation – and only while generating it but annotate the documented symbols with the features. That would enable something like this (generated with cargo doc --open --no-default-features and retouched to mock the idea up):

If several features are required, they would just be presented as a list above the annotated item.

That’s all for me today. Please provide your feedback about that idea. If you like it, I’ll write an RFC because I really think it’s a missing thing in the documentation world of Rust.

Keep the vibes!