Testing your documentation

Out-of-date documentation and examples that aren't working as promised are an unfortunate truth of many technologies. However, these examples can be valuable (black box) regression tests to make sure that we didn't break anything while improving the code, so how can they be used as such? Rust's documentation strings (///) can include executable code snippets—and they can be seen all over the place on https://www.rust-lang.org/learn!

Getting ready

We'll continue to improve the linked list from a previous recipe but focus some more on the documentation. However, the added code will work in any project, so pick one that you want to add documentation to and open it in your favorite editor.

How to do it...

Here are the steps for this recipe:

1. Find a function or struct (or module) to add a documentation string, for example, the new_empty() function of List<T>:

    ///
    /// Creates a new empty list.
    /// 
    /// 
    pub fn new_empty() -> List<T> { 
        ...

2. Use the special (H1) section # Example to provide a cue for the compiler to run any snippet contained in that section:

    ///
    /// Creates a new empty list.
    /// 
    /// 
    /// # Example

3. Now let's add a code example. Since doctests are considered black box tests, we import the struct (only if it's public, of course) and show what we want to show:

    ///
    /// Creates a new empty list.
    /// 
    /// 
    /// # Example
    /// 
    /// ```
    /// use testing::List;
    /// 
    /// let mut list: List<i32> = List::new_empty();
    /// ```
    /// 

4. With that ready, let's see whether the tests work: run cargo +nightly test in the project's root directory. You can see that we cheated a little bit and added tests to the other functions as well:

$ cargo +nightly test
   Compiling testing v0.1.0 (Rust-Cookbook/Chapter01/testing)
    Finished dev [unoptimized + debuginfo] target(s) in 0.86s
     Running target/debug/deps/testing-a0355a7fb781369f

running 6 tests
[...]
   Doc-tests testing

running 4 tests
test src/lib.rs - List (line 44) ... ok
test src/lib.rs - List<T>::new_empty (line 70) ... ok
test src/lib.rs - List<T>::append (line 94) ... ok
test src/lib.rs - List<T>::pop (line 121) ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

5. The code obviously has been augmented with several examples that have been run in this case—is that always what we want? Sometimes, it's all about the output, and adding all of the required imports for the test to successfully run is a pain. Hence, there are options to add to the fenced area (``` inside the fence ```), and ignore will neither compile nor run the code:

/// 
/// A singly-linked list, with nodes allocated on the heap using `Rc`s and `RefCell`s. Here's an image illustrating a linked list:
/// 
/// 
/// ![](https://upload.wikimedia.org/wikipedia/commons/6/6d/Singly-linked-list.svg)
/// 
/// *Found on https://en.wikipedia.org/wiki/Linked_list*
/// 
/// # Example
/// 
/// ```ignore
/// 
/// let list = List::new_empty();
/// ```
/// 
#[derive(Clone)]
pub struct List<T> where T: Sized + Clone { 
[...]

6. By running cargo test again, we see the changes reflected in the output:

$ cargo test
[...]
   Doc-tests testing

running 4 tests
test src/lib.rs - List (line 46) ... ignored
test src/lib.rs - List<T>::append (line 94) ... ok
test src/lib.rs - List<T>::new_empty (line 70) ... ok
test src/lib.rs - List<T>::pop (line 121) ... ok

test result: ok. 3 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out

7. Let's check the HTML output as well: run cargo doc to generate a target/doc/ directory containing all of the CSS/HTML/JavaScript/... required to show the documentation in a local browser. Open target/doc/testing/index.html with your favorite browser:

Note: Replace testing with the name of your project.

8. Let's remove the ugly use statement at the top of the snippet. At that point, it doubles the lines displayed without adding anything—and rustdoc provides a simple way to do that, too. Add # in front of the offending line:

    ///
    /// Creates a new empty list.
    /// 
    /// 
    /// # Example
    /// 
    /// ```
    /// # use testing::List;
    /// let list: List<i32> = List::new_empty();
    /// ```
    /// 
    pub fn new_empty() -> List<T> {
        [...] 

9. Lastly, there are additional ways to configure the testing behavior of doctests. In this case, let's change warnings to errors by denying the warning while ignoring (allowing) unused variables:

#![doc(html_logo_url = "https://blog.x5ff.xyz/img/main/logo.png",
       test(no_crate_inject, attr(allow(unused_variables), 
        deny(warnings))))]

10. One last time, let's check whether the output is what we expect and run cargo doc:

Now, let's see whether we can find out more about how the code works.

How it works...

Rust's documentation is very versatile and allows for variations on doctests that would not be possible to cover in a single recipe. However, the documentation of these tools is also excellent, so, for more details, check out https://doc.rust-lang.org/rustdoc/documentation-tests.html.

What we covered in this recipe is a great way to document structs and functions in your code by adding examples that will be compiled and run on every test run. Not only will these be helpful for your readers and regression testing, but they also require you to think about how the code works as a black box. These tests are executed whenever code (``` in a fence ```) is encountered in the Example section of the documentation. In step 2 and step 3, we create these examples and see the result in step 4 and step 10.

If you are now wondering how some documentation can show a fraction of the code required while it is supposed to be run, step 8 shows the resolution to this riddle: # can hide individual lines while executing them. However, sometimes the code is not executed at all, as step 5 shows. We can declare a section as ignore and this code won't be run (without any visual indication in the output).

Furthermore, these tests can fail just like any other test by panicking (which can be allowed as well) or falling through an assert! macro. All in all, by hiding away boilerplate or other non-essential code, the reader can focus on the important bits, while the test still covers everything.

We've successfully tested our documentation—we can sleep easy and move on to the next recipe.