How Runme Makes Project Documentation Interactive and Accessible

The Stateful team has collaborated with the WebdriverIO project to help improve their documentation by making the documentation examples more interactive and accessible using Runme.

The WebdriverIO framework is a versatile tool for browser and mobile test automation that offers a lot of features for you to play around with. The goal of their project documentation is to clearly communicate it's features and how they can be applied to your project. A central contributor to this are code examples, like a picture that can be worth a thousand words.

It's no surprise that embedded code examples in open source project documentation has become increasingly common. Many are even interactive and allow users to fiddle around with the code in real time, e.g. the new React Docs, or provide "playgrounds" with live examples, like on svelte.dev.

Live Examples on the new React docs

Problems with Existing Solutions

There are however, a number of common problems we need to be take into consideration when it comes to code examples in docs:

• They are made up and often don't reflect reality
• Some contain errors because we are all just humans
• Often become outdated as interfaces change
• Don't perfectly fit your needs and can be difficult to apply to your project

In an attempt to improve WebdriverIO's code examples we started rolling out some changes to the documentation that will hopefully addresses these issues:

As you can see, some examples now have two buttons, one brings you to the code on Github, but the other let's you run them. But what does that mean?

Extract Examples from Docs

As a first step we started to remove all code examples from their documentation page and moved them into a dedicated repository. This allows us to treat these examples as code and set-up the necessary infrastructure, e.g. CI/CD or automated dependency updates, to ensure quality and correctness.

So say hello 👋 to this new repository in the WebdriverIO GitHub organization, it now contains all the examples in use by the official docs.

webdriverio/example-recipes

You can see that every example is self contained in its own directory to keep everything very simple. A big list of NPM scripts allows you to run specific examples with just a single command.

In order to embed the examples back into the website, we are using a plugin for Docusaurus that downloads the code based on a simple GitHub reference link. Instead of having code within our markdown files, we just reference the location on Github, e.g.:

The plugin then downloads the code and only shows provided code lines in the file. Here is the final result:

If you are using a different tool for building your static docs, chances are that it has similar plugins available to just do that.

Testing Examples

Now that we have everything nicely encapsulated into a dedicated repository, we can use CI/CD to test all examples on regular basis. A simple GitHub workflow can trigger the execution of these examples and have the pipeline fail if any of them have an error:

name: Test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        exampleDir:
          - click
          # more example directories here
          # ...
          - api/webdriver
    steps:
        - name: Checkout
          uses: actions/checkout@v3
        - uses: actions/setup-node@v3
          with:
            node-version: 18
        - name: Install
          run: npm install
        - name: Test
          run: npm run ${{ matrix.exampleDir }}
          working-directory: ${{ matrix.exampleDir }}

Most of the examples are written as normal WebdriverIO test files and contain normal assertions like any other test would do, e.g. an example that shows how to fetch elements using command chaining would be written as following:

it('should get the text of a menu link', async () => {
    const menu$ = await $('#menu') // or `browser.$('#menu')`
    console.log(await menu$.$$('li')[2].$('a').getText()) // outputs: "API"

    await expect(menu$.$$('li')[2].$('a')).toHaveText('API')
})

With the ability to reference certain code lines we can just strip out the testing part of the example and focus on what's important:

Keep Examples up to Date

Another advantage of having CI/CD infrastructure is the ability to use workflows that ensure everything stays up to date. Since WebdriverIO code is hosted on GitHub, we setup Dependabot to update all dependencies on weekly basis. An additional GitHub workflow helps us to auto-merge these dependency updates so that we only need to deal with them in case they cause issues due to failing tests.

This process is very important and helps us to ensure that changes in WebdriverIO don't break any of the examples we use in our documentation. It is also a great additional feedback loop and creates more confidence when a new version is released that did not break any of the project examples.

Make Examples Easy Accessible

Lastly, to make every example very easy to access and run, we are using a feature of our VS Code Extension called Runme that helps to check out code locally with a simple click on a button. If you haven't installed Runme yet, go check it out on the VS Code Marketplace or find it in your VS Code:

Runme on the VS Code Marketplace

Many surveys have shown that VS Code is the dominant IDE for many developers around the world. With the Run Example button we allow all these users to access the repository with a single click. The button is a simple link with a custom vscode:// protocol that will prompt you for permission to open it in VS Code. There, the extension will pick up the link information containing which repository it needs to check out and which markdown file to open. If the extension is not installed, it will automatically do that for you, if you consent.

Run Example with Runme

Once the repository is checked out, Runme will open a dedicated README.md for the example in an interactive notebook experience. It will explain the example and walks you through it. It allows to execute the code cells within the VS Code terminal securely so that setting up and running the example is done with a simple click and requires no additional application to be opened.

For folks that don't have VS Code installed can still access the repository, check it out manually and run the examples as well.

We believe this is a particularly useful way to provide an interactive and simple workflow to provide code examples for a framework that requires a specific environment that is not a browser, e.g. Node.js. If you are a framework author and run your docs with Docusaurus, feel free to copy this approach if you like it. It's Open Source and free.

Thanks for reading!