Experiments

Experiments are a versatile way to describe the actions you want Reliably to apply to your system, such as verifications or full-fledged chaos engineering experiments.

Under the hood, Reliably uses the open-source Chaos Toolkit External link will open in a new tab to run experiments.

Definition

Experiments are described as a single JSON file, and contain the following elements :

• a title
• a description
• a method that contains activities:
  • probes, a way of observing your system’s conditions
  • actions, activities that are enacted on your system (such as fault injection or any kind of turbulence)
• steady-state hypothesizes, which describes what normal looks like:
  • before the experiment, to check the system is in a normal state
  • after the experiment, to compare its initial and current states
• rollbacks, a set of actions that revert what was done during the experiment

Additionally, controls, configurations, and secrets can be provided to actions or probes as operational elements.

An experiment doesn’t necessarily contain all of these elements: if you decide to use Reliably to run a simple verification (an experiment that only measures certain values of your system, such as the latency of a service), you won’t need any method activities or rollbacks and can use the steady-state hypothesis object to run your probes.

For a complete overview of experiments, you can refer to the Chaos Toolkit API reference External link will open in a new tab.

Creating experiments

Before you can use an experiment in a plan, you need to create it within Reliably. This can be done by either importing an existing experiment (as a JSON file) or using a starter (an experiment template where you only need to provide some information).

Starters

Starters are experiment templates. They can either be provided by Reliably or specific to your organization. In both cases, a starter presents itself as a form requesting you to fill in values that are requested for the experiment to run.

This form allows you to create an experiment that will terminate Kubernetes pods. It can be used to simulate the failure of a service.

Once the starter form has been filled, clicking the “Create” button will make it available to your plans, while the “Create and run” button will open the page to create a new plan, with the experiment already selected.

Import

You can import existing or custom experiments by providing Reliably with the experiment definition as a JSON file (or by pasting the content of the file).

Once your experiment is imported, it will be available to your plans.

Pasting or selecting a file will process a quick conformity check of your experiment. While it can detect an experiment that is not properly formed, it cannot guarantee that the experiment will run, or that it will perform the expected actions. Consider it as a linter, not as a validation of your experiment’s outcome.

Executions

Your experiment is now ready to be used by a plan. Each time your experiment is run as part of a plan, a new execution will be generated, and display the execution’s result and journal.