Glossary

A list of terms used in the Reliably documentation and their definitions.

Action (Experiment)

An action is a particular activity that needs to be enacted on the system under experimentation. For example, it can be terminating Kubernetes pods, or injecting latency in an API call.

Activity (Experiment)

An activity, in an experiment context, is an action or a probe.

Deviation (Execution)

An execution is considered deviated, and its deviated property is set to true if the steady-state hypothesis was executed after the method, but at least one of its probes failed to match the expected tolerance.

Eg. A latency above the threshold, a 500 response code to an API call, etc.

If an execution is «deviated», we sometimes display its status badge as "Deviated". While this is not a proper status, it tells you what you want to know: the experiment completed, and its deviated property's value is true. On the other hand, if the execution has completed and didn't deviate, its status will be displayed as completed.

Probe (Experiment)

A probe is a way of observing a particular set of conditions in the system that is undergoing experimentation.

Rollbacks

Rollbacks are a sequence of actions that revert what was undone during the experiment. They're defined in the experiment and run after all other activities.

Status (Execution)

Depending on if it is running or not, and on how it ended, an execution can have one of the following statuses:

running: Reliably is currently running the experiment;
pause: Reliably is running the experiment, and it has been paused, either by a user or because it reached a pause instruction;
interrupted: the experiment was interrupted before full completion;
failed: one activity reported a failed condition;
aborted: the execution broke for unknown reasons;
completed: the execution ran entirely and is now completed.

Steady-State Hypothesis

A Steady State Hypothesis describes “what normal looks like” for your system for the experiment to surface information about weaknesses when compared against the declared “normal” tolerances of what is measured.

Reliably experiments use the Steady State Hypothesis for two purposes:

as a check before an experiment is run that the target system is in a recognized normal state;
as the template for comparison of the state of your system after the experiment has been run, forming the results provided by the experiment’s report.

Verification

A Reliably experiment whose goal is to measure («verify») that a particular condition in the system meets requirements. It doesn't need to use its method to run chaos-inducing activities.

Eg. Verify that your database responds in under 200 ms.