Accelerate Resilience Adoption

Reliably helps teams and organizations use chaos engineering to build trust and confidence in their operational process.

Discover how Start for free

CI/CD
Observability
Alerts
and paging
Incident
management
SRE
Reliably

Reliably

Experiments

Verify latency
Verify TLS is valid
Terminate Kubernetes pods
Trigger a failover of the staging DB
Simulate a random AZ outage

Schedule

Integrations

Slack
Honeycomb
Datadog
PagerDuty

MTTR: 37min
Remaining error budget: 92%
Median DB latency: 317ms
Login availability: 99.1%

1
Reliably App 2 minutes ago

Experiment is finished with status deviated

Platform Understand and investigate
vulnerabilities and their impact

Reliably makes chaos engineering experiments more powerful and easier-to-use. Relying on the power and versatility of the open source Chaos Toolkit, Reliably gives you new tools and opportunities to better understand how your system works and reacts.

Create a new plan

Deployment

Chaos Toolkit AWS Aware

Experiments

Losing an AZ should not impact availability
Losing an AZ should not impact latency
Losing an AZ should trigger a PagerDuty alert
Losing an AZ should trigger a Datadog alert

Schedule

now

Submit

Completed 7 days ago

Schedule: now
Via agent: No
Environment: otel
Deployment: cloud

Losing an AZ should not impact availability +1 Reliably Plan Run #32: Commit a2160c2 pushed by Reliably

20 hours ago 5m 30s

Plan and schedule

Use Plans to setup an environment and schedule your experiments with CRON expressions. We'll take care of the rest.

Run from everywhere

Chaos Toolkit experiments can run pretty much everywhere, and play nicely with your CI/CD. With Reliably you can also run them as GitHub Actions or in the Reliably Cloud, with no setup or installation needed.

Integrate with anything

Connect Reliably with your favorite observability platform using OpenTelemetry, or send messages to Slack for each action. You can also use Slack messages as annotations in your execution timeline, making it easier to understand how your team reacted to events.

High-performing teams all have one thing in common: engineers and developers who feel confident navigating the system to identify and fix bugs and anomalies. Reliably's features make it easier than ever to understand how your system works and reacts.

Pause. Observe. Resume.

Pause experiments on-the-fly or at pre-determined breakpoints to investigate during experiments or to run GameDays. And if something unexpected happens, you can now stop experiments completely.

Execution timeline

Explore your executions journals in a more visual way and get relevant results for each probe or action.

Starters

Ready-to-use experiments allow to get started with chaos engineering faster than ever.

Execution has been paused

[2023-02-15 01:00:34,226] - INFO - Reliably execution: b938bd01-4eb4-48cf-956b-a34bc93b6b68
[2023-02-15 01:00:34,452] - DEBUG - Starting Reliably state checker now
[2023-02-15 01:00:34,453] - INFO - Steady-state strategy: default
[2023-02-15 01:00:34,453] - INFO - Rollbacks strategy: always
[2023-02-15 01:00:34,454] - DEBUG - Applying before-control 'reliably' on 'experiment'
[2023-02-15 01:00:34,454] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_experiment_control'
[2023-02-15 01:00:34,455] - DEBUG - Applying before-control 'reliably-integration-slack' on 'experiment'
[2023-02-15 01:00:34,455] - DEBUG - Control 'before_experiment_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:34,833] - DEBUG - Applying before-control 'reliably-integration-opentelemetry' on 'experiment'
[2023-02-15 01:00:34,833] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py' does not declare 'before_experiment_control'
[2023-02-15 01:00:34,833] - DEBUG - Running steady-state hypothesis before the method
[2023-02-15 01:00:34,834] - INFO - Steady state hypothesis: Capture certificate and perform the checks
[2023-02-15 01:00:34,834] - DEBUG - Applying before-control 'reliably' on 'hypothesis'
[2023-02-15 01:00:34,834] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_hypothesis_control'
[2023-02-15 01:00:34,834] - DEBUG - Applying before-control 'reliably' on 'hypothesis'
[2023-02-15 01:00:34,834] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_hypothesis_control'
[2023-02-15 01:00:34,834] - DEBUG - Applying before-control 'reliably-integration-slack' on 'hypothesis'
[2023-02-15 01:00:34,834] - DEBUG - Control 'before_hypothesis_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:35,205] - DEBUG - Applying before-control 'reliably-integration-opentelemetry' on 'hypothesis'
[2023-02-15 01:00:35,205] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py' does not declare 'before_hypothesis_control'
[2023-02-15 01:00:35,205] - DEBUG - Applying before-control 'reliably' on 'activity'
[2023-02-15 01:00:35,206] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_activity_control'
[2023-02-15 01:00:35,206] - DEBUG - Applying before-control 'reliably' on 'activity'
[2023-02-15 01:00:35,206] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_activity_control'
[2023-02-15 01:00:35,206] - DEBUG - Applying before-control 'reliably-integration-slack' on 'activity'
[2023-02-15 01:00:35,206] - DEBUG - Control 'before_activity_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:35,449] - DEBUG - Applying before-control 'reliably-integration-opentelemetry' on 'activity'
[2023-02-15 01:00:35,449] - DEBUG - Control 'before_activity_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py'
[2023-02-15 01:00:35,449] - INFO - Probe: capture-certificate
[2023-02-15 01:00:35,556] - DEBUG - Activity 'capture-certificate' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/activities/tls/probes.py'
[2023-02-15 01:00:35,716] - DEBUG -   => succeeded with '{'conn': {'version': 'TLSv1.3', 'remote_addr': '75.2.60.5:443'}, 'cert': {'fingerprints': {'md5': '0c3464044639adcbc8b94772979f3778', 'sha1': '939f0c65e3e7e3121c1524abfa679facd7806d9f', 'sha256': '0bd6e86c696cfaed1ac97d5f6849a5792c18620c3c8389d0bd98b6fe618e8135'}, 'extensions': {'subjectAltName': ['reliably.com', 'www.reliably.com']}, 'issuer': "CN=R3,O=Let's Encrypt,C=US", 'subject': 'CN=reliably.com', 'serial_int': 262672396126344892037722005403494505859647, 'version': 'v3', 'signature_hash': 'sha256', 'not_valid_before': '2023-01-26T10:10:15', 'not_valid_after': '2023-04-26T10:10:14'}}'
[2023-02-15 01:00:35,837] - DEBUG - Applying after-control 'reliably' on 'activity'
[2023-02-15 01:00:35,837] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_activity_control'
[2023-02-15 01:00:35,837] - DEBUG - Applying after-control 'reliably' on 'activity'
[2023-02-15 01:00:35,837] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_activity_control'
[2023-02-15 01:00:35,837] - DEBUG - Applying after-control 'reliably-integration-slack' on 'activity'
[2023-02-15 01:00:35,838] - DEBUG - Control 'after_activity_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:36,305] - DEBUG - Applying after-control 'reliably-integration-opentelemetry' on 'activity'
[2023-02-15 01:00:36,305] - DEBUG - Control 'after_activity_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py'
[2023-02-15 01:00:36,306] - DEBUG - allowed tolerance is {'name': 'verifiy-tls-certificate', 'type': 'probe', 'provider': {'func': 'verify_tls_cert', 'type': 'python', 'module': 'chaosreliably.activities.tls.tolerances', 'arguments': {'issuer': '${reliably_issuer}', 'alt_names': '${reliably_alt_names}', 'expire_after': '${reliably_expire_after}', 'fingerprint_sha256': '${reliably_fingerprint}'}}}
[2023-02-15 01:00:36,306] - DEBUG - Activity 'verifiy-tls-certificate' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/activities/tls/tolerances.py'
[2023-02-15 01:00:36,306] - DEBUG - Cert expires on 2023-04-26T10:10:14
[2023-02-15 01:00:36,306] - DEBUG - Alt names: {'www.reliably.com', 'reliably.com'} / Expected alt names: ['']
[2023-02-15 01:00:36,306] - DEBUG - Applying after-control 'reliably' on 'hypothesis'
[2023-02-15 01:00:36,307] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_hypothesis_control'
[2023-02-15 01:00:36,307] - DEBUG - Applying after-control 'reliably' on 'hypothesis'
[2023-02-15 01:00:36,307] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_hypothesis_control'
[2023-02-15 01:00:36,307] - DEBUG - Applying after-control 'reliably-integration-slack' on 'hypothesis'
[2023-02-15 01:00:36,307] - DEBUG - Control 'after_hypothesis_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:36,732] - DEBUG - Applying after-control 'reliably-integration-opentelemetry' on 'hypothesis'
[2023-02-15 01:00:36,733] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py' does not declare 'after_hypothesis_control'
[2023-02-15 01:00:36,733] - CRITICAL - Steady state probe 'capture-certificate' is not in the given tolerance so failing this experiment
[2023-02-15 01:00:36,733] - WARNING - Rollbacks were explicitly requested to be played
[2023-02-15 01:00:36,733] - INFO - Let's rollback...
[2023-02-15 01:00:36,734] - DEBUG - Applying before-control 'reliably' on 'rollback'
[2023-02-15 01:00:36,734] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'before_rollback_control'
[2023-02-15 01:00:36,734] - INFO - No declared rollbacks, let's move on.
[2023-02-15 01:00:36,734] - DEBUG - Applying after-control 'reliably' on 'rollback'
[2023-02-15 01:00:36,734] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_rollback_control'
[2023-02-15 01:00:36,734] - INFO - Experiment ended with status: failed
[2023-02-15 01:00:36,734] - DEBUG - Applying after-control 'reliably' on 'experiment'
[2023-02-15 01:00:36,734] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'after_experiment_control'
[2023-02-15 01:00:36,734] - DEBUG - Applying after-control 'reliably-integration-slack' on 'experiment'
[2023-02-15 01:00:36,734] - DEBUG - Control 'after_experiment_control' loaded from '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py'
[2023-02-15 01:00:37,232] - DEBUG - Applying after-control 'reliably-integration-opentelemetry' on 'experiment'
[2023-02-15 01:00:37,232] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py' does not declare 'after_experiment_control'
[2023-02-15 01:00:37,232] - DEBUG - Cleaning up controls
[2023-02-15 01:00:37,232] - DEBUG - Cleaning up control 'reliably'
[2023-02-15 01:00:37,232] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosreliably/controls/experiment.py' does not declare 'cleanup_control'
[2023-02-15 01:00:37,233] - DEBUG - Cleaning up global control 'reliably-integration-slack'
[2023-02-15 01:00:37,233] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaosslack/control.py' does not declare 'cleanup_control'
[2023-02-15 01:00:37,233] - DEBUG - Cleaning up global control 'reliably-integration-opentelemetry'
[2023-02-15 01:00:37,233] - DEBUG - Control module '/home/svc/.venv/lib/python3.11/site-packages/chaostracing/oltp.py' does not declare 'cleanup_control'
[2023-02-15 01:00:37,233] - INFO - Finishing Reliably execution...
[2023-02-15 01:00:37,557] - DEBUG - Stopping Reliably state checker now

2500

5000

7500

10000

Experiment starts

Info Starting probes

Probe starts capture-certificate

Probe ends capture-certificate Tolerance Not Met

Info Probes ended Steady State Not Met

Experiment ends

attach_volume

burn_io

count_pods

create_new_nodepool

create_toxic

delete_cluster

fill_disk

get_hosted_zone

http

import_data

kill_microservice

modify_cluster

network_latency

object_exists

pods_in_phase

pods_not_in_phase

query

recover_az

restart_node

scale_microservice

set_fault

start_chaos

verify_tls

Governance Speed up adoption

Reliably gathers all your experiments, results, and configuration in a common space. It makes it easier the share them across your organization and help more teams benefit from each others work.

Share experiments

Teams can create experiments and make them available for everyone else to run. It significantly facilitates the adoption of chaos engineering throught your organization.

Share results

Did an experiment you ran surface rsults that might be of interest to someone else? They can access your execution data and use it to plan their next move.

Share configurations

Create reusable environments to safely store environments variables and make it easier for everyone to run their experiments. It's less time spent getting ready, and more time debugging.

There are known benefits in using open-source software at the enterprise level. They become even more obvious in a sensitive field as chaos engineering. The inherent transparency of open source makes it easier to ensure everything that runs in your environment is compliant with your security and reliability rules.

The open nature of Chaos Toolkit also makes it possible to adapt it to your most specific needs and make it run experiments in environments where other tools might not fit.

Chaos Toolkit is an active GitHub community with 1,600+ GitHub stars and dozens of contributors to the Chaos Toolkit core or one of its 30+ extensions. These extensions allow Chaos Toolkit to run in multiple environments, as well as to interface with your favorite observability, communication or tracing tool.

Chaos Toolkit on GitHub Chaos Toolkit Docs

Kubernetes
AWS
Azure
Oracle Cloud Infrastructure
Google Cloud Platform
Prometheus
Spring
Istio
SaltStack
ToxiProxy
CloudFoundry
Slack
OpenTracing
OpenTelemetry
Humio
WireMock
OpenStack
Instana
Service Fabric
Dynatrace
Grafana

Ready to get started?

Start better understanding how your system works, to make it more reliable and easier to debug.

Create your free account now (no credit card required) and you will be running experiments in minutes.

Get started for free

Simple pricing

Straightforward plans means no bad surprises.

Pricing details

10-minute starting guide

Whether you're already using Chaos Toolkit or not, you could be sending data to Reliably in minutes (for free).

Read the guide

Accelerate Resilience Adoption