javierbrea
Cypress Fail Fast
Enables fail fast in Cypress, skipping the rest of tests on first failure.
It can be configured to skip all remaining tests in current spec file, in current run, or even in parallel runs.
Table of Contents
Installation
Add the plugin to devDependencies
npm i --save-dev cypress-fail-fastNow, depending on your Cypress version, use one of the next methods:
Installation on Cypress 10 and higher
Inside cypress.config.ts file:
import cypressFailFast from "cypress-fail-fast/plugin";
export default defineConfig({
e2e: {
setupNodeEvents(on, config) {
cypressFailFast(on, config);
},
},
});In case you are using JavaScript, you may explicit the file extension in some cases:
import cypressFailFast from "cypress-fail-fast/plugin.js"Note: This example shows how to install the plugin for e2e testing type. Read Cypress configuration docs for further info.
At the top of your support file (usually cypress/support/e2e.js for e2e testing type)
import "cypress-fail-fast";Installation on Cypress versions lower than 10
Inside cypress/plugins/index.js:
module.exports = (on, config) => {
require("cypress-fail-fast/plugin")(on, config);
return config;
};At the top of cypress/support/index.js:
import "cypress-fail-fast";From now, if one test fail after its last retry, the rest of tests will be skipped:
Limitations and notes
- All spec files will be loaded, even after entering "skip mode", but every tests and hooks inside them will be skipped.
- The
specstrategy does not work in headed mode, because for Cypress events it is like running a single spec, so all remaining tests will be skipped.
Configuration
Environment variables
FAIL_FAST_STRATEGY:'spec'|'run'|'parallel'- If
spec, only remaining tests in current spec file are skipped. This mode only works in "headless" mode. - If
run, all remaining tests in all spec files are skipped (default value). - Use
parallelto provide your own callbacks allowing to notify from one run to the others when remaining tests should be skipped.
- If
FAIL_FAST_ENABLED:boolean = trueAllows disabling the "fail-fast" feature globally, but it could be still enabled for specific tests or describes using configuration by test.FAIL_FAST_PLUGIN:boolean = trueIffalse, it disables the "fail-fast" feature totally, ignoring even plugin configurations by test.FAIL_FAST_BAIL:Number = 1Enable the skip mode immediately upon n number of failing test suite. Defaults to 1.
Examples
CYPRESS_FAIL_FAST_PLUGIN=false npm run cypressor set the "env" key in the cypress.json configuration file:
{
"env":
{
"FAIL_FAST_STRATEGY": "run",
"FAIL_FAST_ENABLED": true,
"FAIL_FAST_BAIL": 2,
}
}Configuration by test
If you want to configure the plugin on a specific test, you can set this by using the failFast property in test configuration. The plugin allows next config values:
failFast: Configuration for the plugin, containing any of next properties:enabled: Indicates wheter a failure of the current test or children tests (if configuration is applied to a suite) should produce to skip the rest of tests or not. Note that the value defined in this property has priority over the value of the environment variableCYPRESS_FAIL_FAST_ENABLED_(but not overCYPRESS_FAIL_FAST_PLUGIN, which disables the plugin totally)_.
Example
In the next example, tests are configured to "fail-fast" only in case the test with the "sanity test" description fails. If any of the other tests fails, "fail-fast" will not be applied.
describe("All tests", {
failFast: {
enabled: false, // Children tests and describes will inherit this configuration
},
}, () => {
it("sanity test", {
failFast: {
enabled: true, // Overwrite configuration defined in parents
},
}, () => {
// Will skip the rest of tests if this one fails
expect(true).to.be.true;
});
it("second test",() => {
// Will continue executing tests if this one fails
expect(true).to.be.true;
});
});Configuration examples for usual scenarios
You want to disable "fail-fast" in all specs except one:
Set the FAIL_FAST_ENABLED key in the cypress.json configuration file:
{
"env":
{
"FAIL_FAST_ENABLED": false
}
}Enable "fail-fast" in those specs you want using configurations by test:
describe("All tests", { failFast: { enabled: true } }, () => {
// If any test in this describe fails, the rest of tests and specs will be skipped
});You want to totally disable "fail-fast" in your local environment:
Set the FAIL_FAST_PLUGIN key in your local cypress.env.json configuration file:
{
"env":
{
"FAIL_FAST_PLUGIN": false
}
}Configuration for parallel runs
The plugin configuration supports defining two callbacks that, used in combination, allow to skip tests in one run when other run starts skipping tests also. Where, or how do you store the "flag" that allows to communicate your runs is in your hands, the plugin does not care about it.
To implement it, the plugin can receive an object with extra configuration as third argument when it is registered in the cypress/plugins/index.js file:
parallelCallbacks: Object containing next properties:onCancel:function()This callback is executed on first test failure that produces the plugin starts skipping tests.isCancelled:function(): booleanIf this callback returnstrue, the plugin skips remaining tests.
These callbacks are executed only when the environment variable FAIL_FAST_STRATEGY is set to parallel.
Here is an example of configuration that would skip tests on many parallel runs when one of them starts skipping tests. It would only work if all parallel runs have access to the folder where the isCancelled flag is being stored as a file (easy to achieve if all of your parallel runs are being executed on Docker images on a same machine, for example). Note that this is only an example, you could also implement it storing the flag in a REST API, etc.
// Example valid for Cypress versions lower than 10. Use config file on Cypress 10
const fs = require("fs");
const path = require("path");
// Flag file is stored in the /cypress folder
const isCancelledFlagFile = path.resolve(__dirname, "..", ".run-is-cancelled");
module.exports = (on, config) => {
require("cypress-fail-fast/plugin")(on, config, {
parallelCallbacks: {
onCancel: () => {
// Create flag file when the plugin starts skipping tests
fs.writeFileSync(isCancelledFlagFile, "");
},
isCancelled: () => {
// If any other run has created the file, start skipping tests
return fs.existsSync(isCancelledFlagFile);
},
},
});
return config;
};Note that this example requires to remove the created file when all of the runs have finished, or tests will always be skipped whenever any run starts again. So, the FAIL_FAST_STRATEGY environment variable should be set to parallel only in CI pipelines where the workspace is cleaned on finish, for example.
Usage with TypeScript
If you are using TypeScript in the Cypress plugins file, this plugin includes TypeScript declarations and can be imported like the following:
import cypressFailFast = require("cypress-fail-fast/plugin");
export default (on: Cypress.PluginEvents, config: Cypress.PluginConfigOptions): Cypress.PluginConfigOptions => {
cypressFailFast(on, config);
return config;
};Note: The example above is only valid for Cypress versions lower than 10. Use the configuration file in Cypress 10.
Tests
To ensure the plugin stability, the current major version is being tested with Cypress major versions 9.x, 10.x, 11.x, 12.x and 13.x, and new releases will be published for each new Cypress minor or major releases, updating the E2E tests.
Minor versions used in the E2E tests can be checked in the devDependencies of the package.json files of the E2E tests:
Even when current major version may work with previous Cypress versions, it is not currently tested, so, to be sure it works you should use:
- Cypress 8.x may work, but it was tested until
cypress-fail-fast7.0.x - If you need Cypress 7 support, use
cypress-fail-fast6.x - If you need Cypress 6 support, use
cypress-fail-fast5.x - If you need Cypress 5 or lower, use
cypress-fail-fast<= 4.x
If you find any issue for a specific Cypress version, please report it at https://github.com/javierbrea/cypress-fail-fast/issues.
Acknowledgements
This plugin has been developed based on the solutions proposed by the community on this Cypress issue, so thanks to all! I hope this plugin can be deprecated soon, as soon as the Cypress team adds native support for this feature. 😃
Contributing
Contributors are welcome. Please read the contributing guidelines and code of conduct.
License
MIT, see LICENSE for details.