Testing

Evolve tests each behavior in the lowest layer that can prove it. A function gets a unit test, a component gets a story, and only a journey through the shop gets an end-to-end test. This page describes the layers, then the end-to-end tests in detail: how they are tagged, how they set up their data through ops, and how the test-support service plays the back office.

To test your own project, see the Testing guide. To write an end-to-end test, see the end-to-end conventions.

The layers

Layer	What it proves	Tools
Unit and integration	A module, resolver or adapter, against an in-memory commercetools	Vitest, MSW, @labdigital/commercetools-mock
Adapter contracts	A search or payment adapter behaves like every other one	The framework’s contract helpers
Stories	A component renders, responds and is accessible, in a real browser	Storybook, Vitest, axe
End-to-end	A journey works through the deployed shop	Playwright, @evolve-storefront/testing
Crawl	Every content page in the sitemap is accessible	Playwright, axe

The framework and the storefront run their unit, integration and story tests on every push. The adapter contracts are suites a package runs against its own adapter; Build your own adapter shows how. A story runs in a real browser, so most component behavior is proved there and needs no end-to-end test. That is why the reference sites have about seventy end-to-end tests each, and not hundreds.

End-to-end journeys

The end-to-end tests live in each reference site’s frontend/site/testing/. Each test is one journey, written as Given, When and Then steps. Each step is a test.step, and its title is what a reader of the test report sees:

// Shortened from the B2C order fulfillment test.
test.describe("Order fulfillment", { tag: ["@desktop", "@firefox", "@backoffice"] }, () => {
  test("Should follow an order from confirmed to delivered", async ({ page, api, t }) => {
    const customer = await api.run(customers.create());
    await api.run(auth.signIn(customer));
    await api.run(cart.addSkus([SKU], { quantity: 2 }));
    const order = await api.run(orders.place({ email: customer.email }));

    await test.step("Given an order the shopper has just placed", async () => {
      await page.goto(`/nl/account/orders/${order.orderNumber}`);
      await expect(info(page)).toContainText(t("Order.status.OPEN"));
    });

    await test.step("When the warehouse confirms it", async () => {
      await api.run(simulate.confirmOrder(order));
      await page.goto(`/nl/account/orders/${order.orderNumber}`);
    });

    await test.step("Then it is being handled, and nothing has shipped", async () => {
      await expect(info(page)).toContainText(t("Order.status.PROCESSING"));
      await expect(page).toBeAccessible();
    });
  });
});

A few rules hold for every journey:

• The browser drives only the feature under test. The customer, the cart and the order above are created through ops, not by clicking through the shop.
• Copy comes from the message files. The t fixture reads the site’s translations, so a test does not break when a text changes, and fails when a key is missing.
• Every journey ends with an accessibility check. See Accessibility.
• A test is stable before it lands. It must pass five times in a row, with --repeat-each=5.

Tags decide where a test runs

A test without tags runs nowhere, so every test says where it belongs. It needs one device tag and one browser tag:

Tag	Meaning
@desktop, @mobile	The viewport the test runs at.
@chromium, @firefox, @webkit	The browser engine. Each test runs on one, and the engines are spread over the tests.
@smoke	The small set that runs after every deploy to the test environment.
@mail	The test reads an email, from a fresh mailbox per test.
@backoffice	The test needs the test-support service. Where it is missing, the test is skipped, and the report shows it as skipped.
@vendor:<id>	The test covers one vendor only, on a reference site that runs several of the same kind.

The floor journey is the one exception to “one engine per test”. It goes from the home page through search and the product page to checkout, on all three engines. It stops before placing an order, because it also runs against production.

Environments

A run picks its target with E2E_ENV: the deployed test environment, which is the default, or a local stack. A local stack reads mail from Mailpit. The deployed environment needs three secrets:

• The trusted-document bypass. The gateway accepts only registered GraphQL documents, and the ops send their own. The secret lets them through.
• The rate-limit bypass. Signing up is limited per IP address, and a test run signs up many customers. The secret gives each request its own limit.
• The test-support secret, for the test-support service.

Ops

An op is one step of test setup, sent to the shop’s API instead of done in the browser. Each op has a forward, and most have a rollback:

type Op<T> = {
  forward: (ctx: OpContext) => Promise<T>;
  /** Omit it where there is nothing to undo — signing in creates no record. */
  rollback?: (ctx: OpContext, result: T) => Promise<void>;
};

A test runs an op with api.run(op), which returns the result of forward and remembers the rollback. When the test ends, the rollbacks run in reverse order, so the test leaves nothing behind. A rollback that fails, fails its test.

@evolve-storefront/testing/ops holds the ops every storefront needs, such as customers.create, auth.signIn, cart.addSkus, orders.place and quotes.request. Most are GraphQL documents sent through the gateway; the package checks each document against the schema. A reference site adds its own in testing/ops/, such as session.guest() in B2C or the company and approval ops in B2B.

The ops and the browser share one session. After auth.signIn, the page is signed in as well, without a step in between. An op that must not touch the shared session, such as a second sign-up, runs in a throwaway cookie jar of its own: withIsolatedJar.

See @evolve-storefront/testing for every op, fixture and matcher.

The test-support service

Some steps in a journey are not the shopper’s: a warehouse confirms and delivers an order, a sales rep sends a quote, accounting raises an invoice. The test-support service does those steps for a test, and cleans up what a test created.

flowchart LR
  test[Test] -- drives --> browser[Browser]
  browser --> gateway[GraphQL gateway]
  test -- "ops, with the bypass headers" --> gateway
  gateway --> services[Services]
  test -- "simulate and cleanup, with its secret" --> support[Test-support service]
  services --> engine[Commerce engine]
  support --> engine

It is a small service in each reference site’s backend/test-support, built from createTestSupportModule in @evolve-framework/core and the commercetools implementation. It takes POST /<area>/<action>, in two groups:

• simulate: confirm, cancel, ship or deliver an order; send, withdraw, decline or renegotiate a quote; raise, pay or credit an invoice; create a company.
• cleanup: delete an order, a payment, a cart, a quote request, a customer or a company, or remove an associate.

The ops in the simulate and cleanup namespaces call it. The same actions are available on the command line, as evolve simulate and evolve cleanup.

The service is built so it cannot harm real data:

• It does not start unless SIMULATION_ENABLED=true and a test identity are set, and it only deletes data of that test identity.
• Every request needs its secret, as a Bearer token. Without a configured secret, it refuses every request.
• It has no GraphQL schema, so it never joins the gateway, and it is never deployed to production.

On an environment without it, @backoffice tests are skipped, and an order a test placed is canceled instead of deleted. A scheduled job in B2C removes the test customers and carts that a failed rollback left behind. It leaves orders alone, because an order is a financial record.

What runs where

When	What runs
A pull request	Unit, integration and story tests
A push to main of a reference site	A deploy to the test environment, then the @smoke tests on desktop and mobile, in Chromium and WebKit, without @mail
On demand	Any selection of tests, against any environment

Your own project

evolve add e2e sets up a Playwright suite in your storefront’s testing/: the config, the environments, the fixtures and a floor journey to start from:

pnpm exec evolve add e2e

From there, the Testing guide and the end-to-end conventions take over.