sitemap API

Everything the package exports, by the path you import it from. Open an export for its signature and options.

`@evolve-framework/sitemap`

SitemapService, SitemapWriter, the partition strategies, the source interface and SitemapModule with its rebuild routes.

Classes

SitemapFileAccumulates the <url> elements of one file, then saves it.

import { SitemapFile } from "@evolve-framework/sitemap";

Held in memory rather than streamed to disk: StorageClient.save takes a string, so a file has to be fully materialised anyway, and the protocol caps a file at 50,000 URLs / 50 MB — around 7 MB in practice.

new SitemapFile(key: string, maxUrls: number, storage: StorageClient, bucket: string)

Method	Description
`add(url: SitemapUrl): boolean`	Returns false when the URL was rejected, so the caller can count it.
`close(): Promise<void>`	Serialise and save.

SitemapModuleHosts sitemap generation as a service.

import { SitemapModule } from "@evolve-framework/sitemap";

The job script stays the primary way to run this — a scheduled container or function invoking rebuild() directly. The module exists so the same routine is reachable over HTTP, for a scheduler that can only make a request and for rebuilding a few partitions after a content change without waiting for the next scheduled run.

A rebuild is triggered rather than awaited: walking a catalog outlasts any sensible request timeout, so the route returns 202 and the report is logged.

Extends AbstractModule<unknown>

new SitemapModule(options: SitemapModuleOptions)

Option	Type	Description
`service`	`SitemapService`	The service the routes run: `/rebuild` calls its `rebuild()` and `/rebuild/partitions` its `rebuildPartitions()`.
`routes?`	`boolean`	Publish the operational routes (`/rebuild`, `/rebuild/partitions`). Disable for a deployment that only runs the job on a schedule. Defaults to `true`.
`secret?`	`string`	Shared secret required on the operational routes. When set, callers must present `authorization: Bearer <secret>`.

SitemapService

import { SitemapService } from "@evolve-framework/sitemap";

new SitemapService(options: SitemapServiceOptions)

Option	Type	Description
`source`	`SitemapSource`
`writer`	`SitemapWriter`
`buildUrl`	`(context: BuildUrlContext) => string \| undefined`	Turn a slug into a URL. The one thing the framework cannot supply: only the consumer knows its route table and which domain serves a store. More Returning undefined skips the entry, for a store with no configured domain. Counted separately from `SitemapServiceOptions.include`, because this is a misconfiguration and that is a policy.
`include?`	`((entry: SitemapEntry, scope: SitemapScope) => boolean)`	Whether this entry should be indexed at all. A sitemap asserts that a URL should be indexed, so anything the site does not want indexed — discontinued lines, navigation-only categories — belongs outside it.

Method	Description
`partitionsFor(sourceIds: string[]): number[]`	Which partitions these entity ids live in, for the dirty-set flow.
`rebuild( options?: { scopes?: SitemapScope[] \| undefined; concurrency?: number \| undefined; }, ): Promise<SitemapReport>`
`rebuildPartitions( scope: SitemapScope, partitions: number[], ): Promise<SitemapReport>`	Rewrite only these partitions.

SitemapWriter

import { SitemapWriter } from "@evolve-framework/sitemap";

new SitemapWriter(options: SitemapWriterOptions)

Option	Type	Description
`storage`	`StorageClient`
`bucket`	`string`
`collection`	`string`	The file set this writer owns: `"products"`, `"pages"`.
`strategy`	`PartitionStrategy`	One writer serves one collection, so one strategy.
`path?`	`SitemapPathFn`	Defaults to `defaultSitemapPath`.
`maxUrlsPerFile?`	`number`	Defaults to the protocol maximum of 50,000.

Method	Description
`keyFor(scope: SitemapScope, partition: number): string`
`open(scope: SitemapScope, partition: number): SitemapFile`
`saveDocument(key: string, xml: string): Promise<void>`	Save an already-serialised document, for the sitemap index.

Functions

consistentPartitionsPartition by hashing the id onto a ring, for ids that are not uniformly distributed.

import { consistentPartitions } from "@evolve-framework/sitemap";

Consistent hashing keeps assignment stable: an id keeps its partition while count is unchanged, and raising count moves roughly 1/count of the ids rather than all of them. Replicas per partition are what make the split even for arbitrary id shapes.

What it cannot do is express a partition as a query. There is no rangeFor, so the only way to write one file is to walk the whole collection and assign every id client-side — every rebuild is a full rebuild, and it cannot be split across workers. Prefer idRangePartitions when ids allow it.

Ported from evolve-reference-b2b's catalog/lib/partition-key.ts, ring semantics unchanged, so partition assignment is identical for a given (count, replicas). The behavioural difference is that count is now a constructor argument instead of a module-level read of a config singleton.

On replicas: the default of 512 is inherited for that compatibility, and it is on the low side. Measured worst-case deviation from an even split, from the ring's arc lengths and confirmed empirically over 200k UUIDs:

partitions	replicas	worst drift
20	512	9.3%
20	2048	3.5%
32	512	5.4%

So one file in twenty can be ~9% larger than its siblings. That is harmless for sitemaps — the 50,000-URL limit is what matters, and SitemapReport surfaces the per-file counts — but raising replicas is the fix if a collection sits close to the limit. It reshuffles every id, so treat it like a change of count.

function consistentPartitions(
  count: number,
  replicas?: number,
): PartitionStrategy

Returns PartitionStrategy

defaultSitemapPathDefault layout.

import { defaultSitemapPath } from "@evolve-framework/sitemap";

function defaultSitemapPath(
  scope: SitemapScope,
  partition: number,
  collection: string,
): string

Option	Type	Description
`storeKey`	`string`	Tenant/channel this content belongs to (a commercetools store key).
`locale`	`string`	The single locale these URLs are written for.

Returns string

escapeXmlEscape a value for XML text content.

import { escapeXml } from "@evolve-framework/sitemap";

All five predefined entities, not just the three that text content strictly requires: the sitemap protocol calls for entity-escaped URLs, and a & in a query string is the case that actually occurs.

function escapeXml(value: string): string

Returns string

idRangePartitionsPartition by leading id characters, so that every partition is expressible as an id range.

import { idRangePartitions } from "@evolve-framework/sitemap";

Requires uniformly distributed ids. Commercetools ids are UUIDs, so their leading hex characters are uniform and this splits evenly. Merchant-assigned SKUs (SFCC) and sequential integers (Magento) are not uniform: prefix ranges over those collapse most entities into a few partitions. Use consistentPartitions there, and accept that a partition is then not independently addressable.

Assignment is stable — an id never moves while count is unchanged — and raising count splits each existing range rather than reshuffling ids across unrelated files.

function idRangePartitions(
  count: number,
  options?: IdRangePartitionsOptions,
): PartitionStrategy

Option	Type	Description
`alphabet?`	`string`	Characters ids are built from, in ascending order. Must match the ordering the backend applies to `sort: "id asc"`, or a range query will not return the entries this strategy assigns to it.

Returns PartitionStrategy

isInRangeTrue when id falls inside range, using the same comparison a backend's where id >= … and id < … predicate would apply.

import { isInRange } from "@evolve-framework/sitemap";

function isInRange(id: string, range: IdRange): boolean

Option	Type	Description
`fromId?`	`string`
`toId?`	`string`

Returns boolean

isSameScopeTrue when both scopes address the same URLs.

import { isSameScope } from "@evolve-framework/sitemap";

function isSameScope(a: SitemapScope, b: SitemapScope): boolean

Option	Type	Description
`storeKey`	`string`	Tenant/channel this content belongs to (a commercetools store key).
`locale`	`string`	The single locale these URLs are written for.
`storeKey`	`string`	Tenant/channel this content belongs to (a commercetools store key).
`locale`	`string`	The single locale these URLs are written for.

Returns boolean

sequentialPartitionsFill file 0, then file 1, and so on.

import { sequentialPartitions } from "@evolve-framework/sitemap";

For collections small enough that stability does not matter — a few hundred to a few thousand CMS pages — where the alternative buys nothing and costs a uniform-id precondition.

Assignment depends on iteration order, not on the id, so a given page can move between files when the collection changes. That is the tradeoff: no rangeFor either, so partitions are not independently addressable.

Overflowing count * maxUrls throws. Silently dropping URLs past the end would delist them, which is the one failure mode worth failing loudly over; raising count is the operator's decision.

function sequentialPartitions(
  options: SequentialPartitionsOptions,
): PartitionStrategy

Option	Type	Description
`count`	`number`	Files this collection is allowed to fill. Required, because the sitemap index is derived from the declared layout rather than from what a run observed — a count that grows with the content could not be declared.
`maxUrls?`	`number`	URLs per file before rolling to the next. Defaults to the spec maximum.

Returns PartitionStrategy

writeSitemapIndexWrite one flat <sitemapindex> covering every file of every collection.

import { writeSitemapIndex } from "@evolve-framework/sitemap";

Flat because the protocol forbids an index listing other indexes, and derived from the declared layout rather than from what any run observed. That is what makes this idempotent: whichever job calls it last writes byte-identical output, so independent per-collection runs — and the many workers inside one run — need no completion barrier between them.

function writeSitemapIndex(
  writer: SitemapWriter,
  layout: SitemapLayout,
  options: WriteSitemapIndexOptions,
): Promise<string[]>

Option	Type	Description
`collections`	`SitemapLayoutCollection[]`
`baseUrl`	`string`	Public base URL the `<loc>` entries are resolved against.
`key?`	`string`	Where the index itself goes, relative to the bucket.

Returns Promise<string[]>

Constants

HEX_ALPHABETLowercase hex, which is what a UUID's leading characters are.

import { HEX_ALPHABET } from "@evolve-framework/sitemap";

const HEX_ALPHABET: "0123456789abcdef"

MAX_URL_LENGTHLongest <loc> the sitemap protocol allows.

import { MAX_URL_LENGTH } from "@evolve-framework/sitemap";

const MAX_URL_LENGTH: 2048

MAX_URLS_PER_FILEMost URLs one file may contain.

import { MAX_URLS_PER_FILE } from "@evolve-framework/sitemap";

const MAX_URLS_PER_FILE: 50000

Types

BuildUrlContextWhat the consumer's route table needs to build a URL.

import { type BuildUrlContext } from "@evolve-framework/sitemap";

export type BuildUrlContext = {
  collection: string;
  sourceId: string;
  slug: string;
  locale: string;
  storeKey: string;
};

ChangeFreq

import { type ChangeFreq } from "@evolve-framework/sitemap";

export type ChangeFreq = "always" | "hourly" | "daily" | "weekly" | "monthly" | "yearly" | "never";

IdRangeA half-open id interval, fromId <= id < toId, compared as strings.

import { type IdRange } from "@evolve-framework/sitemap";

An absent bound is unbounded on that side, so the first and last partition of a collection do not need sentinel values.

export type IdRange = {
  fromId?: string;
  toId?: string;
};

LocalizedSlugA localized value keyed by locale, e.g.

import { type LocalizedSlug } from "@evolve-framework/sitemap";

export type LocalizedSlug = Record<string, string>;

PartitionStrategyHow entries are spread over the files of one collection.

import { type PartitionStrategy } from "@evolve-framework/sitemap";

Two properties matter, and they pull in different directions:

Stability. An entity should keep its file across runs, so a rebuild changes as few files as possible and crawlers re-fetch as little as possible.
Addressability. A partition that can be expressed as an id range can be rebuilt on its own, with one bounded query, instead of by walking the whole collection and assigning client-side. That is what makes both parallel full rebuilds and incremental rebuilds possible.

idRangePartitions has both, but requires uniformly distributed ids. consistentPartitions has only the first, and works on any id.

export type PartitionStrategy = {
  readonly count: number;
  assign(sourceId: string, ordinal: number): number;
  rangeFor?(partition: number): IdRange;
};

ScopeSitemapReport

import { type ScopeSitemapReport } from "@evolve-framework/sitemap";

export type ScopeSitemapReport = {
  scope: SitemapScope;
  partitions: number[];
  urls: number;
  skipped: SkipCounts;
  maxUrlsInFile: number;
};

SitemapEntryOne entity, as a source reports it.

import { type SitemapEntry } from "@evolve-framework/sitemap";

export type SitemapEntry = {
  sourceId: string;
  slug: LocalizedSlug;
  lastModifiedAt?: string;
  changefreq?: ChangeFreq;
  priority?: number;
};

SitemapLayout

import { type SitemapLayout } from "@evolve-framework/sitemap";

export type SitemapLayout = {
  collections: SitemapLayoutCollection[];
};

SitemapLayoutCollection

import { type SitemapLayoutCollection } from "@evolve-framework/sitemap";

export type SitemapLayoutCollection = {
  collection: string;
  strategy: PartitionStrategy;
  scopes: SitemapScope[];
  path?: SitemapPathFn;
};

SitemapPathFnWhere a partition file lands, relative to the bucket.

import { type SitemapPathFn } from "@evolve-framework/sitemap";

export type SitemapPathFn = (scope: SitemapScope, partition: number, collection: string) => string;

SitemapReport

import { type SitemapReport } from "@evolve-framework/sitemap";

export type SitemapReport = {
  collection: string;
  scopes: ScopeSitemapReport[];
  urls: number;
  files: number;
  skipped: SkipCounts;
  maxUrlsInFile: number;
};

SitemapScopeWhere URLs live.

import { type SitemapScope } from "@evolve-framework/sitemap";

Deliberately not IndexScope from @evolve-framework/search: that type carries a locales array whose cardinality is a search-tuning decision (Algolia's indexLanguages is index-level, so granularity is configurable), plus currencies and countries. A URL has exactly one store and one locale, so modelling it as anything else would couple the sitemap file layout to a decision about search indices.

export type SitemapScope = {
  storeKey: string;
  locale: string;
};

SitemapSourceWhere URLs come from.

import { type SitemapSource } from "@evolve-framework/sitemap";

Deliberately much thinner than CatalogSource in @evolve-framework/search. That one yields search documents, so satisfying it means resolving prices, tax, category paths and variant fan-out. A sitemap needs an id, a slug per locale and a timestamp, so a new backend's source should be a single small file. Anything that makes this contract more expensive to implement should be questioned on that basis.

export type SitemapSource = {
  readonly collection: string;
  getScopes(): Promise<SitemapScope[]>;
  iterate(scope: SitemapScope, range?: IdRange): AsyncIterable<SitemapEntry[]>;
  readonly supportsIdRange?: boolean;
};

SitemapUrlOne <url>, after the consumer's route table has produced a real URL.

import { type SitemapUrl } from "@evolve-framework/sitemap";

export type SitemapUrl = {
  loc: string;
  lastmod?: string;
  changefreq?: ChangeFreq;
  priority?: number;
};

SkipCounts

import { type SkipCounts } from "@evolve-framework/sitemap";

export type SkipCounts = {
  noSlug: number;
  noUrl: number;
  excluded: number;
  tooLong: number;
};

`@evolve-framework/sitemap/testing`

A fake source and entries, for testing a sitemap build without a commerce engine.

Classes

FakeSitemapSourceA source with no backend, for testing the service and writer.

import { FakeSitemapSource } from "@evolve-framework/sitemap/testing";

Records the ranges it was asked for, which is how a test asserts that the service used the cheap path rather than walking the collection.

new FakeSitemapSource(options?: FakeSitemapSourceOptions)

Option	Type	Description
`collection?`	`string`
`scopes?`	`SitemapScope[]`
`entries?`	`SitemapEntry[]`
`batchSize?`	`number`	Entries per yielded batch.
`supportsIdRange?`	`boolean`	Whether to claim range push-down. The fake filters in memory either way — it has no backend to push to — so this is what lets a test drive both the ranged and the walking path.
`failOnBatch?`	`number`	Throw on the nth batch, to test that a failure does not truncate a file.
`gate?`	`Promise<unknown>`	Awaited before the first batch of every `iterate` call. Lets a test hold a rebuild open and observe what happens while it is still running — that a route returned before finishing, or that a second request is refused.

Method	Description
`getScopes(): Promise<SitemapScope[]>`	Every store × locale this source covers.
`iterate( scope: SitemapScope, range?: IdRange, ): AsyncIterable<SitemapEntry[]>`	Every entry for one scope, batched, so a source pages its backend once per batch rather than once per entry.

Functions

fakeEntriesSequential hex ids, so a test can predict partition assignment.

import { fakeEntries } from "@evolve-framework/sitemap/testing";

function fakeEntries(
  amount: number,
  overrides?: Partial<SitemapEntry>,
): SitemapEntry[]

Option	Type	Description
`sourceId?`	`string`	Stable identifier. Two things depend on it never changing for an entity: the partition assignment, and the ability to rebuild one partition from an id range.
`slug?`	`LocalizedSlug`	The URL identifier per locale. Backends whose identifier is not localized (SFCC uses the product `ID`) repeat the same value for every locale. More A locale missing from the map means "not published here" and the entry is skipped for that scope.
`lastModifiedAt?`	`string`	Last change a visitor could see — name, slug, price, availability, images, description, structured data. More Omit rather than supply a timestamp that moves for reasons a visitor cannot see. A backend's own modification timestamp usually does: an internal attribute correction or a nightly price feed advances it for every entity, and a `lastmod` that changes daily for everything tells a crawler nothing.
`changefreq?`	`ChangeFreq`	Both are optional, and both are ignored by the major crawlers. Supplying the same value for every URL — as is tempting — conveys nothing at all. Kept only as passthrough for consumers who have a reason.
`priority?`	`number`

Returns SitemapEntry[]

Not listed: 7 options types. They appear as the tables under the constructors and functions they type.

Get started

Guides

Conventions

Reference

Architecture

Changelog

sitemap API

`@evolve-framework/sitemap`

Classes

Functions

Constants

Types

`@evolve-framework/sitemap/testing`

Classes

Functions