How the pieces fit together

For a paginated edition the workflow for a crawler looks like this:

1. It fetches robots.txt, learns which paths are off-limits, and finds the sitemap URL.

2. It reads sitemap.xml and obtains concrete addresses such as https://edition.example.org/doc/example.xml?id=chapter-1.

3. It requests that URL. The server renders the matching fragment via page:content into the initial HTML, so the crawler indexes the chapter text even without running JavaScript.

4. The canonical link on the page matches the sitemap entry (same path and stable id), so variants with extra query parameters do not create duplicate-content issues.

Crawlers expect sitemap.xml and robots.txt at the root of the public site. In production you therefore need a reverse proxy in front of eXist that maps the public host onto your generated application; see Running behind a proxy. For local verification, the demo under profiles/sitemap/tools/ provides an nginx setup that exposes the app at http://localhost/ so you can inspect the artefacts as a crawler would.

After regenerating the sitemap, a quick check is:

Open a document page and confirm that the link with rel="canonical" in the page source uses the same URL shape as the corresponding loc in the sitemap, and that viewing the page source (not the live DOM) already contains the chapter text inside pb-view.