Filters

Exclude paths and apply robots rules

Overview

Filters prune discovered pages before indexing. The CLI exposes exclude patterns and robots.txt filtering. This mirrors indexing filters.

Exclude patterns

Use --exclude to drop paths that match glob patterns. You can repeat the flag:

peam \
  --source fileBased --projectDir . --sourceDir dist \
  --exclude /admin/** --exclude /private/** \
  --store fileBased --indexPath .peam/index.json

robots.txt filtering

Enable robots.txt filtering or provide a custom path:

peam \
  --source fileBased --projectDir . --sourceDir dist \
  --robotsTxt true \
  --store fileBased --indexPath .peam/index.json

peam \
  --source fileBased --projectDir . --sourceDir dist \
  --robotsTxt ./public/robots.txt \
  --store fileBased --indexPath .peam/index.json

Notes

--robotsTxt true attempts to discover robots.txt automatically.
--robotsTxt false disables robots.txt filtering.

Stores

Configure where the CLI exports index artifacts

API Reference

Complete API reference for Peam

On this page

Exclude patterns

robots.txt filtering

Edit this page on GitHub