feat: initial implementation of @astrojs/discovery integration

This commit introduces a comprehensive Astro integration that automatically
generates discovery files for websites:

Features:
- robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.)
- llms.txt for AI assistant context and instructions
- humans.txt for team credits and site information
- Automatic sitemap integration via @astrojs/sitemap

Technical Details:
- TypeScript implementation with full type safety
- Configurable HTTP caching headers
- Custom template support for all generated files
- Sensible defaults with extensive customization options
- Date-based versioning (2025.11.03)

Testing:
- 34 unit tests covering all generators
- Test coverage for robots.txt, llms.txt, and humans.txt
- Integration with Vitest

Documentation:
- Comprehensive README with examples
- API reference documentation
- Contributing guidelines
- Example configurations (minimal and full)
This commit is contained in:
Ryan Malloy 2025-11-03 07:36:39 -07:00
commit d25dde4627
25 changed files with 11001 additions and 0 deletions

View file

@ -0,0 +1,18 @@
import { defineConfig } from 'astro/config';
import discovery from '@astrojs/discovery';
// Minimal configuration - just provide your site URL
// Everything else uses sensible defaults
export default defineConfig({
site: 'https://example.com',
integrations: [
discovery(),
],
});
// This will generate:
// - /robots.txt with LLM bot support
// - /llms.txt with basic site info
// - /humans.txt with basic structure
// - /sitemap-index.xml with all your pages