Initial Crawailer implementation with comprehensive JavaScript API

Browse source

- Complete browser automation with Playwright integration
- High-level API functions: get(), get_many(), discover()
- JavaScript execution support with script parameters
- Content extraction optimized for LLM workflows
- Comprehensive test suite with 18 test files (700+ scenarios)
- Local Caddy test server for reproducible testing
- Performance benchmarking vs Katana crawler
- Complete documentation including JavaScript API guide
- PyPI-ready packaging with professional metadata
- UNIX philosophy: do web scraping exceptionally well

...

This commit is contained in:

Crawailer Developer 2025-09-18 14:47:59 -06:00

parent fd836c90cf

commit d31395a166

17 changed files with 8276 additions and 51 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1046

tests/test_browser_engine_compatibility.py Normal file

View file

File diff suppressed because it is too large Load diff