mrhavens cfbb701284 feat: Add Agent-Driven Web Crawler ...

Created agent_crawler.py:

AgentWebCrawler - AI-powered crawling that:
1. Analyzes site structure (LLM)
2. Decides what to crawl based on purpose
3. Scores relevance dynamically
4. Adapts as it learns more
5. Knows when it has enough

Purpose types:
- DOCUMENTATION - Technical docs, guides
- TRAINING - Learning materials
- KNOWLEDGE - General knowledge base
- RESEARCH - Research papers
- REFERENCE - Reference material

Usage:

Features:
- Content extraction (not HTML dump)
- Relevance scoring
- Rate limiting
- Configurable depth/pages
- Integration with multi-source ingest

2026-03-14 05:00:44 +00:00

..

agents

feat: Issue #20 - Purpose-Based Agent Selection

2026-03-13 22:26:24 +00:00

crews

fix: Agent/workflow bugs - CrewAI config, AutoGen revision

2026-03-13 18:15:24 +00:00

nonfiction

feat: Add RPG & Tabletop Gaming Frameworks

2026-03-14 01:10:11 +00:00

schemas

Initial commit: Opus Orchestrator AI - Full-flow book generation

2026-03-12 17:45:05 +00:00

utils

feat: Add Agent-Driven Web Crawler

2026-03-14 05:00:44 +00:00

__init__.py

Add rigorous nonfiction frameworks

2026-03-13 08:43:54 +00:00

__main__.py

Add CrewAI integration + CLI for standalone running

2026-03-13 02:53:52 +00:00

autogen_critique.py

fix: Agent/workflow bugs - CrewAI config, AutoGen revision

2026-03-13 18:15:24 +00:00

cli.py

feat: Add --thread-id and --resume flags for checkpointing

2026-03-13 23:24:41 +00:00

config.py

fix: Infrastructure - GitHub token, API validation, cost controls

2026-03-13 18:16:38 +00:00

frameworks.py

Add multiple story frameworks

2026-03-12 19:52:47 +00:00

langgraph_state.py

Add LangGraph state management with validation and checkpoints

2026-03-12 20:08:30 +00:00

langgraph_workflow.py

fix: Critical bugs - LLM client, server imports, error handling

2026-03-13 18:14:11 +00:00

nonfiction_frameworks.py

Add rigorous nonfiction frameworks

2026-03-13 08:43:54 +00:00

nonfiction_generator.py

Add rigorous nonfiction frameworks

2026-03-13 08:43:54 +00:00

nonfiction_taxonomy.py

feat: Add nonfiction taxonomy - Purpose × Structure matrix

2026-03-13 20:00:30 +00:00

orchestrator.py

feat: Issue #19 - Taxonomy Integration in Orchestrator

2026-03-13 20:32:48 +00:00

pydanticai_agent.py

Fix: Remove hardcoded paths - now fully standalone

2026-03-13 04:34:22 +00:00

server.py

fix: Restore missing UploadFile/File imports after merge

2026-03-13 19:05:20 +00:00

state_adapter.py

fix: Architecture - state adapter, agent validation

2026-03-13 18:18:03 +00:00

state.py

Initial commit: Opus Orchestrator AI - Full-flow book generation

2026-03-12 17:45:05 +00:00

web_ui.py

Fix: Remove hardcoded paths - now fully standalone

2026-03-13 04:34:22 +00:00