Update README with comprehensive documentation matching implementation
- Complete architecture diagram - All features documented with status - Full CLI, Python, API client examples - Configuration reference - Project structure
This commit is contained in:
@@ -1,161 +1,339 @@
|
|||||||
# Opus Orchestrator AI
|
# Opus Orchestrator AI
|
||||||
|
|
||||||
> Full-flow AI book generation orchestrator using LangGraph, CrewAI, AutoGen, and PydanticAI. Integrates Fiction Fortress and Nonfiction Fortress for professional manuscript production.
|
> Full-flow AI book generation system using **LangGraph**, **CrewAI**, **AutoGen**, and **PydanticAI**
|
||||||
|
|
||||||
## Overview
|
A comprehensive, production-ready system for generating publication-ready manuscripts from raw content.
|
||||||
|
|
||||||
Opus Orchestrator AI transforms raw content (notes, outlines, stream-of-consciousness, essays, logs) into fully edited, publication-ready manuscripts. It combines:
|
---
|
||||||
|
|
||||||
- **LangGraph** — Workflow orchestration and state management
|
## ⚡ Quick Start
|
||||||
- **CrewAI** — Role-based agent crews
|
|
||||||
- **AutoGen** — Complex multi-agent negotiations
|
|
||||||
- **PydanticAI** — Structured output validation
|
|
||||||
- **Fiction Fortress** — Complete fiction writing methodology
|
|
||||||
- **Nonfiction Fortress** — Complete non-fiction writing methodology
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
[GitHub Repo Input]
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌───────────────────┐
|
|
||||||
│ INGESTOR AGENT │ ──► Extracts raw content from repo
|
|
||||||
└────────┬──────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌───────────────────┐
|
|
||||||
│ INTENT ANALYZER │ ──► Analyzes goals, audience, intended outcome
|
|
||||||
└────────┬──────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌───────────────────┐
|
|
||||||
│ BLUEPRINT │ ──► Generates detailed book blueprint
|
|
||||||
└────────┬──────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────┴────┐
|
|
||||||
│ ITERATE │
|
|
||||||
└────┬────┘
|
|
||||||
│
|
|
||||||
┌────▼─────────────┐
|
|
||||||
│ CREW EXECUTION │ ──► Runs agent crews per chapter
|
|
||||||
│ • Writer │
|
|
||||||
│ • Critics (3+) │
|
|
||||||
│ • Editor │
|
|
||||||
│ • Proofreader │
|
|
||||||
└────┬─────────────┘
|
|
||||||
│
|
|
||||||
┌────▼─────────────┐
|
|
||||||
│ REVIEW & REVISE │ ──► Internal critic circle
|
|
||||||
└────┬─────────────┘
|
|
||||||
│
|
|
||||||
└─────────────┬────► [Loop back if needed]
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌─────────────────┐
|
|
||||||
│ COMPILED .MD │
|
|
||||||
│ MANUSCRIPT │
|
|
||||||
└─────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/mrhavens/opus-orchestrator-ai.git
|
# Install
|
||||||
cd opus-orchestrator-ai
|
pip install opus-orchestrator-ai
|
||||||
pip install -e .
|
|
||||||
|
# Generate a manuscript (local mode)
|
||||||
|
opus generate --concept "A robot dreams of electric sheep" --words 5000
|
||||||
|
|
||||||
|
# Or use API server mode
|
||||||
|
opus serve --port 8000
|
||||||
|
opus --api-url http://localhost:8000 generate --concept "Your idea"
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick Start
|
---
|
||||||
|
|
||||||
|
## 🏗️ Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ OPUS ORCHESTRATOR AI │
|
||||||
|
├─────────────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
|
||||||
|
│ │ INGEST │───►│ LANGGRAPH │───►│ OUTPUT │ │
|
||||||
|
│ │ LAYER │ │ WORKFLOW │ │ (Manuscript) │ │
|
||||||
|
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
|
||||||
|
│ │ │ │
|
||||||
|
│ ┌────┴────┐ ┌────┴────┐ │
|
||||||
|
│ ▼ ▼ ▼ ▼ │
|
||||||
|
│ GitHub S3/MinIO CrewAI AutoGen │
|
||||||
|
│ Ingestor Ingestor Agents Critique │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||||
|
│ │ VALIDATION LAYER │ │
|
||||||
|
│ │ PydanticAI │ │
|
||||||
|
│ └─────────────────────────────────────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 Features
|
||||||
|
|
||||||
|
### Core Generation
|
||||||
|
|
||||||
|
| Feature | Description | Status |
|
||||||
|
|---------|-------------|--------|
|
||||||
|
| **Snowflake Method** | 7-stage fractal expansion from sentence to novel | ✅ |
|
||||||
|
| **Story Frameworks** | 7 frameworks: Snowflake, 3-Act, Save the Cat, Hero's Journey, Story Circle, 7-Point, Fichtean | ✅ |
|
||||||
|
| **LangGraph Workflow** | State machine with streaming progress | ✅ |
|
||||||
|
| **AutoGen Critique** | Multi-agent debate (LiteraryCritic, GenreExpert, StoryEditor) | ✅ |
|
||||||
|
| **PydanticAI Validation** | Structured output validation with type-safe schemas | ✅ |
|
||||||
|
|
||||||
|
### Agent Systems
|
||||||
|
|
||||||
|
| Agent | Role | Status |
|
||||||
|
|-------|------|--------|
|
||||||
|
| **ArchitectAgent** | Story structure & blueprint | ✅ |
|
||||||
|
| **WorldsmithAgent** | World-building & setting | ✅ |
|
||||||
|
| **CharacterLeadAgent** | Character development | ✅ |
|
||||||
|
| **VoiceAgent** | Narrative voice & tone | ✅ |
|
||||||
|
| **EditorAgent** | Editorial review | ✅ |
|
||||||
|
| **ResearcherAgent** (Nonfiction) | Fact-finding | ✅ |
|
||||||
|
| **AnalystAgent** (Nonfiction) | Argument analysis | ✅ |
|
||||||
|
|
||||||
|
### Ingestion Sources
|
||||||
|
|
||||||
|
| Source | Description | Status |
|
||||||
|
|--------|-------------|--------|
|
||||||
|
| **GitHub** | Fetch from public/private repos | ✅ |
|
||||||
|
| **S3/MinIO** | S3-compatible object storage | ✅ |
|
||||||
|
| **Local Files** | Direct file input | ✅ |
|
||||||
|
|
||||||
|
### Deployment
|
||||||
|
|
||||||
|
| Mode | Description | Status |
|
||||||
|
|------|-------------|--------|
|
||||||
|
| **CLI** | Standalone command-line tool | ✅ |
|
||||||
|
| **API Server** | FastAPI REST server | ✅ |
|
||||||
|
| **API Client** | Client mode for remote servers | ✅ |
|
||||||
|
| **Python Module** | Import as library | ✅ |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Usage
|
||||||
|
|
||||||
|
### CLI Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Generate manuscript (local)
|
||||||
|
opus generate --concept "Your story idea" --framework snowflake --words 5000
|
||||||
|
|
||||||
|
# Generate from GitHub
|
||||||
|
opus generate --repo owner/repo --framework hero-journey --words 80000
|
||||||
|
|
||||||
|
# Generate from S3/MinIO
|
||||||
|
opus ingest-s3 --bucket my-bucket --prefix notes/ --output content.txt
|
||||||
|
|
||||||
|
# Start API server
|
||||||
|
opus serve --port 8000
|
||||||
|
|
||||||
|
# Use as API client
|
||||||
|
opus --api-url http://localhost:8000 generate --concept "..."
|
||||||
|
|
||||||
|
# List frameworks
|
||||||
|
opus frameworks
|
||||||
|
|
||||||
|
# Show config
|
||||||
|
opus config
|
||||||
|
|
||||||
|
# Show docs
|
||||||
|
opus docs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python API
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from opus_orchestrator import OpusOrchestrator
|
from opus_orchestrator import run_opus
|
||||||
|
|
||||||
orchestrator = OpusOrchestrator(
|
# Simple generation
|
||||||
repo_url="https://github.com/user/my-book-ideas",
|
result = await run_opus(
|
||||||
book_type="fiction", # or "nonfiction"
|
seed_concept="A robot dreams of love",
|
||||||
|
framework="snowflake",
|
||||||
genre="science-fiction",
|
genre="science-fiction",
|
||||||
target_audience="adult sci-fi readers",
|
target_word_count=5000,
|
||||||
intended_outcome="complete novel, ~80k words"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Run the full pipeline
|
manuscript = result["manuscript"]
|
||||||
manuscript = await orchestrator.run()
|
|
||||||
print(f"Generated: {manuscript.word_count} words")
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Configuration
|
### Using CrewAI
|
||||||
|
|
||||||
See `config.example.yaml` for full configuration options.
|
```python
|
||||||
|
from opus_orchestrator.crews import create_fiction_crew
|
||||||
|
|
||||||
## Project Structure
|
crew = create_fiction_crew(
|
||||||
|
genre="science-fiction",
|
||||||
|
tone="literary",
|
||||||
|
target_word_count=2000,
|
||||||
|
)
|
||||||
|
|
||||||
|
story = crew.write_full_story(
|
||||||
|
story_outline="Your outline...",
|
||||||
|
character_sheets="...",
|
||||||
|
style_guide="Tone: literary",
|
||||||
|
num_chapters=5,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using PydanticAI Validation
|
||||||
|
|
||||||
|
```python
|
||||||
|
from opus_orchestrator import create_style_guide_agent
|
||||||
|
|
||||||
|
agent = create_style_guide_agent()
|
||||||
|
result = agent.run_sync("Create a style guide for a literary novel")
|
||||||
|
|
||||||
|
# Result is a validated StyleGuide object
|
||||||
|
print(result.tone) # "Contemplative and introspective"
|
||||||
|
print(result.pacing) # "Deliberate with moments of acceleration"
|
||||||
|
```
|
||||||
|
|
||||||
|
### API Server
|
||||||
|
|
||||||
|
```python
|
||||||
|
from opus_orchestrator.server import app, run_server
|
||||||
|
|
||||||
|
# Run server
|
||||||
|
await run_server(host="0.0.0.0", port=8000)
|
||||||
|
|
||||||
|
# Or with uvicorn directly
|
||||||
|
# uvicorn opus_orchestrator.server:app --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
### API Client
|
||||||
|
|
||||||
|
```python
|
||||||
|
from opus_orchestrator.cli import OpusAPIClient
|
||||||
|
|
||||||
|
client = OpusAPIClient("http://localhost:8000")
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
health = client.health()
|
||||||
|
|
||||||
|
# Generate
|
||||||
|
result = client.generate(
|
||||||
|
concept="A robot dreams",
|
||||||
|
framework="snowflake",
|
||||||
|
target_word_count=5000,
|
||||||
|
)
|
||||||
|
|
||||||
|
print(result["manuscript"])
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Configuration
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
| Variable | Description | Required |
|
||||||
|
|----------|-------------|----------|
|
||||||
|
| `OPENAI_API_KEY` | OpenAI API key | Yes (or MINIMAX_API_KEY) |
|
||||||
|
| `MINIMAX_API_KEY` | MiniMax API key | No |
|
||||||
|
| `GITHUB_TOKEN` | GitHub token for private repos | No |
|
||||||
|
| `AWS_ACCESS_KEY_ID` | AWS access key for S3 | No |
|
||||||
|
| `AWS_SECRET_ACCESS_KEY` | AWS secret key for S3 | No |
|
||||||
|
| `S3_ENDPOINT_URL` | Custom S3 endpoint (MinIO, DO Spaces) | No |
|
||||||
|
|
||||||
|
### Configuration File
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# config.yaml
|
||||||
|
agent:
|
||||||
|
provider: openai
|
||||||
|
model: gpt-4o
|
||||||
|
temperature: 0.7
|
||||||
|
max_tokens: null
|
||||||
|
|
||||||
|
iteration:
|
||||||
|
min_critic_rounds: 2
|
||||||
|
max_critic_rounds: 5
|
||||||
|
approval_threshold: 0.8
|
||||||
|
|
||||||
|
output:
|
||||||
|
format: markdown
|
||||||
|
include_toc: true
|
||||||
|
output_dir: ./output
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 Story Frameworks
|
||||||
|
|
||||||
|
### Implemented Frameworks
|
||||||
|
|
||||||
|
| Framework | Type | Stages/Beats |
|
||||||
|
|-----------|------|---------------|
|
||||||
|
| **Snowflake Method** | Fractal | 7 stages |
|
||||||
|
| **Three-Act Structure** | Linear | 7 beats |
|
||||||
|
| **Save the Cat** | Screenplay | 15 beats |
|
||||||
|
| **Hero's Journey** | Mythic | 12 stages |
|
||||||
|
| **Story Circle** | Circular | 8 beats |
|
||||||
|
| **7-Point Plot** | Structural | 7 beats |
|
||||||
|
| **Fichtean Curve** | Episodic | 7 beats |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧩 Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
opus_orchestrator/
|
opus_orchestrator/
|
||||||
├── __init__.py # Main exports
|
├── __init__.py # Main exports
|
||||||
├── config.py # Configuration management
|
├── __main__.py # CLI entry point
|
||||||
├── state.py # LangGraph state definitions
|
├── cli.py # CLI implementation
|
||||||
├── graph.py # Main workflow graph
|
├── server.py # FastAPI server
|
||||||
├── agents/
|
├── orchestrator.py # Main orchestrator
|
||||||
│ ├── __init__.py
|
├── langgraph_workflow.py # LangGraph pipeline
|
||||||
│ ├── base.py # Base agent class
|
├── autogen_critique.py # AutoGen critique crew
|
||||||
│ ├── fiction/ # Fiction Fortress agents
|
├── pydanticai_agent.py # PydanticAI agents
|
||||||
|
├── config.py # Configuration
|
||||||
|
├── frameworks.py # Story frameworks
|
||||||
|
│
|
||||||
|
├── agents/ # Agent implementations
|
||||||
|
│ ├── fiction/ # Fiction agents
|
||||||
│ │ ├── architect.py
|
│ │ ├── architect.py
|
||||||
│ │ ├── worldsmith.py
|
│ │ ├── worldsmith.py
|
||||||
│ │ ├── character_lead.py
|
│ │ ├── character_lead.py
|
||||||
│ │ ├── voice.py
|
│ │ ├── voice.py
|
||||||
│ │ └── editor.py
|
│ │ └── editor.py
|
||||||
│ └── nonfiction/ # Nonfiction Fortress agents
|
│ └── nonfiction/ # Nonfiction agents
|
||||||
│ ├── researcher.py
|
│ ├── researcher.py
|
||||||
│ ├── analyst.py
|
│ ├── analyst.py
|
||||||
│ ├── writer.py
|
│ ├── writer.py
|
||||||
│ ├── fact_checker.py
|
│ ├── fact_checker.py
|
||||||
│ └── editor.py
|
│ └── editor.py
|
||||||
├── crews/
|
│
|
||||||
│ ├── __init__.py
|
├── crews/ # CrewAI crews
|
||||||
│ ├── fiction_crew.py # Fiction writing crew
|
│ ├── base_crew.py # Base crew class
|
||||||
|
│ ├── fiction_crew.py # Fiction crew
|
||||||
│ └── nonfiction_crew.py
|
│ └── nonfiction_crew.py
|
||||||
├── schemas/
|
│
|
||||||
│ ├── __init__.py
|
├── schemas/ # Pydantic schemas
|
||||||
│ ├── book.py # Book-level schemas
|
│ └── book.py
|
||||||
│ ├── chapter.py # Chapter schemas
|
│
|
||||||
│ └── critique.py # Critique schemas
|
└── utils/ # Utilities
|
||||||
└── utils/
|
├── github_ingest.py # GitHub ingestion
|
||||||
├── __init__.py
|
├── s3_ingest.py # S3/MinIO ingestion
|
||||||
├── github.py # GitHub ingestion
|
├── llm.py # LLM client
|
||||||
└── output.py # Output generation
|
└── docs.py # Documentation generator
|
||||||
```
|
```
|
||||||
|
|
||||||
## Development
|
---
|
||||||
|
|
||||||
### Running Tests
|
## 🧪 Testing
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Run tests
|
||||||
pytest tests/
|
pytest tests/
|
||||||
```
|
|
||||||
|
|
||||||
### Code Style
|
# Run with coverage
|
||||||
|
pytest --cov=opus_orchestrator tests/
|
||||||
|
|
||||||
```bash
|
# Lint
|
||||||
ruff check .
|
ruff check .
|
||||||
|
|
||||||
|
# Format
|
||||||
ruff format .
|
ruff format .
|
||||||
```
|
```
|
||||||
|
|
||||||
## Dependencies
|
---
|
||||||
|
|
||||||
- langgraph
|
## 📄 License
|
||||||
- crewai
|
|
||||||
- autogen
|
|
||||||
- pydantic-ai
|
|
||||||
- pydantic
|
|
||||||
- httpx
|
|
||||||
- pygithub
|
|
||||||
- pyyaml
|
|
||||||
|
|
||||||
## License
|
MIT License
|
||||||
|
|
||||||
MIT
|
---
|
||||||
|
|
||||||
|
## 🤝 Built With
|
||||||
|
|
||||||
|
- **LangGraph** - Workflow orchestration
|
||||||
|
- **CrewAI** - Multi-agent systems
|
||||||
|
- **AutoGen** - Complex agent conversations
|
||||||
|
- **PydanticAI** - Structured output validation
|
||||||
|
- **FastAPI** - REST API server
|
||||||
|
- **OpenAI** - LLM provider
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user