Files

Roberto Musso 63fa119543 feat(batch-agent): add journey eval to E2E harness

- journey_runner.py: orchestrates journey start → simulated user
  messages → template extraction → LLM judge scoring
- config.py: JourneyFixture dataclass with user_messages and
  expected_template_criteria, discover_journey_fixtures()
- langfuse_eval.py: sync_journey_fixture_to_dataset()
- cli.py: new 'journey' subcommand (python -m eval journey)
  with --fixture, --models, --judge-model flags
- fixtures/journey_invoice_setup.yaml: example journey fixture
  with 4 user messages and 8 quality criteria

2026-03-23 23:16:41 +01:00

app

refactor: deduplicate shared code into shared/ module

2026-03-23 23:01:45 +01:00

eval

feat(batch-agent): add journey eval to E2E harness

2026-03-23 23:16:41 +01:00

Dockerfile

feat(batch-agent): extract Batch Agent Service (Step 3)

2026-03-23 07:19:02 +01:00

README.md

docs: add Langfuse integration TODO for batch-agent service

2026-03-23 00:25:42 +01:00

requirements.txt

feat(batch-agent): integrate Langfuse tracing

2026-03-23 08:43:15 +01:00

README.md

Batch Agent Service

Owns: agent_runner, journey builder, filesystem_agent, integrations (Gmail, MS Graph).

Tables owned

local_agent_configs
cloud_agent_configs
agent_run_logs

Endpoints

GET /agents/catalog
POST /agents/can-create
POST /agents/trigger
GET /agents/{id}/history

Redis channels

Subscribe: batch:request:{user_id}
Publish: ws:out:{user_id} (journey replies + tool calls)
BRPOP: tool:result:{call_id} (30s timeout)
SET+EX: journey:{user_id} (session state, TTL 1800s)

TODO

Integrate Langfuse tracing (reuse services/chat/app/tracing.py pattern — trace_span(), get_langfuse_callback(), prompt management). Each batch agent run should create a trace with input/output, link prompts, and pass the LangChain CallbackHandler to LLM calls.