Replace max_turns cap with 90% confidence stopping criterion in agent setup

- Remove fixed _MAX_TURNS=5 instruction from system prompt; LLM now decides
  when to stop based on self-assessed confidence (>= 90%)
- Add _MIN_TURNS_BEFORE_NUDGE=3 and raise safety cap to _MAX_TURNS=15
- Nudge message and hard cap still act as a safety net for infinite loops

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Roberto Musso
2026-03-21 22:54:34 +01:00
parent 1a8bf11f90
commit f07580574b

View File

@@ -43,8 +43,10 @@ _SESSION_TTL_SECONDS: int = 1800 # 30 minutes
_TEMPLATE_START = "PROMPT_TEMPLATE_START" _TEMPLATE_START = "PROMPT_TEMPLATE_START"
_TEMPLATE_END = "PROMPT_TEMPLATE_END" _TEMPLATE_END = "PROMPT_TEMPLATE_END"
# Maximum number of conversation turns before the LLM is nudged to wrap up. # Minimum turns before we consider nudging the LLM to wrap up.
_MAX_TURNS: int = 5 _MIN_TURNS_BEFORE_NUDGE: int = 3
# Hard cap to avoid infinite loops (safety net, not the primary stopping criterion).
_MAX_TURNS: int = 15
# Max tool-calling steps per LLM invocation. # Max tool-calling steps per LLM invocation.
_MAX_TOOL_STEPS: int = 6 _MAX_TOOL_STEPS: int = 6
@@ -128,8 +130,10 @@ and must perform CRUD operations using tools to create records. It should speci
- Concrete examples of mappings based on what you discovered in the directory. - Concrete examples of mappings based on what you discovered in the directory.
{existing_section}\ {existing_section}\
Do not ask more than {max_turns} questions total. Begin by exploring the directory, Keep asking clarifying questions until you are at least 90% confident you have
then ask your first question.\ enough information to generate an accurate prompt_template. Once you reach that
confidence level, stop asking and produce the final template immediately.
Begin by exploring the directory, then ask your first question.\
""" """
@@ -150,7 +154,6 @@ def _build_system_prompt(
template_start=_TEMPLATE_START, template_start=_TEMPLATE_START,
template_end=_TEMPLATE_END, template_end=_TEMPLATE_END,
existing_section=existing_section, existing_section=existing_section,
max_turns=_MAX_TURNS,
) )
@@ -356,8 +359,8 @@ async def handle_journey_message(
prompt_template = _extract_template(ai_reply) prompt_template = _extract_template(ai_reply)
done = prompt_template is not None done = prompt_template is not None
# If the LLM didn't produce a template but we've hit max turns, nudge it # If the LLM didn't produce a template, nudge it once it has asked enough
# and call the LLM one more time to force template generation. # questions (>= _MIN_TURNS_BEFORE_NUDGE) or hits the hard safety cap.
if not done: if not done:
turns = sum(1 for t in session.history if t["role"] == "user") turns = sum(1 for t in session.history if t["role"] == "user")
if turns >= _MAX_TURNS: if turns >= _MAX_TURNS: