ReAct agent emits a tool Action but never runs the tool, fabricating the Observation and Final Answer itself

Maintainer-team work-in-progress branch reworking tool invocation; the reporter tested it and saw no change.

Status basis: documented as not fixing the failure. Verification: not verified.

I tried running 3 tests with this version, but it still behaves the same as beforereporter · github.com/crewAIInc/crewAI

Workaround: modify the tool-usage prompt (en.json) so the model does not write the Observation itself. Some users report no failures in dozens of attempts, others still see fabrication or loops.

Status basis: documented as only partly fixing the failure. Verification: not verified.

I tried modifying the en.json file as you suggested, but the results are still inconsistent.third_party · github.com/crewAIInc/crewAI

I've tested it with both gpt-4.1-mini, and gpt-5-mini, no failures yet on a few dozen attempts.third_party · github.com/crewAIInc/crewAI

Workaround: use GPT-4.1 instead of GPT-5/o-series models.

Status basis: proposed; no evidence it was applied. Verification: not verified.

the issue was fixed for me when I used GPT 4.1 instead of the 'o' reasoning models from OpenAI.third_party · github.com/crewAIInc/crewAI

Replace the dead recovery block with an explicit position check that truncates at the fabricated Observation so the real Action executes, with fails-before/passes-after tests. Independently re-verified by a third party but closed without merge.

Status basis: proposed; no evidence it was applied. Verification: not verified.

Change: https://github.com/crewAIInc/crewAI/pull/6450 · not merged · tests changed

Excerpt of the change (lib/crewai/src/crewai/utilities/agent_utils.py, MIT):

@@ -17,7 +17,7 @@
 from pydantic import BaseModel
 from rich.console import Console
 
-from crewai.agents.constants import FINAL_ANSWER_AND_PARSABLE_ACTION_ERROR_MESSAGE
+from crewai.agents.constants import ACTION_INPUT_REGEX, FINAL_ANSWER_ACTION
 from crewai.agents.parser import (
     AgentAction,
     AgentFinish,
@@ -576,12 +576,20 @@ def process_llm_response(
     Returns:
         Either an AgentAction or AgentFinish
     """
-    if not use_stop_words:
-        try:
-            format_answer(answer)
-        except OutputParserError as e:
-            if FINAL_ANSWER_AND_PARSABLE_ACTION_ERROR_MESSAGE in e.error:
-                answer = answer.split("Observation:")[0].strip()
+    if not use_stop_words and FINAL_ANSWER_ACTION in answer:
+        action_match = ACTION_INPUT_REGEX.search(answer)
+        final_answer_idx = answer.find(FINAL_ANSWER_ACTION)
+        if action_match and action_match.start() < final_answer_idx:
+            # Without the "\nObservation:" stop sequence the model generates past
+            # the real tool call, fabricating an Observation and Final Answer.
+            # Discard the fabricated continuation so the actual Action executes.
+            # Anchor on the newline (the real stop sequence) so an "Observation:"
+            # substring inside the Action Input payload isn't mistaken for it.
+            observation_idx = answer.find(
+                "\nObservation:", action_match.start(), final_answer_idx

Replace the dead `except` block with an explicit position checkthird_party · github.com/crewAIInc/crewAI

applying this PR's diff locally, the same input returns `AgentAction(tool="web_search", ...)` with the fabricated `Observation:`/`Final Answer` truncatedthird_party · github.com/crewAIInc/crewAI

Third-party tool-execution authenticity verification system (filesystem/subprocess monitoring); closed without merge.

Status basis: proposed; no evidence it was applied. Verification: not verified.

Change: https://github.com/crewAIInc/crewAI/pull/3378 · not merged

Excerpt of the change (demo_tool_verification.py, MIT):

@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""
+Tool Execution Verification Demo
+
+This script demonstrates the tool execution verification system by testing
+real vs fake tool implementations. It shows how the system can detect when
+tools are actually executing vs when they're fabricating results.
+
+Usage:
+    python demo_tool_verification.py
+
+The demo will:
+1. Test a real file writing tool that actually creates files
+2. Test a fake file writing tool that only pretends to create files
+3. Show the verification results for each
+"""
+
+import os
+import sys
+import tempfile
+from pathlib import Path
+
+# Add the src directory to the path so we can import our modules
+sys.path.insert(0, str(Path(__file__).parent / "src"))
+
+from crewai.utilities.tool_execution_verifier import (
+    verify_tool_execution
+)
+

Added real-time tool execution monitoring system that:contributor · github.com/crewAIInc/crewAI