Advanced Agent Reasoning and Memory

AI agents become more capable when they can reason through problems, use tools, learn from results, and remember useful information.

Two important capabilities make this possible:

• Reasoning helps an agent decide what to do next.

• Memory helps it use relevant information from previous interactions or actions.

For example, an AI coding agent fixing a bug may inspect the code, form a hypothesis, modify a file, run tests, analyze the result, and change its approach if the fix fails.

This chapter explores important AI agent reasoning and memory techniques, including ReAct, planning, reflection, critic patterns, self-correction, Tree of Thoughts, memory retrieval, memory management, and context management.

Why Advanced Reasoning Matters

Many real-world tasks require multiple steps where each decision depends on information discovered earlier.

Suppose an agent receives this request:

Find why our checkout API became slow after the latest deployment and recommend what we should do.

The agent may need to:

1. Check recent deployments.

2. Analyze metrics and logs.

3. Investigate possible causes.

4. Compare evidence.

5. Reject incorrect hypotheses.

6. Recommend an action.

For example, the agent might initially suspect the database. If database metrics are normal, it should update its reasoning and investigate another cause instead of continuing with the original assumption.

Advanced reasoning is especially useful for coding, research, troubleshooting, data analysis, planning, and tool-based workflows.

However, deeper reasoning is not always necessary. For simple tasks, additional reasoning steps may only increase latency and cost.

Reasoning in AI Agents

AI agent reasoning is the process an agent uses to decide how to move from its current state toward a goal.

A simple AI agent reasoning loop looks like this:

Goal
  ↓
Understand Current State
  ↓
Choose Action
  ↓
Perform Action
  ↓
Observe Result
  ↓
Update State
  ↓
Choose Next Action

For example, a coding agent fixing a failing test might follow this process:

Goal: Fix failing payment test
        ↓
Read error logs
        ↓
Missing "currency" field
        ↓
Inspect database migration
        ↓
Migration was not applied
        ↓
Apply migration
        ↓
Rerun test
        ↓
Test passes

The agent does not need to know the complete solution before starting. It can act, observe the result, update its understanding, and decide what to do next.

This basic idea appears in several modern agent reasoning patterns, including ReAct.

ReAct (Reason + Act)

ReAct stands for Reasoning and Acting. It is an agent reasoning approach where a model combines reasoning with actions and uses observations from the environment to decide what to do next.

Instead of reasoning once and then acting, ReAct follows an iterative loop:

Reason
  ↓
Act
  ↓
Observe
  ↓
Reason Again
  ↓
Continue or Finish

For example, suppose an agent needs to answer:

Is Product X cheaper today than it was last month?

The agent needs real price data rather than relying only on its existing model knowledge.

Goal:
Compare current and previous price

Action:
Get current price

Observation:
$79

Action:
Retrieve last month's price

Observation:
$99

Reason:
Current price is $20 lower

Answer:
Product X is $20 cheaper than last month

The action can involve tools such as a search engine, database, API, calculator, file system, or code execution environment.

A practical ReAct workflow therefore looks like this:

User Goal
   ↓
Decide Next Action
   ↓
Use Tool
   ↓
Observe Result
   ↓
Enough Information?
   ├── No  → Continue
   └── Yes → Final Answer

The key idea behind ReAct is simple: reasoning decides what to do, actions interact with the environment, and observations provide new information for the next decision.

In production AI agents, the model's private reasoning does not need to be exposed. What matters is the underlying loop: decide → act → observe → update → continue.

Planning vs ReAct

Planning and ReAct both help agents solve multi-step problems, but they organize reasoning differently.

• Planning creates or maintains a sequence of steps toward a goal.

• ReAct decides the next action using the latest observations from tools or the environment.

Suppose an agent needs to research a company.

A planning approach might start with:

1. Research company background
2. Collect financial information
3. Identify major competitors
4. Check recent developments
5. Compare findings
6. Prepare report

A ReAct-style agent may proceed more dynamically:

Search company
      ↓
Discover recent acquisition
      ↓
Investigate acquisition
      ↓
Check financial impact
      ↓
Enough evidence?
      ↓
Prepare answer

The main difference is how the next steps are determined.

In practice, the two approaches can be combined. An agent can create a high-level plan and then use a ReAct-style loop to execute and adapt each step.

Iterative Reasoning and Action

Iterative reasoning and action means solving a problem through repeated cycles of action, feedback, and improvement.

A simple loop is:

Observe → Decide → Act → Evaluate → Repeat

For example, an AI coding agent may modify code and run tests:

Edit Code
    ↓
Run Tests
    ↓
3 Tests Fail
    ↓
Inspect Failures
    ↓
Revise Code
    ↓
Run Tests
    ↓
All Tests Pass

The key advantage is feedback. Each tool result or environmental observation tells the agent whether its previous action worked and helps it decide what to do next.

However, every iterative agent needs a stopping condition. Otherwise, it may continue calling tools or retrying without making useful progress.

Common limits include:

• Maximum attempts or tool calls

• Time limits

• Token or cost budgets

• Success or confidence thresholds

• Human escalation

A good agent should not only know how to continue, but also when to stop.

Self-Reflection

Self-reflection is a reasoning technique where an agent evaluates its own previous output or actions, identifies weaknesses, and improves its next attempt.

A simple reflection loop is:

Generate
   ↓
Evaluate
   ↓
Find Weakness
   ↓
Revise

For example, a research agent may conclude:

Company A is growing faster because its revenue increased by 40%.

Before finalizing, it checks the evidence and discovers that Company B grew by 55%. The agent should recognize the mistake and revise its conclusion.

Reflexion

Reflexion extends this idea by allowing an agent to learn from feedback across attempts.

After a failure, the agent can generate a useful lesson and store it in episodic memory:

Previous Failure:
Changed the API handler before checking the schema.

Lesson:
For schema-related errors, inspect the schema and migrations
before modifying application logic.

On a later attempt, the agent can use this reflection to make a better decision.

Importantly, this does not require retraining or changing the model's weights. The improvement comes from providing relevant lessons from previous attempts as context for future reasoning.

Critic and Reviewer Patterns

In self-reflection, the same agent evaluates its own work. In a critic or reviewer pattern, generation and evaluation are separated.

Writer Agent
     ↓
Draft
     ↓
Reviewer Agent
     ↓
Feedback
     ↓
Writer Agent
     ↓
Revised Draft

The reviewer can focus on specific quality checks. For example:

• Code Reviewer — correctness, security, edge cases, and missing tests.

• Research Reviewer — unsupported claims, weak evidence, contradictions, and missing sources.

This separation can improve evaluation because the reviewer receives a different role and instructions.

However, a critic adds extra model calls, latency, and cost, and its feedback can still be wrong.

For important tasks, use objective verification whenever possible:

Code        → Run Tests
SQL         → Execute on Test Data
Calculation → Calculator
Citation    → Verify Source
JSON        → Schema Validation

A useful principle is: prefer verifiable evidence over another model's opinion whenever reliable verification is available.

Retry and Self-Correction

Retry and self-correction allow an agent to recover from failures by using error feedback to improve the next attempt.

For example, an agent generates:

SELECT customer_name
FROM orders;

The database returns:

Error: column "customer_name" does not exist

Instead of repeating the same query, the agent can inspect the schema, identify the correct column, and generate a new query.

A useful retry loop is:

Attempt
   ↓
Failure
   ↓
Analyze Error
   ↓
Identify Cause
   ↓
Change Approach
   ↓
Retry

The key principle is that a retry should use new information. Simply repeating the same failed action is usually not self-correction.

Before retrying, the system should determine:

• What failed?

• What information did the failure provide?

• What should change in the next attempt?

Retries should also have a limit. After repeated failures, the system can use a fallback, try a different tool or strategy, or escalate to human review.

Tree of Thoughts (ToT)

Tree of Thoughts (ToT) is a reasoning approach where a language model system explores multiple possible reasoning paths, evaluates them, and continues with the most promising ones.

Instead of following only one path:

Problem
   ↓
Idea A
   ↓
Idea A1
   ↓
Answer

Tree of Thoughts can explore alternatives:

                Problem
              /    |    \
             A     B     C
            / \   / \   / \
          A1  A2 B1 B2 C1 C2

The system can evaluate branches, continue promising paths, discard weak ones, and backtrack when necessary.

This approach is useful when early decisions can significantly affect the final solution, such as:

• Complex planning

• Search and strategy problems

• Puzzles

• Some mathematical reasoning tasks

The main drawback is cost and latency. Exploring and evaluating multiple paths requires more inference than following a single reasoning path.

Therefore, Tree of Thoughts is most useful when exploring alternatives provides enough value to justify the additional computation.

Advanced Planning Strategies

Different tasks require different planning strategies.

A simple task may need only a few steps:

Search → Analyze → Answer

A complex task may require hierarchical planning, where a large goal is divided into smaller tasks:

Goal
├── Research
│   ├── Find Sources
│   └── Extract Evidence
├── Analysis
│   ├── Compare Evidence
│   └── Find Contradictions
└── Output
    ├── Write Draft
    └── Verify Claims

Common planning strategies include:

• Plan-First Execution — Create the main plan before execution begins.

• Dynamic Replanning — Update the plan when new information changes the situation.

• Hierarchical Planning — Break a large goal into smaller goals and tasks.

• Parallel Planning — Execute independent tasks concurrently when possible.

• Search-Based Planning — Explore and evaluate multiple possible paths before choosing one.

The right strategy depends on the task. For predictable workflows, deterministic application logic is often simpler and more reliable.

Use agentic planning when the next steps depend on uncertain, changing, or newly discovered information.

Memory Retrieval

Memory retrieval is the process of finding stored information that is relevant to an agent's current task.

For example, suppose a customer tells a support agent:

The replacement laptop has the same battery problem as the first one.

The agent may need to retrieve:

• Original laptop complaint

• Previous troubleshooting steps

• Replacement order details

• Information about the replacement device

• Previous support decisions

It does not need the customer's entire conversation history.

A useful principle is:

Retrieve what is relevant now, not everything that has been stored.

Agent memory can be retrieved using signals such as:

• Semantic similarity — Memories related in meaning

• Keywords — Exact or related terms

• Metadata — User, product, date, category, or task information

• Recency — More recent memories

• Importance — High-value or significant memories

In practice, memory systems often combine several of these signals to select the most useful information.

This allows an AI agent to use past information without filling its context with unnecessary history.

Memory Storage and Updates

An LLM agent memory system needs rules for deciding what to store, update, and remove.

Not every interaction should become long-term memory. For example:

"My project uses PostgreSQL." → Useful to store
"Thanks."                     → Usually not useful

Common types of agent memory include:

Memory also needs to be updated when information changes.

For example, if the stored deployment date is September 10 but later changes to September 14, the system should update the current fact instead of treating both dates as equally valid.

A reliable memory system therefore needs lifecycle rules for:

Create → Update → Resolve Conflicts → Expire or Delete

It should also maintain provenance where necessary—information about where a memory came from and when it was recorded or updated.

The goal is not to remember everything. It is to maintain useful, relevant, and trustworthy memory for future tasks.

Memory Selection

Memory selection decides which stored memories should be included in the agent's active context for the current task.

A system may store millions of memories, but only a small number may be relevant to a particular request.

Memory selection can consider signals such as:

• Relevance — How closely the memory relates to the current task

• Recency — How recently the information was created or updated

• Importance — How significant the memory is

• Task relationship — Whether it belongs to the current project, user, or workflow

For example, while preparing for a meeting about Project Atlas, the agent might retrieve:

✓ Atlas launch moved to October.
✓ Atlas database migration failed last week.
✓ Atlas security review is incomplete.

✗ User ordered lunch yesterday.
✗ User asked about Python six months ago.

Good memory selection improves the signal-to-noise ratio and reduces unnecessary token usage.

Poor selection can hurt agent performance because irrelevant memories consume context and compete with useful information.

Memory Summarization

Long-running agents can accumulate large conversation and task histories. Sending the entire history to the model on every request increases token usage, cost, and context size.

Memory summarization compresses older information while preserving the most important facts and decisions.

For example:

Raw History:
Monday    → Discussed database options
Tuesday   → Selected PostgreSQL
Wednesday → Changed schema design
Thursday  → Migration failed
Friday    → Fixed migration and approved final schema

This can be summarized as:

Project Memory:
The team selected PostgreSQL. The schema was revised,
the first migration failed, and the corrected migration
succeeded. The final schema is approved.

The summary requires fewer tokens while keeping the important project state.

However, summarization is lossy. Important details may be omitted or simplified.

For important applications, a useful design is:

Raw History → Durable Source of Detail
Summary     → Compact Working Memory

The agent can normally use the summary and retrieve the original records when exact details are needed.

Context Management

Memory and context are related, but they are not the same.

• Memory is information stored for possible future use.

• Context is the information currently provided to the model for a specific task.

A simple flow is:

Long-Term Memory
       ↓
Retrieve Relevant Memories
       ↓
Current Request + Instructions
+ Task State + Tool Results
       ↓
Active Context
       ↓
Model

An agent's active context may include:

• System instructions

• Current user request

• Recent conversation

• Retrieved memories

• Task state

• Tool results

• Relevant documents

Good agent context management means providing enough information to complete the task without filling the context with unnecessary data.

More context is not always better. Too much irrelevant information increases token usage and cost and can make important information harder for the model to use effectively.

The goal is not maximum context, but relevant and useful context.

Memory in Long-Running Agents

Memory becomes especially important when an agent works across multiple tasks, sessions, or days.

For example, a project-management agent may learn:

Monday    → Release target: September 20
Wednesday → Payment integration is blocked
Friday    → Release moved to September 25

If someone later asks:

What is putting the release at risk?

the agent needs relevant information from the project's history, not just the latest conversation.

A long-running AI agent memory architecture may organize information into layers:

Current Working Context
        ↓
Short-Term Task State
        ↓
Session / Conversation History
        ↓
Long-Term Memory
        ↓
External Source of Truth

The external source of truth is especially important. Stored or generated memory may become outdated, while systems such as a database, project tracker, or CRM may contain the latest authoritative information.

A useful principle is: memory should help the agent use past information, but it should not replace the current source of truth.

Common Memory and Reasoning Challenges

Advanced reasoning and memory improve agent capabilities, but they also introduce new failure modes.

The key principle is to add reasoning, reflection, retries, and memory only where they improve reliability or task performance.

A simpler agent architecture is often better when it can solve the task reliably.

Summary

Advanced agent reasoning and memory help AI agents solve complex, multi-step tasks.

• ReAct combines reasoning, actions, and observations.

• Planning organizes tasks into steps.

• Reflection and critics help evaluate and improve results.

• Retries and self-correction use failures to improve the next attempt.

• Tree of Thoughts explores multiple possible reasoning paths.

• Memory stores and retrieves useful past information.

• Context management provides only the information needed for the current task.

The key principle is simple: use reasoning and memory only when they improve the agent's performance, reliability, or ability to adapt.