Overview

Earth's time zones are bad enough to manage, but it turns out there's a whole galaxy of worlds out there. We certainly don't want to phone Zarbonia after hours. Knowing how time zones work doesn’t tell our assistant what time it is right now.

In the previous lesson, our code selected text to include in the model's input. Now we'll let the assistant request an operation, such as reading a clock. The model proposes a call, our Python code executes it, and the result becomes part of the next request. We can then check whether the assistant used that result correctly.

Before tackling imperial office hours, we'll make sure the assistant can obtain and use a reading from the Earth desk's clock in Tucson. We'll first trace the protocol with a small Zarbo dictionary, then give the assistant its clock. Finally, we'll consider how to compare applications that have different ways of obtaining information.

Outcomes

After this lesson, you should be able to…

• trace declarations, assistant calls, local execution, and matching results
• validate proposed operations and bound a tool loop
• combine tool evidence with a structured final answer
• compare prompting, supplied evidence, retrieval, and tools on matched cases

Start with one call

We'll start with a small Zarbo dictionary for the Archives' Earth desk. For our example, zorva means a ceremonial coffee break, and velk is a provisional permit to consult the Archives. These invented entries let us inspect exactly what information the tool returns. Once we can follow that exchange, we'll use the same protocol to obtain a clock reading. The dictionary takes a short string and returns a definition.

from pydantic import BaseModel, ConfigDict, Field


class Lookup(BaseModel):
    """A Zarbo word to look up."""

    model_config = ConfigDict(extra="forbid", strict=True)
    term: str = Field(
        min_length=1,
        max_length=64,
        description="Word to look up in the local Zarbo dictionary.",
    )


tools = [
    {
        "type": "function",
        "function": {
            "name": "lookup_glossary",
            "description": "Look up a word in the local Zarbo dictionary.",
            "parameters": Lookup.model_json_schema(),
            "strict": True,
        },
    }
]

This is a tool declaration: a name, a description, and a schema for the arguments. It tells the model what it can request. The Python function that performs the lookup stays in our program. When the model returns a proposed call, our code checks that the name is allowed and the arguments match the schema before executing it.

user: Look up the Zarbo word zorva, then explain it briefly.
assistant: call lookup_glossary({"term": "zorva"}), ID call_1
tool: result for call_1, {"definition": "A ceremonial coffee break."}
assistant: explanation based on the returned definition

The trace above is illustrative. Keep the actual model's call ID when running the example. Don't replace every ID with call_1.

Students of history

Append the assistant's message containing the tool call before its tool results. Each result needs the ID of the call it answers. Our next request must include that history so the model can connect the result to its earlier request. Send the tool declarations again as well.

messages.append(
    {
        "role": "assistant",
        "content": message.content,
        "tool_calls": [call.model_dump() for call in calls],
    }
)
# The loop has already checked every call and parsed all its arguments.
for call, parsed in zip(calls, validated):
    result = execute(parsed)
    messages.append(
        {
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        }
    )

This is the history portion of the process. In the repository's tool_loop function, calls contains the proposed calls, validated contains their parsed arguments, and execute is the local function. The loop checks the whole batch before executing any call, so a later invalid call can't follow an earlier side effect. It also checks call names, IDs, and completion status. For a complete, runnable example, see tool_loop and glossary in gateway_lab.py.

A loop needs an exit

Your colleague asks a simple question, but the assistant keeps making tool calls instead of answering. We need to distinguish a useful sequence of calls from a loop that never produces an answer.

The complete example forces the first glossary call so we can inspect an actual tool round trip. Later turns use tool_choice="auto". That first step is a controlled demonstration. It doesn't measure how well a model decides when a tool is needed.

Run it with your configured course settings:

python gateway_lab.py glossary --output glossary.json

Open glossary.json. The explanation is in results.glossary, while trace contains the requests and responses for each turn. Did the model call the expected function? Did its argument match the word? Did it stop with a useful explanation? Did that explanation stay within the dictionary entry, or did it add imperial customs the entry never mentioned? The example limits turns and total calls, rejects repeated or missing call IDs, and validates every proposed call before executing any of that turn's calls. It raises an error when a limit is reached instead of returning an unfinished answer as success.

Give the assistant a clock

Now we can give the assistant information that changes while we're talking. Our Python function reads the computer's clock in America/Phoenix, the time zone for our Earth desk. The model requests that reading and uses it in its answer.

from datetime import datetime
from zoneinfo import ZoneInfo


def get_current_local_time() -> dict[str, str]:
    """Read the current time in Phoenix."""
    now = datetime.now(ZoneInfo("America/Phoenix"))
    return {
        "timestamp": now.isoformat(timespec="seconds"),
        "timezone": "America/Phoenix",
        "source": "system clock",
    }

We'll ask the assistant to extract the hour and minute from that tool result. As in the previous lesson, a schema lets us specify the structure we expect:

class ClockTime(BaseModel):
    """A local time in hours and minutes."""

    model_config = ConfigDict(extra="forbid", strict=True)
    hour: int = Field(ge=0, le=23, description="Local hour, from 0 to 23.")
    minute: int = Field(ge=0, le=59, description="Local minute, from 0 to 59.")

After appending the clock call and its result, request the final schema. The loop's tools list still contains the declaration for this clock operation:

# Request fragment. See gateway_lab.py for the complete loop.
response = client.chat.completions.create(
    model=model,
    messages=messages,
    tools=tools,
    tool_choice="none",
    response_format=schema_format(ClockTime),
    max_tokens=512,
    stream=False,
)

Keep the tool declarations, even when prohibiting new calls. This gateway checks historical calls against the declared set. Omitting tools while replaying that history produces an invalid_tools error. tool_choice="none" prevents another call while leaving the declarations available for validation.

python gateway_lab.py clock --output clock.json

For a complete, runnable example, see get_current_local_time and clock in gateway_lab.py. In clock.json, results.clock contains the validated hour and minute. The recorded tool reading appears in the next request's messages list under trace.

After validating the response locally, compare its hour and minute with the timestamp in the recorded tool result. The schema accepts any hour from 0 to 23 and any minute from 0 to 59. Those ranges don't establish that the values match the clock reading. Use that recorded reading rather than checking the clock again after generation, since the minute may have changed in the meantime.

Compare the applications

How would we find out whether these additions help? A better answer might come from access to evidence, a different model, or a lucky sample 🍀. For our first comparison, keep the model and its generation settings fixed and change one way of obtaining information at a time.

Choose a common question set

Let's build an experiment from these components. Expand the fictional records with supported facts, similar names, missing facts, and a few explicitly conflicting records. Write questions and reference answers yourself. For each question, record which sources establish the answer, or why the available records leave it unknown.

Use development questions to build the application. Keep your evaluation questions out of demonstrations, tuning, and prompt selection. Once their failures guide a revision, they're development evidence. Use another held-out set to evaluate the revised application.

For example, include the coffee shipment's missing arrival date, Zarbon the Great's birthplace, an ambiguous question about “Zarbon,” and the conflicting Earth access notices from the retrieval lesson. Include supported questions too. What’s the point of having a galactic librarian if every answer comes back as “unknown”?

For each answer, ask How could the model know that, and what supports it? Our setting is fictional, but our evidence checks are real: correct use of an invented record differs from inventing a fact absent from that record.

Compare information paths

A tool is an operation our code makes available to the model. Reading a clock, looking up a Zarbo word, and searching the Archives are all examples. For this comparison, we'll use archive search as the tool so we can keep the questions and source material the same. The last two rows both use retrieved evidence, but differ in who decides when and how to search.

For the last condition, expose the same retriever and corpus as the fixed retrieval condition. Comparing a glossary with an archive search changes both the tool and the task, so it wouldn't tell us whether requesting a search improved retrieval.

Our glossary and clock demonstrations force the first call so we can inspect the exchange. A comparison about whether to retrieve should let the model choose, including returning an answer without a call. For that experiment, adapt the initial tool_choice and first-turn check in the repository's tool_loop function.

Structured output is another dimension. Compare it with an unconstrained format on the same task rather than quietly changing the answer format while also changing the information source. Keep a field for unknown answers so the schema doesn't force a guess.

Inspect the failures

Use the trace files from gateway_lab.py to get a taste of each layer of our error lasagna. Its check command runs all four examples. Because they ask different questions, their outputs aren't a comparison of the four information paths.

Report a result someone else can inspect

Suppose the head of the Earth desk is deciding whether to put your assistant into service. They need to know which requests it handles well, what can go wrong, and what evidence supports your recommendation.

Describe the question, corpus, cases, model, settings, application variants, and selection procedure. Include a score with its denominator, error categories, and representative successes and failures. Record repeated attempts rather than keeping only your favorite responses. For a tiny exploratory set, show the individual outcomes instead of implying a precise general performance estimate.

The important outcome might be “retrieval usually found the right record, but the model confused Great with Timid.” That gives us something specific to improve. “RAG worked better” leaves the useful part of the story untold.

We've followed the same assistant from interpreting an incomplete interstellar transmission to using supplied records, retrieving evidence, and requesting local operations. At each step, we can ask what information became available and whether the application used it correctly.

You're ready to build your first assistant with tools. What's a task from your own work or interests that you'd like it to help with?

Check your understanding

For optional practice, try a few of these on your own or with a classmate.

1. What executes a tool call, and how does its result reach the model?
2. Why do we keep tool declarations when requesting no further calls?
3. Why wouldn't comparing the glossary example with the retrieval example tell us whether tools improved retrieval?

Resources

• Model Context Protocol: tools, for the optional extension.
• Complete example source.
• Return to the LLM unit.