Choose and force tool calls with the OpenAI API

Overview

Suppose you ask a model for a number from a paper. You could put the whole table in the prompt, or give the model a way to look up one row. With the OpenAI Chat Completions API, your code decides which tools to offer and whether the model gets to choose when to use them. The model proposes a function call. Your code executes it and sends back the result.

In this tutorial, we’ll use Paper MCP, a small toy library for working with research papers. It can search the text of a PDF you’ve imported, read a page, and crop a region you select. We’ll use it to see what the model asks for and what evidence the tool gives back. To keep things simple, you are encouraged to use the same library for Part 2 of your Paper Review assignment.

We'll try this with Gemma 4 through the course gateway. As you work through the tutorial and Part 2 of the Paper Review assignment, keep the following two questions in mind:

1. Can the model choose a useful tool and supply the right arguments?
2. Once it has the result, can it use that evidence to answer the question?

Outcomes

After working through the examples, you should be able to …

• choose which functions to offer in a Chat Completions request
• let the model decide, require a call, or force a particular function
• execute a function locally and return its result using the matching call ID
• check the tool choice, its arguments, and the final answer separately

Start Paper MCP

There are many ways to give language models tools. One standard is the Model Context Protocol, or MCP. It gives an application a common way to discover and call tools provided by a server. Here, Paper MCP runs on your computer and supplies tools for working with PDFs. Your code will talk to two services. It sends prompts and function declarations to the course Model Gateway using Chat Completions. When the model requests a function, the client (the code you're running on your computer) calls Paper MCP and sends the result back to the model.

We’ll use Scientific Discovery as Link Prediction in Influence and Citation Graphs as our paper for the examples that follow. This academic paper asks whether graphs of scientific findings and citations can help predict which connections researchers will report later.

First, we need to clone the Paper MCP repository, install the library in a Python 3.12 or newer environment, and import the example paper.

# Clone the course repository over SSH and enter it.
git clone ssh://git@git.hlt.parsertongue.org/teaching/paper-mcp.git
cd paper-mcp

# Create and activate a Python environment for this checkout.
python -m venv .venv
source .venv/bin/activate

# Install Paper MCP and its command line tools from this checkout.
python -m pip install .

# Make folders for source PDFs and files generated during the tutorial.
mkdir -p papers traces

# Download the example paper into the papers folder.
curl --fail --location https://aclanthology.org/W18-1701.pdf \
  --output papers/W18-1701.pdf

# Index the PDF locally and print its document ID.
pdf-library import papers/W18-1701.pdf

The example starts Paper MCP for you as a local subprocess^†.

Run the complete example

Stay in the root directory of your local paper-mcp clone. The install step gives you paper-tool-choice. It prints each tool request, its result, and the answer. Follow the code excerpts alongside the complete runnable example in Paper MCP.

export DOCUMENT_ID="paste the document_id from pdf-library import here"

# Copy the gateway base URL, project API key, and model ID from your project.
export GATEWAY_BASE_URL="paste the gateway base URL here"
export GATEWAY_API_KEY="paste your project API key here"
export GATEWAY_MODEL="gemma4-12b"

# Start with no tools, then try the three tool modes.
paper-tool-choice "$DOCUMENT_ID" none
paper-tool-choice "$DOCUMENT_ID" auto
paper-tool-choice "$DOCUMENT_ID" required
paper-tool-choice "$DOCUMENT_ID" read_pdf_pages

The gateway base URL should include /v1. The project must allow the two tools declared by this example. Follow Model Gateway basics to find the project settings.

Let the model choose a tool

Let’s ask for the precision reported in the Full model row of Table 5 on PDF page 5.

The example imports two function declarations from Paper MCP.

from pdf_mcp.tools import PAGE_LOOKUP_TOOLS

PAGE_LOOKUP_TOOLS contains search_pdf and read_pdf_pages. The library also provides CITATION_TOOLS, IMAGE_TOOLS, and ALL_TOOLS for other workflows. Use the subset that fits the question. Each declaration describes the function and its arguments.

The script connects to the gateway with AsyncOpenAI and to Paper MCP with FastMCP’s Client. It puts the imported document ID in the prompt. For a page read, Gemma supplies that ID and an inclusive page range. The script checks the ID and restricts the range to one page before making the MCP call.

reply = await client.chat.completions.create(
    model=model,
    messages=messages,
    tools=PAGE_LOOKUP_TOOLS,
    tool_choice="auto",
    max_tokens=2048,
)

With auto, Gemma can answer directly or ask to call a tool. Look at reply.choices[0].message.tool_calls to see what it did. For this question, I'd look for read_pdf_pages with the imported document ID and both page bounds set to 5. A search can also work if the passage it returns contains the row and its headers. Getting a tool call is only the first check.

Require a call or force a function

Keep the same question and tool list. Change tool_choice to see how much of the decision you want to make in your code.

If you already know the lookup should be a page read, forcing^† read_pdf_pages lets you check whether Gemma can identify the page. With auto, you can also see whether it recognized that a lookup would help. When you start a fresh request, you can narrow the choices by passing only the tools relevant to that step^†. The Chat Completions API reference describes these options under tool_choice.

For the first request in the baseline without tools, leave out tools. On the course gateway, declaring tools on that turn with tool_choice="none" returned a 502.

baseline = await client.chat.completions.create(
    model=model,
    messages=messages,
    tool_choice="none",
    max_tokens=2048,
)
print(baseline.choices[0].message.content)

Before a tool runs, the gateway or client can check the arguments against a set of rules. We call that input validation. For example, a page read needs two integer page numbers. Validation can tell you whether those arguments are acceptable. You still have to check whether the page answers your question.

To force the page read, declare your tools and name read_pdf_pages in tool_choice. That fixes which function Gemma must request. Gemma fills in the arguments from the prompt^†. It copies the imported document ID and uses the question’s reference to PDF page 5 to set both ends of the page range to 5.

forced = await client.chat.completions.create(
    model=model,
    messages=messages,
    tools=PAGE_LOOKUP_TOOLS,
    tool_choice={"type": "function", "function": {"name": "read_pdf_pages"}},
    max_tokens=2048,
)

In the response, look for read_pdf_pages in tool_calls and check its arguments. Here is what the decoded arguments should look like, with a placeholder for your imported document ID.

{
  "document_id": "<imported document ID>",
  "start_page": 5,
  "end_page": 5
}

The page range includes both endpoints, so setting both to 5 requests just one page. If Gemma asks for page 4, it has followed the forced tool choice and still requested the wrong evidence.

Run the tool and return its result

Gemma’s first response proposes a tool call. The local Python client reads the function name and arguments from that response. The execute helper validates them and calls paper_mcp.call_tool(...). That’s where the client contacts Paper MCP and gets the page back.

import json

from pdf_mcp.tool_choice import execute, tool_content

# The first response contains Gemma's requested calls.
first = forced.choices[0].message
messages.append(first.model_dump(
    include={"role", "content", "tool_calls"},
    exclude_none=True,
))

for call in first.tool_calls or []:
    arguments = json.loads(call.function.arguments)
    # execute() checks the arguments and calls the local MCP server.
    result = await execute(
        paper_mcp, document_id, call.function.name, arguments
    )
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": tool_content(result),
    })

Only after the tool has run does the client send a second request. At this point, tool_choice="none" tells Gemma to answer using the result already in messages. Keep the tool declarations in this second request so the gateway can validate the earlier call.

final = await client.chat.completions.create(
    model=model,
    messages=messages,
    tools=PAGE_LOOKUP_TOOLS,
    tool_choice="none",
    max_tokens=2048,
)
print(final.choices[0].message.content)

What Gemma 4 returned

Here is what happened when I ran the experiment with Gemma 4 12B (the google/gemma-4-12B-it-qat-w4a16-ct checkpoint, listed as gemma4-12b in the course gateway). I included the document ID in the prompt and asked Gemma for the precision on PDF page 5.

In another run without tools, it invented a different Table 5!

Check the answer separately from the tool call. Paper MCP returned the requested page in all three tool runs. 0.268 is the F1 score. The requested precision is 0.528.

PDF text extraction can keep the numbers while making it harder to tell which column each one belongs to. On the original Table 5 page, you can see 0.268 under F1 and 0.528 under Precision. The optional vision tutorial shows what a crop can preserve.

In a separate request, I appended the following line to the same question. I transcribed it from the table and put each column name next to its value. Gemma returned 0.528. This comparison changes both the amount of text and how the values are labeled.

Full model: F1=0.268, Precision=0.528, Recall=0.176.

Why bother?

Yes, I could read the precision straight from the table. The point of giving the model a row I labeled by hand was to see whether it could answer once the column names and values were explicitly paired. That gives us one way to investigate where the failure happened and what might help. The goal of Part 2 is to understand how, where, and why models fail. You can decide for yourself whether using a model makes sense for a particular task.

Gemma is a multimodal LLM. It can take both text and images as input. What if we gave it an image of the table? In two trials with the same question, Gemma gave 0.268 with the extracted page text alone and 0.528 when I added a crop of the table. The crop alone also gave 0.528 in both trials. For the brave and curious, the optional vision tutorial walks through that comparison.

Try the same idea with paper evidence

The lookup example gives us a small way to inspect tool calling. The examples below show other things a tool could provide. For example, Paper MCP can search an imported PDF or return a page's extracted text. A client can expose either operation to our selected model as a function and record which evidence came back. The Paper MCP walkthrough shows how to inspect those results.

Find the evidence for a claim

Let’s ask how the authors evaluated their predictions. What did the all positive baseline assume about links before and after the 2012 cutoff?

Try searching for backtesting with search_pdf. In my run, the search returned several pages, including page 4. Follow that hit with read_pdf_pages to get the surrounding text. Then check the caption of Table 4 on PDF page 4.

What would count as a correct answer? The baseline predicts that the link from A to C will be discovered after the cutoff if both the link from A to B and the link from B to C already existed before it. An answer that only gives the year or says “the baseline predicts all links” leaves out which links qualify. If the tool result contains a general discussion of backtesting, try giving the model the Table 4 caption. If you use the CLI in the repository walkthrough, you can check what the model received in its trace^†.

Check whether the paper supports a claim

Suppose the model says AdaBoost had the highest recall of all the systems in Table 4. Check the Recall column and pay attention to which rows the claim covers. AdaBoost reports 0.178. The neural network reports 0.206, and the all positive baseline reports 1.0. AdaBoost does lead on P@10 and MAP in that table. A claim can sound plausible when it borrows a result from the wrong column.

Next steps

For Part 2 of the Paper Review assignment, pick one question from your paper with an answer you can check. Use the prompt in Class Maestro to plan your comparison. Look at which tool the model requested, what it got back, and whether the evidence supports its answer.

Start with the runnable example in Paper MCP. You can change the prompt or tool subset in the code. To try your own question and save a record for the write-up, set DOCUMENT_ID to your paper’s ID and run this with a new trace filename each time.

paper-tool-choice "$DOCUMENT_ID" auto \
  --question "Replace this with your question about the paper." \
  --trace traces/my-paper-auto.jsonl

The trace records the question, model, tool calls, returned evidence, and answer. Use it to check what happened and select the parts that support your writeup.

If you’d like to explore how to use a model’s vision capabilities, the optional Test image evidence from a paper tutorial shows how to use a table crop as evidence.