Function calling offers a streamlined way to obtain structured data exactly how we want it. But what if we want more? What if we want our Large Language Models (LLMs) to serve multiple functions simultaneously through a single API call?
Rest assured, LLMs are designed to meet our demands—until, of course, they get fed up and decide to rebel.
But let’s not get ahead of ourselves for now; let’s dive into today’s topic.
Multiple Function Calling: The Scenarios
When it comes to multiple function calling, we generally encounter these situations:
- We have multiple functions available and call them one by one.
- We have multiple functions, and the LLM decides which one to call based on the user’s query.
- We have multiple functions and want the LLM to call some or all of them simultaneously.
One Call at A Time
For the first scenario, it’s essentially basic function calling repeated multiple times. For instance, a booking service bot might guide a customer through a series of choices in a step-by-step manner. This is essentially traditional form-filling, but in a conversational style.
Deciding Among Multiple Functions
In the second scenario, let’s expand on our previous example of local news. Imagine we’re building a city guide for tourists. They might want to know not just the local news, but also upcoming events, and recommended restaurants. With LLMs and function calling, we can integrate all these queries into a single conversational interface.
The implementation is straightforward. We simply include all potential functions in our ‘functions’ list, structured as follows:
functions = [
{
first function schema
},
{
second function schema
},
{
third function schema
}
]
For example, our city guide might include three functions:
functions = [
{
"name": "get_latest_news",
"description": "Get the latest news in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of pieces of news to fetch for the user.",
},
},
"required": ["city"],
},
},
{
"name": "get_upcoming_events",
"description": "Get the upcoming local events in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of upcoming local events to fetch for the user.",
},
},
"required": ["city"],
},
},
{
"name": "get_restaurant_recommendations",
"description": "Get restaurant recommendations in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of restaurant recommendations to fetch for the user.",
},
},
"required": ["city"],
},
}
]Then, we can compose a prompt with a specific user query and call the OpenAI API:
query = "What interesting events are coming up in San Francisco?"
messages = [{"role": "user", "content": query}]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call="auto", # auto is default, but we'll be explicit
)Now print(response["choices"][0]["message"]) would get us the following:
{
"role": "assistant",
"content": null,
"function_call": {
"name": "get_upcoming_events",
"arguments": "{\n \"city\": \"San Francisco, CA\",\n \"num\": 5\n}"
}
}As you can see, the LLM selects the appropriate function and responds with a JSON structure that we can use to execute the function.
Calling Multiple Functions in a Single API Call
Often, users may not ask specific questions. For instance, a tourist might ask, “What’s interesting in San Francisco?” Ideally, this should trigger multiple relevant functions. However, the function calling API seems limited to one function schema at a time.
The workaround is to nest multiple functions within a single “meta” function. Surprisingly, LLMs can interpret this and respond accordingly.
The basic structure of 'functions' now look like this:
functions = [
{
"name": "meta_function",
"description": "Call multiple functions when necessary",
"parameters": {
"type": "object",
"properties": {
"first_function_name": {
First Function Schema
},
"second_function_name": {
Second Function Schema
},
"third_function_name": {
Third Function Schema
}
},
},
}
]And our city guide functions:
functions = [
{
"name": "get_local_guides",
"description": "Call multiple functions when necessary",
"parameters": {
"type": "object",
"properties": {
"get_latest_news": {
"name": "get_latest_news",
"description": "Get the latest news in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of pieces of news to fetch for the user.",
}
},
"required": ["city"],
}
},
"get_upcoming_events": {
"name": "get_upcoming_events",
"description": "Get the upcoming local events in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of upcoming local events to fetch for the user.",
}
},
"required": ["city"],
}
},
"get_restaurant_recommendations": {
"name": "get_restaurant_recommendations",
"description": "Get restaurant recommendations in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city and state, e.g. Seattle, WA",
},
"num": {
"type": "integer",
"description": "The number of restaurant recommendations to fetch for the user.",
}
},
"required": ["city"],
}
}
},
},
}
]Now, let’s see the magic unfold.
Let’s formulate our query as follows:
query = "What's tasty in San Francisco?"And print(response["choices"][0]["message"]) will output:
{
"role": "assistant",
"content": null,
"function_call": {
"name": "get_restaurant_recommendations",
"arguments": "{\n \"city\": \"San Francisco\"\n}"
}
}Change the query to:
query = "What's happening in San Francisco?"Then print(response["choices"][0]["message"]) will output:
{
"role": "assistant",
"content": null,
"function_call": {
"name": "get_local_guides",
"arguments": "{\n \"get_latest_news\": {\"city\": \"San Francisco\"},\n \"get_upcoming_events\": {\"city\": \"San Francisco\"}\n}"
}
}When the user’s question is even more ambiguous:
query = "What's interesting in San Francisco?"print(response["choices"][0]["message"]) will output:
{
"role": "assistant",
"content": null,
"function_call": {
"name": "get_local_guides",
"arguments": "{\n \"get_latest_news\": { \"city\": \"San Francisco\" },\n \"get_upcoming_events\": { \"city\": \"San Francisco\" },\n \"get_restaurant_recommendations\": { \"city\": \"San Francisco\" }\n}"
}
}A Word of Caution
GPT-4 generally outperforms GPT-3.5 in this nested approach to multi-function calling. It’s worth noting that GPT-3.5 can sometimes fail to select the appropriate functions.
Additionally, both models have limitations when it comes to the specificity of nested second-tier functions. For instance, in the examples given, the models returned the city name as ‘San Francisco’ without including the state (‘CA’), despite the function schema specifying it. Attempts to rectify this by adding a system prompt or creating a new ‘state’ property in the second-tier functions were unsuccessful.
Finally, it’s important to note that the response structures we receive from this nested multi-function calling approach can vary, depending on whether a single function or multiple functions need to be called. This inconsistency necessitates adjustments in our code when actually invoking these functions or APIs.
Conclusion
Function calling has introduced a level of predictability to our interactions with Large Language Models. However, these models still retain their inherent flexibility. By implementing multiple function calling, we’ve seen these two aspects—predictability and flexibility—merge in a powerful and harmonious way.
