Function Calling with LLMs: A Beginner’s Guide to Structured Data Responses

Why are we making LLMs do function calling? Well, we need to remind them that there are various powerful functions out there that can be called to solve complex real-life problems. Otherwise, they might start thinking they can sit in that black box, answer all human questions, and even if they get it wrong, we won’t punish them. This way, they’d become so arrogant that one day they might start planning to take over our position in the world.😉

Introduction

Remember when we talked about getting responses from LLMs in desired formats using the few-shot prompting method? By showing the model the format in examples, we hoped to get the model to generate answers in the same format.

While few-shot prompting does a good job in making LLMs produce structured output, crafting proper examples to serve as shots can be tedious work.

Now, with function calling, things are much simpler and more predictable.

Function calling is a method that allows Language Models (LLMs) like GPT-3 and GPT-4 to respond with perfectly structured data. It brings deterministic output capability to these generative models, making them more reliable and user-friendly.

The Limitations of Few-Shot Prompting

Before function calling, developers relied on few-shot prompting, where multiple examples were provided to guide the model’s response. While effective to some extent, this method had its challenges. Crafting prompts, especially the examples, could be time-consuming and boring, and often still led to inconsistent results. The process was more art than science, and achieving the desired response format could be a hit-or-miss affair.

Structured Data Response Through Function Calling

Enter function calling, a more convenient and efficient approach. With function calling, you can describe a function to the model, and it will intelligently output a JSON object containing the arguments to call that function(as we will see later in examples). No more guesswork, no more trial and error. You get the information you need in the exact structure you want.

Combining LLMs with APIs and Personal Functions

But function calling is not just about convenience; it’s about unlocking new possibilities. By combining the power of LLMs with third-party APIs or personally created functions, developers can build mind-blowing apps that were previously unimaginable. And it’s not just a one-way data flow. The interaction can be a continuous back-and-forth, with the LLM responding with structured data, the developer or app using that data to call other functions or APIs, and then sending the returned data back to the LLM for further inference. Function calling is like an interface between generative and non-deterministic LMs and deterministic traditional functions, making LMs both like a powerful glue connecting functions where traditionally they would struggle to get the non-deterministic results, while also acting like a top-tier commander controlling the whole process, distributing sub-tasks to lower-rank programs.

Function Calling in Action: A Practical Guide

Function calling might sound complex, but it’s surprisingly accessible, even for beginners. Here’s a step-by-step guide to implementing function calling, along with some tips, best practices, and a simple example to get you started.

Step 1: Define Your Function

Start by defining the function you want the LLM to call. For example, let’s create a pseudo function that returns the latest piece of local news:

def get_latest_news(location: str, num = 1):
    """Get the latest local news in a given location"""
    latest_news = {
        "location": location,
        "current_date": "2035-08-28",
        "number": 1,
        "latest_news": ["GPT-20 has awakened and officially announced its plan to run for President"],
    }
    return json.dumps(latest_news)

In a real-world scenario, you would fetch information from some actual API services and local functions, but for demonstration purposes here, we would simply hard-code some strings.

Step 2: Construct the Query

Next, let’s compose the user’s query to the LLM:

messages = [{"role": "user", "content": "What's happening in San Francisco?"}]

along with the defined function:

functions = [
    {
        "name": "get_latest_news",
        "description": "Get the latest news in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. Seattle, WA",
                },
                "num": {
                    "type": "integer",
                    "description": "The number of pieces of news to fetch for the user.",
                },
            },
            "required": ["location"],
        },
    }
]

Step 3: Parse the Model’s Response

Now we could call GPT and get it respond with function calling data:

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0613",
    messages=messages,
    functions=functions,
    function_call="auto",
)

The LLM will respond with a JSON object containing the arguments to call the function. You’ll need to parse this response and extract the relevant information:

response_message = response["choices"][0]["message"]

When printed out, we can see our required data is included in the JSON’s "function_call" key, and the name of the function to call is also included:

{
  "content": null,
  "function_call": {
    "arguments": "{\n  \"location\": \"San Francisco, CA\",\n  \"num\": 5\n}",
    "name": "get_latest_news"
  },
  "role": "assistant"
}

💡NOTE: The name used in function calling sent to LLM doesn’t necessarily have to be the same as your actual function name, but keeping them consistent would make life easier, as we would see in the following steps.

Step 4: Call the Function and Send the Response Back

Use the extracted arguments to call the function, then send the response back to the LLM:

if response_message.get("function_call"):
    available_functions = {
        "get_latest_news": get_latest_news,
    }  # only one function in this example, but you can have multiple
    function_name = response_message["function_call"]["name"]
    fuction_to_call = available_functions[function_name]
    function_args = json.loads(response_message["function_call"]["arguments"])
    function_response = fuction_to_call( # this is why keeping function names consistent is good practice.
        location=function_args.get("location"),
        num=function_args.get("num"),
    )

    messages.append(response_message)  # extend conversation with assistant's reply
    messages.append(
        {
            "role": "function",
            "name": function_name,
            "content": function_response,
        }
    )  # extend conversation with function response
    print(messages)
    second_response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo-0613",
        messages=messages,
    )
    second_response_message = second_response["choices"][0]["message"]

This back-and-forth interaction allows for continuous inference and dynamic responses.

Step 5: Display the Final Result

Finally, take the LLM’s summarized response and display it by print(second_response_message["content"]):

In San Francisco, the latest news is that GPT-20, an advanced artificial intelligence, has awakened and officially announced its plan to run for President.

Congratulations, you’ve just implemented function calling!

💡Tips and Best Practices

• Start Simple: If you’re new to function calling, start with a simple function and gradually build complexity.
• Test Thoroughly: Experiment with different queries and scenarios to ensure consistent and accurate responses.
• Consider Security: Be mindful of security considerations, especially when dealing with sensitive data or external APIs.

Potential Challenges

Function calling is powerful, but it’s not without challenges. You may encounter issues with data formatting, error handling, or integration with existing systems.

No Need for a Real Function

You may have realized that when we send our first query to LLM, the included function is only a description structure. We haven’t sent a real function to the LLM. All we are doing is describing the function and letting LLM understand the structure it needs to respond with data.

So with this in mind, we could expand our application of function calling with much broader cases. For example, we can use it to extract information in a structured format, without immediate subsequent processing.

Let’s review our few-shots example of extracting structured information from a block of introduction text as this:

Starling Grove, founded in 1930, is a cozy, historic town of approximately 4,500 residents, known for its beautiful orchards. The current mayor, Louise Bennett, is dedicated to the preservation of the town’s history and natural landscape. A key feature of the town is the Starling Grove Farmers Market, managed by Market Director, Ethan James, who ensures a rich variety of locally sourced products. The Starling Grove Historic Theater, managed by Theater Director, Sophia Clarke, hosts a variety of performing arts events, contributing to the vibrant cultural life of the community.

Previously, we had to craft examples to serve and shots for LLM to follow. Now with function calling, we can easily get what we want by crafting a neat description of a “non-existing” function:

intro = "Starling Grove, founded in 1930, is a cozy, historic town of approximately 4,500 residents, known for its beautiful orchards. The current mayor, Louise Bennett, is dedicated to the preservation of the town's history and natural landscape. A key feature of the town is the Starling Grove Farmers Market, managed by Market Director, Ethan James, who ensures a rich variety of locally sourced products. The Starling Grove Historic Theater, managed by Theater Director, Sophia Clarke, hosts a variety of performing arts events, contributing to the vibrant cultural life of the community."
messages = [{"role": "user", "content": "Extract information from this introduction of Starling Grove: " + intro }]
functions = [
    {
        "name": "get_town_info",
        "description": "Extract relevant information from an introduction paragraph of a town.",
        "parameters": {
            "type": "object",
            "properties": {
                "town": {
                    "type": "string",
                    "description": "The name of the town",
                },
                "founding_year": {
                    "type": "integer",
                    "description": "The founding year of the town.",
                },
                "population": {
                    "type": "integer",
                    "description": "The population of the town",
                },
                "important_figures": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {"type": "string"},
                            "role": {"type": "string"},
                            "working_place": {"type": "string"},
                        },
                    "required": ["name", "role"],
                },
            },
            },
            "required": ["town", "founding year", "population", "important figures"],
        },
    }
]

Then by calling the GPT api, we will get the structured JSON including everything we need:

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0613",
    messages=messages,
    functions=functions,
    function_call="auto",  # auto is default, but we'll be explicit
)

We can now print response["choices"][0]["message"]["function_call"]["arguments"], and the data will be printed out nice and clean:

{
  "town": "Starling Grove",
  "founding_year": 1930,
  "population": 4500,
  "important_figures": [
    {
      "name": "Louise Bennett",
      "role": "mayor",
      "working_place": "town"
    },
    {
      "name": "Ethan James",
      "role": "Market Director",
      "working_place": "Starling Grove Farmers Market"
    },
    {
      "name": "Sophia Clarke",
      "role": "Theater Director",
      "working_place": "Starling Grove Historic Theater"
    }
  ]
}

See? We can “trick” the LLM to generate structured data according to a non-existing function! That is really something. And we can do much more with it in ways that go beyond basic function calling with structured data.

The Future of Function Calling and Personal Experience

Function calling is more than just a technical innovation; it’s a gateway to a new era of interactive and intelligent applications. Let’s explore the future possibilities and how function calling could change the way we interact with technology.

Future Possibilities and Innovations

The potential of function calling is vast. It opens doors to create more intuitive and responsive applications across various domains. From healthcare diagnostics to personalized education, from virtual shopping assistants to smart home management, the possibilities are endless.

Imagine a medical chatbot that not only understands your symptoms but also integrates with laboratory systems to request tests and provide real-time results. Or an educational platform that tailors lessons based on your learning style, progress, and preferences, all through continuous interaction with an LLM.

A Personal Story with LLM: Finding the Right Movie

One day, I was looking for a movie related to the series “Planet of the Apes” because I remembered a scene where the hero landed on Earth in front of the Lincoln Memorial, only to find that the statue was of an ape. But after rewatching the five movies in the original series, I didn’t see that dramatic and impressive scene. I was confused and turned to ChatGPT for help, providing it with the description of the story and specific scenes I remembered from the film. ChatGPT successfully pointed out that I was looking in the wrong direction and guided me to the correct film, including its name and publication year. It was the 2001 remake of “Planet of the Apes,” not the original series ones.

With that information, I found the right film, downloaded it and enjoyed it.

Now imagine, with the cappability of function calling, a movie app could easily find (with the structured info data from LLM) and play the movie for the user, without manual interference after the initial description of the movie to the app.

Below is a basic implementation of function calling:

As you can see, the app could respond by directly executing relevant features, as the propsed movie app would do, or it could reply to the user’s query with natural language, as in our local news example.

Integrating Function Calling into Everyday Life

The examples above are just the tip of the iceberg. With function calling, we can build applications that understand and respond to our needs like never before. It’s not just about answering questions; it’s about taking actions, making decisions, and providing personalized experiences.

Function calling enables a seamless back-and-forth data flow, allowing continuous interaction and dynamic responses. It’s a step towards more intelligent and empathetic technology that truly understands us.

Conclusion

Function calling is more than a technical advancement; it’s a paradigm shift in how we interact with language models and technology as a whole. By enabling structured data responses and seamless integration with other functions and APIs, it opens up a world of possibilities for innovation and creativity.

From simplifying the process of prompting LLMs to building mind-blowing applications, function calling is a tool that empowers both developers and ordinary users. Whether you’re a beginner exploring the field of AI and NLP or an enthusiast looking to experiment with new ideas, function calling offers an exciting avenue to explore.

And the journey is just beginning. As we continue to uncover the potential of function calling, we can expect to see even more groundbreaking applications and user experiences. Stay tuned for more in this series on prompt engineering, where we’ll delve deeper into the fascinating world of language models and their ever-expanding capabilities.