The Promise and Reality of Function Calling
Function calling lets LLMs interact with external systems - databases, APIs, file systems. In theory, you define functions, the model calls them, magic happens.
In practice, models hallucinate function names, pass invalid arguments, call functions in the wrong order, and confidently return made-up results.
This post documents the patterns we use to achieve 98% tool call accuracy in production.
The Failure Modes
Before solutions, understand the problems:
| Failure Mode | Frequency | Example |
|---|---|---|
| Invalid arguments | 34% | String where number expected |
| Hallucinated functions | 18% | Calls tool that does not exist |
| Wrong function choice | 22% | Uses search when should use lookup |
| Missing required args | 15% | Omits required parameter |
| Type coercion errors | 11% | "123" instead of 123 |
Pattern 1: Schema-First Tool Definition
Do not let the model guess. Define explicit schemas.
Tool Schema Structure
Every tool needs:
| Field | Purpose | Example |
|---|---|---|
| name | Unique identifier | get_customer_orders |
| description | What it does (for LLM) | Retrieves order history for a customer |
| parameters | JSON Schema | See below |
| required | Mandatory params | ["customer_id"] |
| examples | Usage examples | Input/output pairs |
Description Best Practices
Good descriptions dramatically improve accuracy:
| Bad | Good |
|---|---|
| "Gets orders" | "Retrieves the complete order history for a specific customer, including order IDs, dates, items, and totals" |
| "Search" | "Searches the product catalog by keyword. Returns up to 10 matching products with names, prices, and availability" |
Parameter Descriptions
Every parameter needs a description:
| Without Description | With Description |
|---|---|
| 76% accuracy | 94% accuracy |
Pattern 2: Validation Layer
Never trust model output. Always validate.
Validation Pipeline
Every tool call goes through validation:
| Step | What It Checks | Action on Failure |
|---|---|---|
| Schema validation | Types match schema | Return error to model |
| Business rules | Values make sense | Return error with guidance |
| Authorization | User can do this | Block and log |
| Rate limiting | Not too many calls | Queue or reject |
Validation Results
| Validation Type | Catches | Recovery Rate |
|---|---|---|
| Schema | 89% of type errors | 96% |
| Business rules | 78% of logic errors | 84% |
| Combined | 94% of all errors | 91% |
Pattern 3: Error Messages That Help
When validation fails, tell the model how to fix it:
| Bad Error | Good Error |
|---|---|
| "Invalid input" | "customer_id must be a positive integer, received: abc" |
| "Failed" | "search_products requires at least one of: keyword, category, or price_range" |
| "Type error" | "quantity must be a number between 1 and 100, received: 150" |
Error Message Impact
| Error Quality | Retry Success Rate |
|---|---|
| Generic | 34% |
| Specific | 78% |
| With examples | 91% |
Pattern 4: Tool Selection Guidance
Help the model choose the right tool.
When Multiple Tools Could Apply
For overlapping functionality, add selection hints:
| Scenario | Tool | When to Use |
|---|---|---|
| Find specific order | get_order | Have order_id |
| Find customer orders | get_customer_orders | Have customer_id, want all orders |
| Search orders | search_orders | Have search criteria, no IDs |
Disambiguation Examples
Add examples that show when to use each tool:
| User Query | Correct Tool | Why |
|---|---|---|
| "What is order 12345?" | get_order | Specific order ID |
| "Show my recent orders" | get_customer_orders | Customer context, no order ID |
| "Find orders over $100" | search_orders | Search criteria |
Pattern 5: Structured Output Enforcement
Use structured output modes when available:
JSON Mode Benefits
| Without JSON Mode | With JSON Mode |
|---|---|
| 82% valid JSON | 99.7% valid JSON |
| Manual parsing | Guaranteed structure |
| Format drift | Consistent format |
Constrained Output
When the model should only call tools (not respond directly):
Force tool use when appropriate. This prevents the model from making up answers when it should use tools.
Pattern 6: Multi-Step Tool Orchestration
Complex tasks require multiple tool calls.
Orchestration Patterns
| Pattern | Use Case | Example |
|---|---|---|
| Sequential | Dependent steps | Get customer, then get orders |
| Parallel | Independent steps | Get orders AND get preferences |
| Conditional | Based on result | If order exists, get details |
| Iterative | Repeat until done | Page through results |
Sequential Execution
When steps depend on each other:
| Step | Tool | Depends On |
|---|---|---|
| 1 | get_customer | None |
| 2 | get_customer_orders | Step 1 result |
| 3 | calculate_total | Step 2 result |
Parallel Execution
When steps are independent, run them together to reduce latency:
| Sequential | Parallel |
|---|---|
| 3.2s total | 1.1s total |
Pattern 7: Graceful Degradation
When tools fail, handle gracefully.
Fallback Strategy
| Primary Fails | Fallback Action |
|---|---|
| Database timeout | Use cached data |
| API rate limited | Queue and retry |
| Service down | Explain limitation |
| Invalid data | Ask for clarification |
User Communication
When tools cannot complete:
| Bad | Good |
|---|---|
| "Error occurred" | "I could not retrieve your orders because the order system is currently slow. Would you like me to try again in a moment?" |
Pattern 8: Observability
Track everything about tool usage.
Metrics to Track
| Metric | Purpose | Alert Threshold |
|---|---|---|
| Call success rate | Overall health | Less than 95% |
| Avg latency | Performance | More than 2s |
| Error by type | Debug issues | Any spike |
| Retry rate | Model struggling | More than 10% |
| Tool usage distribution | Understand patterns | Anomalies |
Logging Tool Calls
Every tool call should log:
| Field | Purpose |
|---|---|
| request_id | Correlation |
| tool_name | What was called |
| arguments | What was passed |
| validation_result | Did it validate |
| execution_time | How long |
| result_summary | What happened |
| error | If failed, why |
Production Results
After implementing all patterns:
| Metric | Before | After |
|---|---|---|
| Tool call accuracy | 71% | 98% |
| First-attempt success | 58% | 89% |
| Avg retries needed | 1.8 | 0.3 |
| User task completion | 64% | 94% |
| Support escalations | 23% | 4% |
Key Takeaways
- 1Schemas are documentation for models - Detailed descriptions and examples dramatically improve accuracy.
- 2Validate everything - Never trust model output. Schema validation catches most errors.
- 3Helpful errors enable recovery - Good error messages let the model self-correct.
- 4Guide tool selection - When tools overlap, add explicit guidance on when to use each.
- 5Structured output is worth it - JSON mode and constrained generation eliminate entire error classes.
- 6Observe and iterate - Track tool usage patterns to find and fix systematic issues.
Function calling turns LLMs from chat interfaces into capable assistants. But reliability requires deliberate engineering - the patterns above are how we get there.