Tool calling

Tool calling lets the LLM driving your voice session take action: query a database, schedule a visit, transfer to a human. The model decides when to invoke a tool from your prompt; Speko POSTs a Standard Webhooks-signed request to your endpoint, folds the JSON response back into the model's next turn, and the agent verbalizes the result.

This guide walks through registering a tool, hooking it into a Speko voice session, and confirming it fires.

Architecture

Voice session                 Speko proxy                  Your endpoint
─────────────                 ───────────                  ─────────────
LLM emits tool call    ─→   /v1/complete loop      ─→     POST /your/webhook
                                                          (signed body)
LLM verbalizes result  ←─   response folded back   ←─     200 + JSON

Three pieces meet:

1. Your endpoint — a public HTTPS URL that receives the tool call and returns JSON.
2. The Speko dashboard — where you register the tool (name, description, JSON Schema parameters, your endpoint URL). Speko stores an HMAC signing secret you save once.
3. A Speko voice session — the worker fetches your registered tools at session start, exposes them to the LLM, and routes invocations through the executor.

1. Build your endpoint

The executor POSTs the LLM-generated arguments as JSON. Whatever you return becomes the model's next observation, so keep responses small and specific.

import { Hono } from 'hono';

const PETS: Record<string, unknown> = {
  luna: { name: 'Luna', species: 'corgi', age: 3, status: 'available' },
  max: { name: 'Max', species: 'tabby cat', age: 5, status: 'available' },
};

const app = new Hono();

app.post('/lookup', async (c) => {
  const { name } = (await c.req.json()) as { name?: string };
  const pet =
    PETS[
      String(name ?? '')
        .toLowerCase()
        .trim()
    ];
  if (!pet) return c.json({ error: 'Pet not found' }, 404);
  return c.json(pet);
});

export default { port: Number(process.env.PORT ?? 8080), fetch: app.fetch };

Deploy this anywhere with a public HTTPS URL — Cloud Run, Fly.io, Render, Vercel functions.

Verifying the signature

Production endpoints MUST verify the Standard Webhooks signature on every request. Speko sends three headers:

Use the standardwebhooks package — constant-time comparison and clock-skew tolerance are tricky to roll yourself.

import { Webhook } from 'standardwebhooks';

const wh = new Webhook(process.env.LOOKUP_PET_SIGNING_SECRET!);
app.post('/lookup', async (c) => {
  const raw = await c.req.text();
  try {
    wh.verify(raw, Object.fromEntries(c.req.raw.headers));
  } catch {
    return c.text('signature mismatch', 401);
  }
  const { name } = JSON.parse(raw) as { name?: string };
  // …
});

2. Register the tool

Via the dashboard

Open Tools in the dashboard, click Add tool, fill in:

• Name — snake_case, ≤ 64 chars (e.g. lookup_pet). The model sees this; pick something it'll match against the user's intent.
• Description — tell the model when to call this. Be explicit ("ALWAYS call this when the user asks about a specific pet by name").
• Parameters — a JSON Schema. Strict typing works; vague typing leads to the model passing garbage args.
• Webhook URL — your public HTTPS endpoint from step 1. Speko rejects HTTP, private/loopback hosts, and known cloud-metadata IPs at registration time.

On save, the dashboard shows the signing secret once. Copy it into your secrets manager — rotation requires creating a new tool (UUID-stable secret keys are coming).

Via the API

curl -X POST "https://api.speko.dev/v1/agents/$SPEKO_AGENT_ID/tools" \
  -H "Authorization: Bearer $SPEKO_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "lookup_pet",
    "description": "Look up a pet by name. ALWAYS call this when the user asks about a specific pet.",
    "parameters": {
      "type": "object",
      "required": ["name"],
      "properties": {
        "name": { "type": "string", "description": "First name of the pet." }
      }
    },
    "source": {
      "kind": "webhook",
      "url": "https://your-endpoint.example.com/lookup",
      "secret": "<32-char hex you supply — Speko stores it encrypted>"
    }
  }'

The secret you POST is what Speko uses to sign webhook deliveries. The server stores an encrypted copy and never echoes it back, so keep your local copy.

Async webhook tools

Webhook tools default to responseMode: "sync": Speko waits for your endpoint response and feeds the JSON body into the next model turn. For work that should not block the conversation, set responseMode: "async" and provide an asyncAck:

{
  "source": {
    "kind": "webhook",
    "url": "https://your-endpoint.example.com/create-ticket",
    "secret": "<32-char hex you supply>",
    "responseMode": "async",
    "asyncAck": "I started that request and will continue helping while it runs."
  }
}

In async mode, Speko dispatches the signed webhook in the background and immediately returns the acknowledgement text to the model. Use this for ticket creation, CRM updates, notifications, and other side effects where the caller does not need the result before the next assistant turn.

Use the actual agent id

Tools are scoped to one persisted agent. Use the agent id returned by POST /v1/agents or shown on the dashboard agent page. The unique key is (organization, agentId, toolName), so two agents can use the same tool name without sharing webhook config.

3. Wire the worker

If you run a LiveKit Agents worker, the adapter loads your registered tools at session start and merges them with anything the framework provides at runtime. Use createSpekoComponents with the registered-tools options:

import { defineAgent, voice } from '@livekit/agents';
import * as silero from '@livekit/agents-plugin-silero';
import { Speko } from '@spekoai/sdk';
import { createSpekoComponents } from '@spekoai/adapter-livekit';

const speko = new Speko({
  apiKey: process.env.SPEKO_API_KEY!,
  baseUrl: process.env.SPEKO_BASE_URL,
});

export default defineAgent({
  prewarm: async (proc) => {
    proc.userData.vad = await silero.VAD.load();
  },
  entry: async (ctx) => {
    const vad = ctx.proc.userData.vad as silero.VAD;

    const { stt, llm, tts } = createSpekoComponents({
      speko,
      vad,
      intent: { language: 'en-US', optimizeFor: 'latency' },
      // Enable the registered-tools loader. The adapter calls
      // speko.agents.tools.listChatTools(agentId) once per session — reusing
      // the Speko client above for auth and base URL — and merges the result
      // with whatever LiveKit's ToolContext provides. Registered tools win on
      // name collision.
      agentId: process.env.SPEKO_AGENT_ID!,
      onRegisteredToolsError: (err) =>
        console.error('SpekoWorker: tools fetch failed', err),
    });

    const session = new voice.AgentSession({ vad, stt, llm, tts });
    await session.start({
      agent: new voice.Agent({
        instructions:
          'You are a brief, friendly assistant. ' +
          'When the user asks about a specific pet by name, ' +
          'IMMEDIATELY call lookup_pet — never make up information.',
      }),
      room: ctx.room,
    });
    await ctx.connect();
  },
});

Without agentId, the loader stays disabled and the agent only sees runtime tools — useful when you want to opt in selectively.

Outside a LiveKit worker, load the same tools yourself with speko.agents.tools.listChatTools(agentId) and pass them to speko.complete({ tools }). It returns every source kind (inline, webhook, builtin, integration) already in the ChatTool[] shape /v1/complete accepts.

4. Run a call

The simplest client is a browser using @spekoai/client:

import { VoiceConversation } from '@spekoai/client';

const res = await fetch('/api/session', { method: 'POST' });
const { transportToken, transportUrl } = await res.json();

const conv = await VoiceConversation.create({
  transportToken,
  transportUrl,
  onModeChange: (mode) => console.log(mode), // 'listening' | 'speaking'
});

Your /api/session server route mints browser-safe transport credentials via Speko:

const r = await fetch(process.env.SPEKO_BASE_URL + '/v1/sessions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: 'Bearer ' + process.env.SPEKO_API_KEY,
  },
    body: JSON.stringify({
      mode: 'cascade',
      agentId: process.env.SPEKO_AGENT_ID!,
      ttlSeconds: 900,
    }),
});
const { transportToken, transportUrl } = await r.json();

What gets sent over the wire

When the model invokes a registered tool, Speko's executor signs the request with your secret and POSTs to your URL:

POST https://your-endpoint.example.com/lookup
content-type: application/json
webhook-id: msg_2KQfP3QH8Gv7B
webhook-timestamp: 1735603214
webhook-signature: v1,F7ZxQk8j3p6m2N9...

{
  "name": "Luna"
}

Your response body is what the model sees as the tool result. Errors propagate too — if your endpoint returns 4xx/5xx, the executor surfaces the error so the agent can apologize or retry instead of silently swallowing it.

Debugging

Common failure modes:

• Tool never invoked. The model didn't decide to call it. Tighten the description (be explicit about when to call), or set toolChoice: "required" in your call options to force one.
• Webhook never lands. Check the worker logs for the executor span. Common: 403 from your endpoint (signature mismatch), 5xx (your code threw), or timeout (your endpoint is too slow — budget under 4 seconds).
• Agent says "couldn't find" instead of the real result. Your endpoint returned 4xx. Either the query genuinely missed, or the model passed empty/wrong args. During development, have your endpoint echo back the body it received so you can spot the latter.
• Two voices overlap in the room. A second agent dispatched into the same room without ending the previous session. Always call endSession() on your VoiceConversation (or disconnect the participant) before opening a new conversation.

Beyond webhooks

Webhook tools are the most common, but a registered tool's source can also be:

• builtin — Speko-managed helpers you opt into without running your own endpoint. Current built-ins include search_knowledge_base, transfer_call, and end_call. transfer_call supports warm or blind transfers from the active phone session when configured with destinations.
• integration — an action from an org-installed Speko app (Google Calendar, Slack, …), resolved and executed server-side.
• inline — your own worker runs the tool; Speko just ships the schema to the model and returns the call to you.

All four kinds come back from speko.agents.tools.listChatTools(agentId) ready to hand to speko.complete.

What's next

• Streaming tool results for long-running queries.

Track progress on the public roadmap.