Send traffic to Searchable via the REST API

Send AI-bot traffic events to Searchable from any language or platform — a single HTTP POST with a signed Bearer token.

What this is

Searchable’s REST API is the language-agnostic way to ship request events from your app. It’s a single POST to a Cloudflare-hosted ingest endpoint with an Authorization: Bearer sk_live_… header and a JSON body of one or more events. The server-side AI-bot classifier filters non-AI user agents, so even if you POST every request from your app, only crawlers like GPTBot, ClaudeBot, and PerplexityBot end up in your dashboard.

Use this when the Middleware SDK doesn’t fit — non-Node stacks, in-house CDN workers, batch jobs that replay logs, or anywhere you want fine-grained control over the payload.

Prerequisites

The two credentials from the common prerequisites: a project site token (st_…) and a workspace API key (sk_live_…)

Any runtime that can make an authenticated HTTPS POST — curl, fetch, requests, Go’s net/http, etc.

Endpoint

POST https://searchable-tracker.searchable.workers.dev/v1/events

Requests are authenticated, verified, and forwarded at Cloudflare’s edge — there’s no DB round-trip on the auth path, so you can call this from latency-sensitive contexts.

Header	Value
Authorization	Bearer sk_live_… — the workspace API key
Content-Type	application/json

Request body

{
  "site_token": "st_your_token_here",
  "events": [
    {
      "event_name": "server_request",
      "timestamp": 1716105600000,
      "method": "GET",
      "path": "/blog/my-post",
      "url": "https://example.com/blog/my-post",
      "status_code": 200,
      "response_time_ms": 42,
      "user_agent": "GPTBot/1.0",
      "referrer": "",
      "ip_address": "203.0.113.42",
      "country": "US"
    }
  ]
}

Top-level fields

Field	Type	Required	Description
site_token	string	yes	The project’s private site token (starts with st_)
events	array	yes	One or more event objects. Batch up to 1MB of payload per request.

Event fields

Field	Type	Required	Description
event_name	string	yes	Use "server_request" for HTTP request events
timestamp	number	yes	Unix timestamp in milliseconds
method	string	yes	HTTP method (GET, POST, …)
path	string	yes	Request path. Query strings are stripped server-side.
url	string	yes	Full request URL
status_code	number	yes	HTTP response status
response_time_ms	number	yes	Server response time in milliseconds
user_agent	string	—	User-Agent header. Empty string if unavailable.
ip_address	string	—	Client IP. Defaults to 0.0.0.0 if omitted. We recommend anonymizing the last octet client-side.
referrer	string	—	HTTP Referer header
referrer_domain	string	—	Pre-parsed referrer hostname (we’ll derive it from referrer if omitted)
country	string	—	2-letter ISO country code (e.g. US). Geo enrichment otherwise comes from Cloudflare’s edge metadata.
region	string	—	Region / state code
city	string	—	City name
utm_source, utm_medium, utm_campaign, utm_term, utm_content	string	—	Standard UTM fields
headers	object	—	Map of safe request headers you want to retain
query_parameters	object	—	Non-UTM query parameters as a string map
custom_properties	object	—	Arbitrary string-keyed properties exposed as parameters in the dashboard

Unknown fields are ignored. Fields with bad types are coerced where safe (counts → unsigned int, status codes → uint).

Response

Status	Meaning
202 Accepted	Events were accepted and forwarded to ingest. The body is empty.
400 Bad Request	Body wasn’t valid JSON, or events wasn’t an array.
401 Unauthorized	Missing Authorization header.
403 Forbidden	Token signature invalid, or site_token missing from body.
413 Payload Too Large	Request body exceeded 1 MB. Split your batch.

202 is the success status — events are accepted and dispatched asynchronously. There is no synchronous confirmation that an event reached ClickHouse; use the LLM Analytics → Setup status strip to verify end-to-end flow.

Quick start — curl

curl -X POST https://searchable-tracker.searchable.workers.dev/v1/events \
  -H "Authorization: Bearer sk_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "site_token": "st_YOUR_SITE_TOKEN",
    "events": [{
      "event_name": "server_request",
      "timestamp": '"$(date +%s%3N)"',
      "method": "GET",
      "path": "/blog/my-post",
      "url": "https://example.com/blog/my-post",
      "status_code": 200,
      "response_time_ms": 42,
      "user_agent": "GPTBot/1.0"
    }]
  }'

A successful call returns an empty body and 202 Accepted. Within a few seconds, the Custom card in LLM Analytics → Setup flips to Connected.

Examples

Node — fetch

async function reportRequest(req: {
  method: string;
  path: string;
  url: string;
  status: number;
  durationMs: number;
  userAgent: string;
}) {
  await fetch("https://searchable-tracker.searchable.workers.dev/v1/events", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SEARCHABLE_API_KEY!}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      site_token: process.env.SEARCHABLE_SITE_TOKEN!,
      events: [
        {
          event_name: "server_request",
          timestamp: Date.now(),
          method: req.method,
          path: req.path,
          url: req.url,
          status_code: req.status,
          response_time_ms: req.durationMs,
          user_agent: req.userAgent,
        },
      ],
    }),
  }).catch(() => {
    // Fire-and-forget. Never fail the user's request on a Searchable error.
  });
}

Don’t await this from a request handler in latency-sensitive paths — either fire it after the response is flushed, or push it onto a worker queue.

Python — requests

import os
import time
import requests

def report_request(method, path, url, status, duration_ms, user_agent):
    payload = {
        "site_token": os.environ["SEARCHABLE_SITE_TOKEN"],
        "events": [{
            "event_name": "server_request",
            "timestamp": int(time.time() * 1000),
            "method": method,
            "path": path,
            "url": url,
            "status_code": status,
            "response_time_ms": duration_ms,
            "user_agent": user_agent,
        }],
    }
    try:
        requests.post(
            "https://searchable-tracker.searchable.workers.dev/v1/events",
            json=payload,
            headers={"Authorization": f"Bearer {os.environ['SEARCHABLE_API_KEY']}"},
            timeout=2,
        )
    except requests.RequestException:
        pass  # fire-and-forget

Go — net/http

package searchable

import (
	"bytes"
	"context"
	"encoding/json"
	"net/http"
	"os"
	"time"
)

type Event struct {
	EventName      string `json:"event_name"`
	Timestamp      int64  `json:"timestamp"`
	Method         string `json:"method"`
	Path           string `json:"path"`
	URL            string `json:"url"`
	StatusCode     int    `json:"status_code"`
	ResponseTimeMs int    `json:"response_time_ms"`
	UserAgent      string `json:"user_agent"`
}

type Payload struct {
	SiteToken string  `json:"site_token"`
	Events    []Event `json:"events"`
}

func Report(ctx context.Context, e Event) error {
	body, _ := json.Marshal(Payload{
		SiteToken: os.Getenv("SEARCHABLE_SITE_TOKEN"),
		Events:    []Event{e},
	})
	req, _ := http.NewRequestWithContext(ctx, "POST",
		"https://searchable-tracker.searchable.workers.dev/v1/events",
		bytes.NewReader(body))
	req.Header.Set("Authorization", "Bearer "+os.Getenv("SEARCHABLE_API_KEY"))
	req.Header.Set("Content-Type", "application/json")

	client := &http.Client{Timeout: 2 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return err
	}
	resp.Body.Close()
	return nil
}

Batching

The endpoint accepts up to 1 MB per request and any number of events in the events array. For high-volume sources, batch events on a short flush interval (e.g. every 5 s or every 100 events) rather than sending one POST per request — fewer network round-trips, same data.

Each event in a batch is independent; ingest validates and persists them individually, so a single malformed event doesn’t drop the rest of the batch.

What gets recorded

Even though you can post any HTTP request through this endpoint, Searchable’s server-side classifier only records events whose user_agent matches a known AI crawler. Everything else is dropped silently.

That’s intentional: it lets you instrument your app once and not worry about which UAs to filter in your code. The bot list is refreshed daily — new AI agents are picked up automatically.

You can sanity-check the classifier against the public bot artifact at:

GET https://searchable-tracker.searchable.workers.dev/v1/bots.json

That’s the same list the worker uses internally — useful if you want to do client-side filtering to reduce ingest load.

Verifying the connection

In Searchable:

1. Go to LLM Analytics → Setup
2. Hit the endpoint with a known AI user agent to force a first event (see the curl example)
3. Click Refresh in the status strip

Status	What it means
Waiting for first event	API key + body are valid but no event has matched the bot classifier yet.
Connected	Events are arriving. The strip shows the count from the last 24 hours.

Troubleshooting

401 Unauthorized

The Authorization header is missing.

• Check the header is named exactly Authorization (not authorisation, X-Authorization, etc.)
• The value must be Bearer followed by your sk_live_… key, with one space and no quotes
• Some CDNs strip Authorization on internal hops — verify the header is present when the request leaves your edge

403 Forbidden — `invalid_signature`

The API key’s signature failed verification.

• Confirm you copied the key in full — it’s two URL-safe base64 segments separated by a .
• If you’ve recently revoked the key in Searchable, generate a new one (Settings → API Keys → New key)
• Make sure you’re using a key with the Log Events permission — generating from the Custom connector dialog assigns it by default

403 Forbidden — `missing_site_token`

The body must include site_token at the top level. Don’t put it inside an event object.

{ "site_token": "st_…", "events": [ /* … */ ] }

400 Bad Request — `invalid_events_array`

events must be an array, even for a single event. Wrap your event in [ … ]:

{ "site_token": "st_…", "events": [ { /* event */ } ] }

413 Payload Too Large

Request body exceeded 1 MB. Either split into multiple POSTs, or trim large fields (headers, query parameters, custom properties) from each event.

Events return 202 but never appear in the dashboard

The classifier is dropping them because the user_agent isn’t a known AI crawler. Use an AI UA in your test:

curl -H "User-Agent: GPTBot/1.0 (+https://openai.com/gptbot)" ...

Or fetch the live AI-bot list and confirm your UA matches one of the patterns:

curl https://searchable-tracker.searchable.workers.dev/v1/bots.json

Removing the integration

1. Stop sending POSTs from your app
2. In Searchable → Settings → API Keys → revoke the API key

Revoking the key is the cleanest stop — every subsequent POST returns 403, regardless of where it’s coming from.

Next steps

Middleware SDK

On Node? The SDK is one import and handles the payload for you.

See the data

Open LLM Analytics to see which assistants are crawling your site.