AI efficiency, measured and billed on results

Your GPUs are busy.
That isn’t the same as useful.

Tenosra profiles where your AI workload actually spends its time — from kernel stalls and memory-bound decode at the bottom, up to the agent loop that calls the same tool four times and re-sends the whole context each turn. Then it lands the fixes on the stack you already run. No migration. No model switch. No rewrite. We bill a share of the verified saving, measured on your own telemetry.

Book a 30-minute auditWatch it cut a prompt in half

Downside is zero. If we find nothing, you owe nothing.

−48%
input tokens on RAGBench, at a 50% keep-ratio
1.99–2.7×
faster time to first token, measured
93 → 93%
answer accuracy held (McNemar p = 1.0)
$0 upfront
we take a share of what we verifiably save you
kiki-compressor · shipped

Half of what you send the GPU never earns its place.

Below is a real retrieved-context payload and a real question about it. A cross-encoder scores every sentence against that question, keeps the ones carrying the answer, and drops the rest. Extractive, never generative — every sentence that survives is verbatim. The model sees less and answers the same.

kiki-compressor · offline sample · discard 65%query: “What is the p99 latency SLO for the inference API, and…”
Full prompt555 tok

The Platform Reliability Handbook covers service level objectives for all customer-facing systems. It is reviewed quarterly by the reliability guild and the on-call leads. Revision 14 was ratified in March and supersedes all previous editions. Teams are expected to read the handbook during onboarding. The inference API is the highest-traffic service in the estate, serving roughly 40,000 requests per minute at peak. Its p99 latency SLO is 800ms measured at the edge, excluding client network time. Breach of the p99 SLO for more than 15 consecutive minutes triggers a Sev-2 incident and pages the on-call engineer immediately. Sustained breach across a rolling 7-day window burns the error budget, and once the budget is exhausted all feature deploys to the service are frozen until the budget recovers. The error budget policy is deliberately strict. Freezes are lifted by the reliability guild, not by the owning team, and the guild meets twice weekly. Historically the freeze has been invoked four times, most recently after the batching regression in the autumn. Our office locations include San Francisco, Berlin, and Bangalore. The Bangalore office moved to a new building last year and now has a much better cafeteria. Desk booking is handled through the internal facilities portal, which requires SSO. Parking is limited and allocated by lottery each quarter. The billing subsystem reconciles usage nightly at 02:00 UTC. Invoices are generated on the first business day of the month. Disputes must be raised within 30 days. The finance team owns this pipeline and it is not part of the inference estate. Capacity planning for the inference fleet is done monthly. The fleet currently runs on A100 and H100 nodes across two regions. GPU utilisation averages 61% during business hours. The team has an open project to improve batching efficiency, which is tracked in the platform roadmap. Employee expense reports should be filed through the expenses tool within 60 days of the expense being incurred. Receipts are required for anything above 25 dollars. Managers approve expenses weekly.

Sent to the GPU555 tok

waiting…

cut 0 of 26 sentencesyour question always passes throughtokens 555 → 251input −55%

The live engine is unreachable right now, so this panel is running a bundled offline stand-in — not the cross-encoder. Set KIKI_BASE_URL to point at your kiki-compressor host and it goes live.

Two products

One is running today. One pays for itself or it costs you nothing.

Shipped

kiki-compressor

Context compression proxy

An OpenAI-compatible proxy that compresses context before it reaches your inference server. Point your SDK at it and input tokens fall by roughly half. Responses stream back untouched.

  • Cross-encoder reranks every sentence against the live query
  • Extractive — kept text is verbatim, never paraphrased
  • Never breaks a request: final turn and tool calls are protected
  • Stateful mode stores the full history so clients send only new turns
# point it at your inference server
UPSTREAM_BASE_URL=http://localhost:8000/v1 \
  kiki-compressor-proxy
See it run
EnterprisePaid on results

Headroom

AI efficiency audit

We attach a profiler to your live stack and rank where the time is really going — down in the kernels and up in the agent loop. Then we land the fixes it names, on the stack you already run. The headroom is already in the boxes you rent; Headroom finds it and takes it.

  • Part one — the profiler. One dashboard over both levels. Up top, what your agents spend time and tokens on: redundant tool calls, context re-sent every turn, serial steps, retry loops. Underneath, prefill, batch formation, kernel stalls, memory-bound decode, cache misses. No more billing surprises at the end of the month.
  • Part two — the fixes. Tool-call dedup, stateful context, step parallelism, retry budgets, context compression, KV cache reuse, batching policy, fused kernels, phase-aware clocking. No migration, no model switch, no rewrite.
  • The bill. A share of the verified saving, measured on your telemetry. Find nothing, charge nothing.
Book a 30-minute auditWhat it measures
Headroom · the profiler

Nobody budgets for the agent that retried forty times.

Agent bills arrive as one number at the end of the month, and by then the money is gone. The profiler tracks every run and ranks what your agents actually spend their time and tokens on — redundant tool calls, the context re-sent on every single turn, steps queued in series that had no reason to be, retry loops nobody capped. Then it names the fix for each one. Faster, cheaper, and more reliable are usually the same fix.

Underneath it, the same dashboard shows where the GPU itself is burning the time the agent asked for. One view, both levels — because “GPU utilisation: 61%” is not a diagnosis, it is a shrug.

One dashboard. Both levels.
Illustrative profiles — a 7B-class chat workload on A100s and a tool-using agent. This is the shape the profiler produces, not a customer result. Yours will look different, and that is the entire point of measuring it.

The agent loopWhere an agent run burns wall-clock and billed tokens

Context re-sent per turn
34%
Redundant tool calls
26%
Serial steps that could run in parallel
22%
Retry loops
18%

The GPU underneathWhere the box spends the time the agent asked for

Memory-bound decode
34%
Prefill
24%
Batch formation
17%
Kernel stalls
15%
Cache misses
10%
Headroom · the fixes

Named by the profiler. Landed on the stack you already run.

None of these require you to change model, change vendor, or rewrite a serving path. That constraint is the product. A fix you cannot deploy is not a fix.

Context compression

Shipped
↳ Prefill

Rank every sentence against the live query with a cross-encoder, keep the ones that carry the answer, drop the rest. Extractive — kept text is verbatim, never paraphrased.

Stateful context

Shipped
↳ Context re-sent per turn

Hold the conversation server-side so the agent sends only the new turn instead of re-uploading its entire transcript every step. Already shipped in the compression proxy.

Tool-call dedup

Audit
↳ Redundant tool calls

Memoise tool results within a run. The same call with the same arguments returns from cache instead of paying for a second round trip and a second set of tokens.

Step parallelism

Audit
↳ Serial steps that could run in parallel

Find the steps in an agent's plan that have no dependency on each other and stop making them queue. Wall-clock drops from the sum to the max.

Retry budgets

Audit
↳ Retry loops

Cap what a failing step is allowed to spend before it gives up, and surface which step is actually flaky — so the retry stops being an invisible line on the invoice.

KV cache reuse

Audit
↳ Cache misses · Prefill

Stop recomputing attention over prefixes the box has already seen. Shared system prompts and long documents are re-prefilled far more often than teams think.

Batching policy

Audit
↳ Batch formation

Requests wait for a batch that is already full, or ship in a batch that is half empty. The policy is usually a default nobody has revisited since the stack was stood up.

Fused kernels

Audit
↳ Kernel stalls · Memory-bound decode

Decode is memory-bound, not compute-bound. Fusing the pointwise work collapses round trips to HBM that the GPU is otherwise sitting idle waiting on.

Phase-aware clocking

Audit
↳ Memory-bound decode

Prefill and decode want different clocks. Running one clock for both burns power on the phase that cannot use it, and leaves headroom on the phase that can.

Proof

The numbers, including the ones that cost us.

Compression is not free — it costs 2.52s to run, and we count that against the gain rather than hiding it. Every figure below is a measured benchmark run, and we will walk you through the methodology on a call.

RAGBench

n = 150 · 50% keep-ratio
MetricBeforeAfter
Prompt tokens2,2341,152 (−48%)
Judged correct93%93% (Δ −0.007)
Evidence retained—69% (of relevant sentences)

Accuracy difference is statistically indistinguishable from zero: 95% CI −0.047 … +0.040, McNemar p = 1.0.

Code bug localization

n = 100 · 20% keep-ratio
MetricBeforeAfter
Accuracy46%61% (+15pp)
Time to first token4.40s1.66s (2.7× faster)

Dropping irrelevant code raised accuracy — less haystack, easier needle.

Latency

uncontended GPU, ~5k tokens
MetricBeforeAfter
Time to first token15.39s7.74s (1.99× faster)
Compression cost—2.52s (net −5.13s)

Compression is not free. It is counted against the gain, not hidden from it.

How this works

We only get paid out of money you were already burning.

01

We profile

The profiler attaches to your live inference stack and ranks where GPU time goes on your real traffic. You keep the report either way.

02

We land the fixes

On your stack, your model, your serving path. Nothing gets migrated and nothing gets rewritten.

03

We measure the saving

On your telemetry, not ours. Before and after, on the same traffic. If the number isn’t real, it doesn’t count.

04

We take a share

A cut of the verified saving. No retainer, no seat licence, no upfront. If we find nothing, you owe nothing.

Find out what your AI stack is actually doing.

Thirty minutes. We’ll walk your inference stack and your agent traces, tell you which phase is eating your time, and show you what the first fix is worth. If there’s nothing there, we’ll say so.

Book a 30-minute audit