Shipping Barklog: Part 2

It’s late, you’re on your phone, a game goes past in a video, and you want to keep it. So you do the most ordinary thing a phone can do: hit share, pick an app. Nobody thinks about that gesture while making it. It’s two taps and a row of icons and it’s worked the same way for a decade.

What the receiving app gets is a string. Usually something like https://www.youtube.com/watch?v=1vs0lLIRt7w: eleven characters of opaque identifier attached to a hostname. The game you meant isn’t in there. The title isn’t in there. Nothing about the thing you wanted to remember survived the tap.

Part 1 was the architecture an agent-assisted build pushed me into. This one is a single feature inside it, and it’s the feature where all the interesting work is recovery: turning that string back into a specific row in a games database, without handing a stranger a server that fetches whatever URL they type.

What a share sheet actually hands you

Barklog takes shares through expo-sharing, first-party as of Expo SDK 57. Its config plugin generates the iOS Share Extension, the App Group and the activation rules, so there’s no hand-written Swift in the repository.

Expo marks iOS share-receiving experimental, and the reason matters more than the label. The generated extension doesn’t process the payload in its own ViewController, which is how a share extension is normally built and what Apple officially supports. It writes the payload into the App Group and opens the main app target. There’s no small sheet that appears over Instagram, does its work and vanishes. The share journey happens inside Barklog, with Barklog fully launched.

You design for that rather than hide it. Expo Router’s redirectSystemPath returns a path to navigate to and there’s no return value meaning “stay where you are”, so an incoming share has to land on a route. Barklog’s root layout was deliberately a Slot with (tabs) as the only root route, so a sibling /shared would have replaced the tabs instead of presenting over them, unmounting tab state with nowhere to go back to. The root became a Stack, and /shared is a fullScreenModal on top of it.

Two details in there are easy to get wrong. The payload outlives the process, so a share the user already dealt with gets re-presented on the next cold launch unless you clear it. Every path that actually leaves the flow calls clearSharedPayloads(). Picking a candidate doesn’t, because that pushes deeper in rather than out, and a “wrong pick, go back” should find the share still sitting there.

The other is knowing when nothing’s coming. The hook exposes isResolving, and emptiness can’t stand in for it, because a resolve that finished and produced nothing looks exactly like one that hasn’t started yet. Read it as the second and the screen waits forever.

Two spellings of the same video

Before anything touches the network the URL gets canonicalised. It’s unglamorous, and it’s the cheapest leverage in the pipeline.

normalise.ts drops the fragment, strips utm_* and a short list of tracking parameters, sorts what’s left so two spellings differing only in query order collide, and rebuilds YouTube and TikTok URLs from their native ids. Every youtu.be/x, youtube.com/shorts/x and m.youtube.com/watch?v=x becomes one string, whose SHA-1 is the shareId. That’s the cache key for both the metadata and the extraction, so canonicalisation decides how often the expensive parts run at all.

The interesting part is what deliberately isn’t stripped. t, ref and si are all common tracking parameters and none of them is on the list:

const TRACKING = new Set([
  "fbclid",
  "gclid",
  "igsh",
  "mc_cid",
  "mc_eid",
  "is_from_webapp",
  "sender_device",
]);

All three are short and generic enough that some arbitrary page could be using one to select content rather than to track a referrer. ?si=42 might be a size. Collapsing two genuinely different pages onto one shareId serves one page’s cached answer for the other, which is worse than a duplicate entry and shows up once in a thousand shares with no way to reproduce it. YouTube and TikTok get rebuilt from their ids anyway, so the short list costs nothing on the traffic that matters.

One host skips the pipeline entirely. An igdb.com/games/<slug> link identifies a row in the mirror exactly, since IGDB is where the mirror’s data came from. No fetch, no model call, no cache entry, just a slug lookup against a table Barklog already owns.

The ladder

Everything else needs metadata, and there are three places to get it, tried in order.

The rungs fail differently, and flattening that is how you ship something subtly terrible. An oEmbed endpoint answering 401, 403 or 404 is terminal: the video is private or removed, and falling through to scrape the page would faithfully report “Video unavailable” as the title and then hunt for a game by that name. A 5xx, a timeout, or a payload that doesn’t match the schema isn’t terminal and should fall through. TikTok answers 200 with a blank title for a removed video, which is why the schema puts a minLength(1) after trim.

Short links need no dedicated step. safeFetch reports the URL it ended on, so a vm.tiktok.com link resolves through the HTML fetch it was making anyway, and re-running the provider match on the final URL routes it to TikTok’s oEmbed endpoint.

Open Graph is parsed with htmlparser2 driven as a streaming SAX parser, no DOM built, calling parser.reset() on </head>. A regex over <meta> tags gets attribute order and quoting wrong in ways that matter when your input is every website on earth.

Saying which rung answered

The model at the end of this produces a guess, and guesses have different qualities. A title that literally names a game is a different claim from “this channel mostly covers one game, so probably that”. The pipeline reports which one it made:

export const EXTRACTED_BASES = ["title", "author", "web", "none"] as const;
export type ShareBasis = (typeof EXTRACTED_BASES)[number] | "unavailable";

That value goes on the wire and drives the header above the results: “Matches for the title”, or “Games this channel usually covers”, or “Matches from a web search”. On unavailable, where extraction itself failed and the results are a raw title search, there’s no header and an orange notice instead, because a header would claim a match the server explicitly disclaimed.

That last case is the one I’d defend hardest. When extraction fails soft, guesses holds the source’s raw title rather than anything a model believed, so quoting it back as “we think this is about X” asserts a belief nothing in the system holds. The empty state gates on basis for that reason, not on guesses being non-empty. A guess labelled “based on the video title” is honest. The same guess unlabelled isn’t.

332 endpoints in a file

oEmbed has a registry at oembed.com/providers.json listing several hundred sites that’ll hand you a title and an author for free, no key and no quota. Barklog doesn’t fetch it. A script does, and writes providers.generated.ts: 57KB of vendored data committed to the repository.

Fetching it at runtime would put a third-party JSON file on the request path, which is both a cold-start dependency and a place to be compromised from. That endpoint list is effectively the outbound allowlist for the oEmbed rung, and I’d rather it change in a reviewed diff than silently at 3 a.m.

Matching is where the trap is. Schemes are glob-ish, like https://*.youtube.com/watch*, and the obvious compilation turns each * into .*:

const body = anchored
  .split("*")
  .map((literal) => literal.replace(REGEXP_SPECIAL, "\\$&"))
  .join("[^/\\\\]*");

A wildcard that can match / lets https://evil.com/x.youtube.com/watch?v=1 satisfy YouTube’s scheme, because .* swallows evil.com/x and picks up the rest on the far side of the slash. Excluding / fixes that. Excluding \ too is the part I wouldn’t have thought of unaided: WHATWG URL treats a backslash as a path separator for https:, so https://evil.com\x.youtube.com/watch parses to host evil.com, and a class that only excluded / would let the wildcard cross that separator as well. There’s a test for that exact string.

“THIS CHANGES EVERYTHING”

The first version of this pipeline accepted YouTube and TikTok and refused everything else. The design document that replaced it says why that was wrong in one sentence I keep coming back to: the two-provider limit “is not a product decision that was made; it is the shape the first implementation happened to take”.

Two failures made it untenable. An IGN review or a Vimeo upload came back UNPROCESSABLE_SHARE, and the share sheet couldn’t know that before the round trip. And the model only ever saw a title and an author, which breaks on a very common case:

When a video is called “THIS CHANGES EVERYTHING” on a channel nobody has heard of, there is nothing in the input to work with, and the answer is none — even though the link itself, handed to a search engine, would settle it in one query.

So extraction became two passes against OpenAI’s Responses API, on gpt-5.4-mini by default.

Both passes share one instruction about what a title should look like, because the answer feeds a trigram search against the mirror and RE2 matches nothing there:

const CANON = `Reply with the game's canonical English title as a games
database would list it, not as the source spells it. Expand abbreviations
and nicknames: "RE2" is "Resident Evil 2", "BOTW" is "The Legend of
Zelda: Breath of the Wild", "GTA V" is "Grand Theft Auto V".

Give up to ${MAX_GUESSES} titles, most likely first. When a title could
mean an original or its remake, list both, original first. Never pad the
list to reach ${MAX_GUESSES}.`;

MAX_GUESSES is 3 because each guess costs one search against the mirror, so the model’s fan-out is bounded by what the next stage pays for it. The last sentence is there because a model asked for up to three will cheerfully find three.

Pass one sees the title, the author and the site, and is told to work down a ladder and report which rung it stopped on:

const PASS_1_SYSTEM = `You identify which video game a shared page or
video is about, from its title, its author and the site it came from.

${CANON}

Work down these tiers, stop at the first that applies, and set \`basis\`
to the tier you used:
- "title": the title names or clearly implies a game.
- "author": it does not, but you recognise the author or site, a channel,
  a creator, a publication, and it covers a game or a small set of games.
  Guess those.
- "none": neither. Return no titles.

Never invent an author or site you do not recognise.`;

That basis field is the part I would take to another project. I didn’t ask for a confidence score, because a number between zero and one out of a language model is a number I can’t act on. A tier is different. It says what kind of evidence produced the answer, and each tier maps to a sentence I’m willing to put in front of a user. “Matches for the title” is a different claim from “Games this channel usually covers”, and the screen renders whichever one the model earned.

Two calls, and where the trust ends

Each pass gets a strict JSON schema, and the two don’t share an enum:

export const PASS_1_BASES = ["title", "author", "none"] as const;
export const PASS_2_BASES = ["web", "none"] as const;

function schemaFor(bases: readonly string[]) {
  return {
    type: "json_schema" as const,
    name: "game_titles",
    strict: true,
    schema: {
      type: "object",
      properties: {
        titles: { type: "array", items: { type: "string" } },
        basis: { type: "string", enum: [...bases] },
      },
      required: ["titles", "basis"],
      additionalProperties: false,
    },
  };
}

Pass one can’t answer web and pass two can’t answer title, because neither value exists in the other’s enum. The provenance claim is enforced by the decoder rather than by the wording above it, and that distinction is the whole reason I trust the label enough to print it. A prompt is advice. A schema isn’t.

The two calls are tuned differently on purpose:

const first = await client.responses.create({
  model: options.model,
  instructions: PASS_1_SYSTEM,
  input: described,
  max_output_tokens: PASS_1_MAX_OUTPUT_TOKENS,
  reasoning: { effort: "none" },
  text: { format: schemaFor(PASS_1_BASES) },
});

const pass1 = parseExtraction(first.output_text, PASS_1_BASES);
if (pass1.basis !== "none") return pass1;

const second = await client.responses.create(
  {
    model: options.model,
    instructions: PASS_2_SYSTEM,
    input: `${described}\nURL: ${meta.pageUrl}`,
    max_output_tokens: PASS_2_MAX_OUTPUT_TOKENS,
    reasoning: { effort: "low" },
    tools: [{ type: "web_search", search_context_size: "low" }],
    text: { format: schemaFor(PASS_2_BASES) },
  },
  { maxRetries: 0 },
);

Pass one runs at reasoning effort none. Deciding whether a title names a game is recall, not thinking, and paying for reasoning tokens to answer it is paying for the wrong thing. Pass two gets low, because it has to read search results and weigh them.

The token budgets differ for a reason that cost me a debugging session. Pass one caps at 256, which is generous for three short strings. Pass two needs 1024, because reasoning items and web-search tool calls also land in the output array, and 256 doesn’t survive them. The failure mode when it doesn’t survive is not an error, it’s a truncated response that fails to parse, which is a much more annoying thing to chase.

maxRetries: 0 on the second call is the one setting I’d have got wrong without reading the billing page first. The SDK retries twice by default, and the client is already built with maxRetries: 1 and a 20-second timeout instead of the default ten minutes. On pass two even one retry is wrong, because the call that failed had already paid for a web search, and the retry buys a second one for the same answer.

Then the output stops being trusted:

export function parseExtraction(raw: string, allowed: readonly string[]): Extraction {
  const schema = v.object({
    titles: v.array(v.pipe(v.string(), v.trim())),
    basis: v.picklist(allowed),
  });

  const parsed = v.parse(schema, JSON.parse(raw));
  const titles = parsed.titles.filter((title) => title !== "").slice(0, MAX_GUESSES);

  if (parsed.basis === "none" || titles.length === 0) return { titles: [], basis: "none" };

  return { titles, basis: parsed.basis as Extraction["basis"] };
}

Structured output guarantees the shape, not the sense. A response can satisfy that schema perfectly and still claim basis: "title" with an empty array, or hand back three strings where one is "". The last branch collapses any of that to none, so the one thing the rest of the app relies on holds: a basis other than none always has at least one title behind it. Splitting this out from the API call also means the parsing rules are tested without a network or a key, which is most of what I wanted to be able to change quickly.

Everything here throws rather than degrades. A malformed response from pass two doesn’t fall back to pass one’s none, and the reason is the cache.

The gate, and what a cached mistake costs

The gate exists because of arithmetic. A search bills at $10 per 1000 calls plus roughly 8k input tokens, about $0.016 against $0.0002 for pass one, so roughly eighty times the price. Every YouTube link keeps the cost profile it already had, and only the links that actually stumped the cheap path pay for the expensive one.

An extraction is cached for thirty days, which is what makes a wrong answer expensive in a different currency:

export function extractKey(promptVersion: number, model: string, meta: SourceMeta): string {
  return `extract:v${promptVersion}:${model}:${meta.shareId}`;
}

Both the prompt version and the model are in the key. Either one changing changes the answer, and thirty days outlives several deploys, so an edit to PASS_1_SYSTEM that isn’t accompanied by a bump of EXTRACT_PROMPT_VERSION would keep serving answers from the prompt I just replaced. That constant sits directly above the prompts with a comment telling you to bump it, which is the least sophisticated possible solution and has not failed yet.

A throw propagates uncached. Returning pass one’s none after pass two broke would pin a dead end in Valkey for a month, for a link that might have resolved fine on the next attempt.

Pass one deliberately doesn’t see the URL. A slug is free signal, but a page with a usable slug has a title carrying the same words, and handing pass one the URL would blur the title tier into “title or URL”, which isn’t what the header promises.

The cache is doing product work

Caching a model call isn’t the same kind of optimisation as caching a database query, and the difference took me a while to see properly.

The obvious half is money and time. Pass two costs about eighty times pass one, and the person who shared the link is watching a spinner while it runs. A hit costs nothing and returns immediately.

The half that matters more is consistency. A model isn’t a function. Ask it the same question twice and you can get two different answers, both defensible. Share the same YouTube link into Barklog twice and get two different games back, and nothing about that reads as “the model is probabilistic” to the person holding the phone. It reads as a bug. The cache is what makes a repeated share idempotent from the outside, which is a product property rather than a performance one.

That is also why the canonicalisation two sections up is load-bearing rather than tidy. If the key were the raw URL, youtu.be/x and m.youtube.com/watch?v=x would miss each other, and the same video would get a second answer at a second price.

There are two caches here, with separate budgets:

export const SOURCE_TTL_SECONDS = 604_800;
export const EXTRACT_TTL_SECONDS = 2_592_000;

A week for the metadata, because a page’s title effectively never changes. Thirty days for the extraction, and that number is deliberately not derived from the one above it. The extraction key already names everything its answer depends on, so it can’t go stale underneath the metadata it was computed from.

That last bit is the part I’d carry to anything else built on a model. Caching a pure function is easy, because the key is the arguments. With a model call the arguments aren’t only what came in the request. They’re the prompt, the model, the tool configuration, the reasoning effort, everything you would have to change in source control to change the answer. Anything left out of the key is a cache quietly serving answers from code you already deleted.

The part most hackathon apps skip

Opening the endpoint to any link deleted a sentence I liked: “we only ever talk to four hostnames”. What replaces a host allowlist is the only genuinely dangerous thing in this project.

A server that fetches URLs strangers hand it is a server-side request forgery risk. Your API almost certainly sits inside a private network: cloud metadata at 169.254.169.254, a Postgres on 10.x, an admin panel bound to loopback, a Redis with no password because it’s “internal”. If somebody can make your server issue a GET, they reach all of that from inside your perimeter, and your app helpfully reads back whatever comes out of it as a page title.

The replacement rule is that Barklog only ever talks to public unicast addresses, over https, on port 443, following at most three redirects, reading at most 512KB, inside a total budget of 8 seconds.

The pinning is what matters most. Checking an address and then calling fetch(hostname) is the version everybody writes, and it doesn’t work: the connector re-resolves the hostname at connect time. An attacker running their own DNS server answers with a public address for your check and a private one a moment later for your connection. That’s DNS rebinding. The fix is to make the check binding rather than advisory, by connecting to the address you already verified:

function createPinnedDispatcher(address: string, family: 4 | 6): Agent {
  return new Agent({
    connect: {
      lookup: (_hostname, lookupOptions, callback) => {
        if (lookupOptions.all === true) {
          callback(null, [{ address, family }]);
        } else {
          callback(null, address, family);
        }
      },
    },
  });
}

Everything else follows from taking that seriously. Every address a resolver returns gets checked, not just the first, because a resolver answering with one public and one private address mustn’t be usable by picking the private one. Redirects are manual and every hop re-enters the whole check, since a redirect to an internal address is the cheapest bypass there is. Ports other than 443 are refused. That does refuse legitimate https://host:8443/ links, deliberately: arbitrary ports turn the endpoint into an internal port scanner.

The address policy is worth reading if you ever write one. IPv4 is the familiar table. IPv6 is where people get it wrong, and the trap that gets forgotten is that ::ffff:127.0.0.1 is loopback wearing an IPv6 hat, so mapped forms have to be unwrapped and re-checked against the IPv4 rules. Two carve-outs landed after review: 2001::/23, the IETF protocol assignments block where Teredo and benchmarking and AS112 live, none of them an ordinary destination; and 3ffe::/16, the former 6bone, returned to IANA and never reallocated, so a route to it means somebody’s routing reserved space internally.

The reviewer suggested driving that off the IANA special-purpose address registry instead. I didn’t, for a reason that generalises: the table churns as new blocks are delegated, and a stale copy fails by refusing legitimate sites, which is worse than the narrow gap it shuts.

One more, small and easy to get wrong. The content-type check compares the MIME essence exactly:

function mimeEssence(header: string): string {
  return header.split(";", 1)[0]!.trim().toLowerCase();
}

The original was includes("application/json"), which also accepts application/jsonp, and accepts any type at all that merely names an allowed one in a parameter. A content-type header has structure, and matching it by substring throws that structure away.

The body cap had a wrinkle I didn’t anticipate. Refusing an oversized body is right for JSON, where a truncated document isn’t a document. It’s wrong for HTML, where the metadata sits in the first 20KB and the tail is inlined application script. An Instagram reel is about 745KB in one document, and refusing it turned a perfectly readable page into a 502. HTML truncates now, JSON still refuses, and the read stops at the cap either way, so memory stays bounded.

The wait, and the thumbnail

While identification runs, the screen shows the dog mascot with a spinner and the line “Working out which game this is”, because a request that makes an outbound fetch and a model call deserves an honest wait rather than a skeleton pretending to be fast.

The source preview sizes its thumbnail from the dimensions the provider reported, clamped, with the height fixed and the width left to Yoga’s aspectRatio. A bare height gives a zero-width view: layout runs before the image decodes, and expo-image fills the box it’s handed rather than reporting a size back into layout.

What transfers

If you take one thing from this, take the pinning. Anything that accepts a URL and fetches it has this exact problem: a link preview, an avatar-by-URL field, a webhook tester, an RSS reader. The check-then-fetch version looks correct and isn’t. The rest is the same instinct further down. Canonicalise before you cache, so the key means something. Let each rung fail in the way that’s true of it instead of uniformly. Tell the user which rung answered. Put the expensive call behind a gate that only opens once the cheap one has given up.

Part 3 is the free tier: where the cap is enforced, and what happens when a purchase and a webhook race each other. Part 4 is App Review.

Every design document behind this post is in the repository under docs/, including the one admitting the two-provider limit was never a decision. And if you’ve shared a link into Barklog and it got the game wrong, I’d like that link. The failures I can reproduce are the ones I can fix.

Comments