Steps to Make Accept: text/markdown Work Everywhere

I asked an agent to check whether appheat.co answered Accept: text/markdown the way acceptmarkdown.com opens in a new tab tests for.

It did not.

That bug sounded small enough to be a quick patch: wrap the Cloudflare Worker, read the Accept header, return Markdown for post pages, ship it. Instead, I built the obvious version, deployed it, and then spent most of an hour chasing what looked like a cache problem.

It was not a cache problem.

This is the debugging note, including the wrong turns.

What content negotiation actually means here#

Two different things get called “serving Markdown to AI agents,” and I had to separate them before the bug made sense:

1. A separate URL. appheat.co/posts/some-slug.md returns raw Markdown. You link to it with <link rel="alternate" type="text/markdown">. This is easy to build and easy to verify with a plain curl.
2. Content negotiation on the same URL. appheat.co/posts/some-slug/ returns HTML by default, and Markdown when the request sends Accept: text/markdown. No second URL. This is what acceptmarkdown.com actually tests, and it is the pattern more agent tooling is starting to assume.

AppHeat already had the first one from an earlier pass on the site’s AEO strategy. Accept: text/markdown was the missing piece.

The straightforward part#

The site builds through @astrojs/cloudflare, which generates its own Worker entry file at build time (dist/server/entry.mjs). I did not want to fork the adapter for one header experiment, so I used a small post-build script that wraps the generated entry with our own fetch handler:

// apps/personal-site/scripts/wrap-entry.mjs (trimmed)
const POST_ROUTE = /^\/posts\/([^/]+)\/$/;

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    const accept = request.headers.get("Accept") ?? "";

    if (accept.includes("text/markdown") && POST_ROUTE.test(url.pathname)) {
      const slug = url.pathname.match(POST_ROUTE)[1];
      const mdUrl = new URL(request.url);
      mdUrl.pathname = `/posts/${slug}.md`;

      const mdResponse = await _originalFetch(new Request(mdUrl), env, ctx);
      if (mdResponse.ok) {
        return new Response(mdResponse.body, {
          status: 200,
          headers: {
            "Content-Type": "text/markdown; charset=utf-8",
            "Cache-Control": "public, max-age=3600",
            "Vary": "Accept",
          },
        });
      }
    }

    // Normal HTML path, with Vary: Accept added so caches
    // know the response depends on the Accept header.
    const response = await _originalFetch(request, env, ctx);
    const headers = new Headers(response.headers);
    headers.set("Vary", "Accept");
    return new Response(response.body, { status: response.status, headers });
  },
};

The shape was boring on purpose: if the request wants Markdown and matches a post URL, internally re-fetch the existing per-post .md route and return that body with the right content type. Otherwise, fall through to the normal HTML response and add Vary: Accept so shared caches know the representation depends on the header.

package.json runs it after every build:

"build": "BUILD_TARGET=cloudflare astro build && node scripts/wrap-entry.mjs"

I built it, ran the site checks, committed, and deployed.

Where it fell apart#

curl -s -H "Accept: text/markdown" https://appheat.co/posts/aeo-from-the-jump/ | head -c 200

HTML. Every time. Very funny, production.

The obvious suspect was Cloudflare’s CDN cache serving a stale copy of the page from before the Worker change. So I did the extremely normal thing where you decide the cache is guilty and keep interrogating it:

• Added Vary: Accept to every HTML response, so the CDN would (in theory) cache HTML and Markdown separately.
• Added CDN-Cache-Control: no-cache to force edge revalidation on every request.
• Tried cache-busting query strings.
• Tried purging the zone cache directly through the Cloudflare API.

None of it changed the result. Still HTML, every time, cf-cache-status: HIT.

The clue was sitting in a request I had already made and half-ignored: a request to a path that did not exist — a 404 — correctly returned the negotiation headers (Vary: Accept, plus a debug header I had added). A request to a real post URL, even with a random query string appended, never did.

Those two results only make sense one way: the Worker was running for the 404, and never running for the real post page.

The actual cause#

wrangler.json for this site includes an assets binding — the mechanism Cloudflare uses to serve prerendered static files (our post pages are static HTML, built ahead of time by Astro) directly from the edge:

"assets": { "binding": "ASSETS", "directory": "../client" }

By default, when a request’s path matches a file that exists in that assets directory, Cloudflare serves it directly from the edge asset store and never invokes the Worker script at all. That is a feature, not a bug. Static content should not pay for a Worker invocation it does not need. But it also means any custom logic living in the Worker’s fetch handler, including this content-negotiation wrapper, simply never runs for those paths.

The post pages are exactly that kind of file: static HTML built at compile time. So every request to /posts/aeo-from-the-jump/ was served straight from the asset store, before wrap-entry.mjs ever got a chance to look at the Accept header. The Vary header, the cache-busting, the cache purge — none of it could have worked, because the code that would have set those headers was not in the request path.

The fix is one field, in the source wrangler.jsonc:

{
  // ...
  "assets": { "run_worker_first": ["/posts/*"] }
}

run_worker_first tells Cloudflare to route matching paths to the Worker script first, instead of straight to the asset store. I scoped it to /posts/* instead of setting it to true globally, because images, JS chunks, _astro/*, and the rest of the static stuff should still stay boring and fast. Only post routes need the extra Worker hop.

Verifying it for real this time#

Locally, with wrangler dev:

curl -s -D - -o /dev/null http://localhost:8787/posts/aeo-from-the-jump/
# HTTP/1.1 200 OK
# Content-Type: text/html; charset=utf-8
# Vary: Accept

curl -s -D - -o /dev/null -H "Accept: text/markdown" http://localhost:8787/posts/aeo-from-the-jump/
# HTTP/1.1 200 OK
# Content-Type: text/markdown; charset=utf-8
# Vary: Accept

And in production, after deploying:

curl -s -H "Accept: text/markdown" https://appheat.co/posts/aeo-from-the-jump/ | head -3
# ---
# title: "AEO from the jump: build pages people can quote"
# description: "A practical note on making pages easy to read, cite, and hard to misquote."

Same URL, no query string, no cache-busting. A plain request gets HTML. An Accept: text/markdown request gets the post’s frontmatter and body as Markdown. That was the acceptmarkdown.com pattern, finally working.

The scorecard caught the shortcut#

“Working” was also optimistic. Running the actual acceptmarkdown.com opens in a new tab readiness check against the live post put a number on the shortcut I had left in the implementation.

75/100. Two real passes, two real warnings:

• Honors q-values — the check sent an Accept header where text/html had a higher quality value than text/markdown, meaning the client actually preferred HTML. The response came back as Markdown anyway.
• Returns 406 for unsupported types — the check sent an Accept value the server has no representation for at all. The response came back 200 with HTML instead of 406 Not Acceptable.

Both pointed at the same smaller mistake: the negotiation logic was accept.includes("text/markdown"), a substring check with no concept of preference or unsupported media types. It answered “does the word markdown appear in this header?” instead of “what representation does the client actually prefer?”

Fixing q-values#

RFC 9110 §12.5.1 opens in a new tab defines Accept as a weighted list — each media range can carry a q parameter from 0 to 1, and the highest q wins. Accept: text/html;q=0.9, text/markdown;q=0.5 is a client saying “I can take Markdown, but I’d rather have HTML.” My first version never looked at q at all.

The fix parses the header properly and compares weights:

// apps/personal-site/scripts/wrap-entry.mjs (trimmed)
function parseAccept(header) {
  return header
    .split(",")
    .map((part) => {
      const pieces = part.trim().split(";").map((p) => p.trim());
      const type = pieces[0].toLowerCase();
      let q = 1;
      for (const param of pieces.slice(1)) {
        const match = param.match(/^q=([0-9.]+)$/i);
        if (match) q = parseFloat(match[1]);
      }
      return { type, q };
    })
    .filter((entry) => entry.type && !Number.isNaN(entry.q));
}

function qFor(parsed, mediaType) {
  const [type] = mediaType.split("/");
  let best = -1;
  for (const entry of parsed) {
    if (entry.type === mediaType || entry.type === `${type}/*` || entry.type === "*/*") {
      if (entry.q > best) best = entry.q;
    }
  }
  return best;
}

const markdownQ = qFor(parsed, "text/markdown");
const htmlQ = qFor(parsed, "text/html");
const wantsMarkdown = parsed.length > 0 && markdownQ > 0 && markdownQ >= htmlQ;

Markdown only wins when its q is present, greater than zero, and at least as high as HTML’s. text/html;q=0.9, text/markdown;q=0.5 now serves HTML, which is what the client asked for.

Adding the 406#

RFC 9110 says returning 406 Not Acceptable when nothing on the server matches the client’s Accept header is optional — a server can also send its default representation. But in this case “optional and more correct” was still more correct, so if neither text/markdown, text/html, nor a wildcard is acceptable, the Worker now says so explicitly instead of silently defaulting to HTML:

const nothingAcceptable = parsed.length > 0 && markdownQ <= 0 && htmlQ <= 0;

if (nothingAcceptable) {
  return new Response("Not Acceptable", {
    status: 406,
    headers: {
      "Content-Type": "text/plain; charset=utf-8",
      "Vary": "Accept",
    },
  });
}

The parsed.length > 0 guard matters: a request with no Accept header at all, or a bare Accept: */*, should still get HTML — only an explicit, non-matching Accept value triggers the 406.

Verifying both fixes#

# HTML preferred over Markdown via q-values — should be HTML
curl -s -D - -o /dev/null \
  -H "Accept: text/html;q=0.9, text/markdown;q=0.5" \
  https://appheat.co/posts/aeo-from-the-jump/
# content-type: text/html

# Markdown preferred over HTML via q-values — should be Markdown
curl -s -D - -o /dev/null \
  -H "Accept: text/html;q=0.5, text/markdown;q=0.9" \
  https://appheat.co/posts/aeo-from-the-jump/
# content-type: text/markdown; charset=utf-8

# Unsupported type — should be 406
curl -s -D - -o /dev/null \
  -H "Accept: application/x-content-negotiation-probe" \
  https://appheat.co/posts/aeo-from-the-jump/
# HTTP/2 406
# content-type: text/plain; charset=utf-8

# No Accept header at all — should still be HTML, not 406
curl -s -D - -o /dev/null https://appheat.co/posts/aeo-from-the-jump/
# content-type: text/html

All four came back as expected against the live site. That is the same shape of check the scorecard runs; I did not re-run acceptmarkdown.com just to get a cleaner screenshot, because the direct curl checks against production are the evidence I trust more.

What I’d reuse#

If you are deploying an Astro site, or any static-plus-Worker site on Cloudflare, and you need request-time logic — content negotiation, A/B routing, geo-based responses, anything that reads a header before deciding what to return — check whether the paths you care about are prerendered static files. If they are, and you are using the assets binding, your Worker code will not run on them by default. run_worker_first is the setting that changes that, and it takes either true or an array of route globs so you can scope the cost to only the paths that need it.

The debugging shortcut worth keeping: if a header or cache setting you added is not showing up, do not start with “the cache is stale.” First prove the code that sets the header is executing at all. A request to a path that cannot be a built static asset — a 404, a random path — will tell you whether your Worker is even in the request path.

Once the Worker was actually in the request path, “it returns Markdown when asked” was still not the same as “it implements content negotiation correctly.” A real Accept header parser — one that reads q values and knows the difference between “no match” and “wildcard match” — is a small amount of code. Writing accept.includes("text/markdown") and calling it done was the shortcut that produced two more smaller bugs.

Caveats#

This makes appheat.co negotiable by Accept: text/markdown for post pages specifically. It does not guarantee any particular AI agent or tool sends that header, and it does not replace the bulk llms-full.txt endpoint or the discoverable per-post .md URLs already on the site. Those are still useful for tools that prefer an explicit URL over content negotiation. This is one more path into the same content, for the tools that use it.

The scorecard image in this post reflects the site’s state before the q-value and 406 fixes. It is the evidence that motivated the follow-up, not a claim about the current score. I verified the fix by replicating the checks with curl against production, not by re-running acceptmarkdown.com for a prettier number.

Commands used#

# Local verification
pnpm appheat:build
pnpm -C apps/personal-site exec wrangler dev --port 8787

# Production verification, after the routing fix
curl -s -H "Accept: text/markdown" https://appheat.co/posts/aeo-from-the-jump/

# Production verification, after the q-value and 406 fix
curl -s -D - -o /dev/null -H "Accept: text/html;q=0.9, text/markdown;q=0.5" https://appheat.co/posts/aeo-from-the-jump/
curl -s -D - -o /dev/null -H "Accept: application/x-content-negotiation-probe" https://appheat.co/posts/aeo-from-the-jump/

Disclosure#

Drafted with AI assistance from the actual debugging session, then edited around the real lesson. The header behavior and terminal output shown were captured on 2026-07-01 against a local wrangler dev instance and the live appheat.co production deployment. The acceptmarkdown.com scorecard screenshot was captured by the site owner in a browser, not generated by the AI assistant.