01/13T+ 0.000 sYOU

You ask a question

You type one sentence and press Enter. To you it is a message. To the machine it is the start of a relay race through thousands of kilometres of cable and billions of calculations.

≈ 60 bytes of text

In tech terms: HTTPS request

aiinfra.blog / The anatomy of an answer

Every answer
begins with Enter

Behind one keystroke, an agent plans, a GPU cluster wakes up and billions of parameters fire in sequence. Follow a single prompt through the entire stack, from the first token to the last.

Scroll to send it

Illustrative numbers. The flow is real.

Every answer begins with Enter

  1. 01YOU

    You ask a question

    You type one sentence and press Enter. To you it is a message. To the machine it is the start of a relay race through thousands of kilometres of cable and billions of calculations.

    ≈ 60 bytes of text · HTTPS request

  2. 02INFRA

    It crosses the world in a blink

    Your message flies to a data centre, often on another continent. A gatekeeper checks who you are, makes sure nobody is flooding the line, and sends you to a free machine.

    ≈ 40 ms across the ocean · API gateway · TLS · rate limiting

  3. 03AGENT

    An assistant picks up the job

    Before any thinking happens, a program wrapped around the AI reads your request, recalls what you said earlier, and checks which tools it is allowed to use. This program is the agent.

    3 tools on hand · Agent runtime

  4. 04AGENT

    It looks things up first

    The AI never memorised this company's refund policy. So the agent turns your question into a point in a "map of meaning" and fetches the documents sitting closest to it.

    top 5 of 1.2 million passages · Retrieval-augmented generation (RAG)

  5. 05MODEL

    Words become numbers

    Computers cannot read. Every word is chopped into small pieces called tokens, and each piece is swapped for a number. From here on, the AI only ever sees numbers.

    "refund" → 21639 · Tokenizer

  6. 06INFRA

    It joins a very fast queue

    Thousands of people are asking at the same moment. A scheduler packs everyone's requests together, like a bus that never leaves half empty, so the expensive chips are never idle.

    256 conversations per batch · Continuous batching · KV cache

  7. 07INFRA

    The heavy lifting begins

    Eight chips, each costing about as much as a car, hold the AI's brain between them. To write a single word, the whole brain has to stream through their memory once.

    ≈ 3 TB per second, per chip · GPU cluster · tensor parallelism

  8. 08MODEL

    Billions of dials turn at once

    Your numbers rise through 80 layers. In each one, every word asks "which other words matter to me?" and updates itself. The dials were set months earlier by reading trillions of words.

    70 billion parameters · Transformer · self-attention

  9. 09MODEL

    It guesses the next word

    Out comes a list of odds for every possible next word. It picks one, adds it to the sentence, and runs the whole brain again for the next. That is the secret: extraordinarily good autocomplete.

    1 word every ≈ 25 ms · Next-token prediction · sampling

  10. 10AGENT

    It can act, not just talk

    Instead of a word, the AI can ask for a tool. The agent runs it, here by looking up your invoices, and hands back the result, so the answer is built on facts rather than guesses.

    1 live database query · Tool calling

  11. 11INFRA

    The answer arrives word by word

    Each new word is turned back into text and sent to your screen the moment it exists. That is why AI answers seem to type themselves.

    first word in ≈ 0.3 s · Streaming · server-sent events

  12. 12ALL

    So what is AI, really?

    A model that learned language by predicting the next word. An agent that gives it memory and hands. An infrastructure that makes it fast enough to feel like magic. Three layers, one answer, under two seconds.

    ≈ 1.8 s end to end · Model · Agent · Infra