In plain words

time to first token

The duration between sending a prompt to an AI model and receiving its very first generated word.

A bit more detail

This metric measures the responsiveness of an inference system. It is heavily influenced by how quickly the server processes the prompt history before generating new content.

Stories that use this word