Token

The unit of text a language model reads and writes, typically a word or part of a word. Model usage and pricing are counted in tokens.

Used in these articles

Related terms

AI and machine learning

• AI agent
• Brain-computer interface
• Combinatorial optimization
• Computer vision
• Context window
• Deep learning
• Domain generalization
• Electroencephalography
• Frontier model
• Hallucination
• Inference
• Large language model
• Machine learning
• Matrix multiplication
• Memristor
• Metaheuristic
• Model drift
• Multiply-accumulate
• Neural network
• Neuromorphic computing
• NP-hard
• Operations research
• Quantization
• Reinforcement learning
• Reservoir computing
• Self-play
• Spiking neural network
• SSVEP
• Training
• Transformer