generative pre-trained transformer (GPT)

Generative pre-trained transformer (GPT) is a family of autoregressive large language models (LLMs) that use the transformer architecture and are pre-trained on large text corpora using a generative next-token objective. Recent GPT models extend the same next-token objective beyond text to multimodal data such as images and audio.

Then, they’re adapted to tasks through prompting, fine-tuning, intermediate methods such as instruction tuning, or preference alignment with reinforcement learning from human feedback (RLHF). Current GPT models are also post-trained to reason: they spend internal reasoning tokens before answering, and you can tune how much of that thinking they do at inference time with a reasoning effort setting.

Most GPT models use a decoder-only stack of the transformer with masked self-attention and causal decoding, which enables efficient parallelization during training and fluent text generation across a wide range of downstream tasks.

The newest flagships vary that stack. GPT-6 Astra, for example, reportedly uses a recurrent-depth or “looped transformer” design that reuses layers to spend more computation per token, though OpenAI hasn’t confirmed its architecture.

A person holding a pen beside a large machine showing vaporwave-styled retro computer images, with panels labeled OpenAI, image data, and Base64, next to a Python logo chip wired into a brain.

Tutorial

Generate Images in Python With OpenAI's GPT Image API

Learn how to generate images from text prompts with OpenAI's GPT Image API in Python, tune their size and quality, and edit them with a follow-up prompt.

intermediate ai api

For additional information on related topics, take a look at the following resources:

Have a question about this? Mentor AI can show you examples, compare related terms, and point you to tutorials.

  • Ask Mentor AI
  • Pick a question and Mentor AI takes it from here.
By Leodanis Pozo Ramos • Updated Oct. 20, 2026
AI Coding Glossary Share Feedback