AI and ML glossary
99 terms explained in plain English, each with a longer explanation, real examples and the related concepts worth learning next.
99 terms
Concepts
Generative AI
AI systems that can create new content (text, images, code, audio) based on learned patterns from training data.
Natural Language Processing (NLP)
The branch of AI focused on enabling computers to understand, process, and generate human language.
Computer Vision (CV)
The branch of AI that enables computers to interpret and understand visual information from images and videos.
Overfitting
When a model learns training data too well, including noise, and performs poorly on new unseen data.
Underfitting
When a model is too simple to capture underlying patterns in data, performing poorly on both training and test data.
Token
The smallest unit of text that an LLM processes, typically a word, subword, or character.
Context Window
The maximum number of tokens an LLM can process in a single input, determining conversation length.
Hallucination
When an LLM generates confidently incorrect, fabricated, or nonsensical information.
Fine-Grained Access Control (FGAC)
Security mechanism that provides detailed control over who can access specific resources or data.
Benchmark
A standardized dataset and evaluation metric used to assess and compare model performance.
Model Capacity
The ability of a model to learn complex patterns, determined by its architecture and number of parameters.
Parameter
A learnable weight in a neural network that is adjusted during training to minimize loss.
Hyperparameter
Configuration settings for model training that are set before training, not learned during training.
Learning Rate
A hyperparameter that controls the step size for updating model weights during training.
Batch Size
The number of training examples processed together in one iteration before updating model weights.
Epoch
One complete pass through the entire training dataset during model training.
Activation Function
A mathematical function applied to neuron outputs to introduce non-linearity enabling complex pattern learning.
Softmax
An activation function that converts model outputs to probability distributions across classes.
Imbalanced Data
A dataset where classes have significantly different numbers of examples, creating training challenges.
Evaluation Metric
A quantitative measure of model performance on specific tasks, enabling objective assessment and comparison.
Confusion Matrix
A table summarizing prediction results by comparing predicted and actual labels for classification tasks.
Precision
The proportion of predicted positive instances that are actually positive, measuring false positive rate.
Recall
The proportion of actual positive instances correctly predicted, measuring false negative rate.
F1 Score
The harmonic mean of precision and recall, providing a balanced single metric for classification.
AUC-ROC (Area Under the Receiver Operating Characteristic Curve)
A metric measuring the tradeoff between true positive rate and false positive rate across classification thresholds.
Object Detection
A computer vision task that locates and classifies objects in images using bounding boxes.
Semantic Segmentation
A computer vision task that labels each pixel in an image with a class, assigning semantic meaning.
Instance Segmentation
A computer vision task combining object detection and semantic segmentation, detecting and segmenting individual objects.
Pose Estimation
A computer vision task that detects and localizes human body keypoints to understand posture and movement.
Sentiment Analysis
An NLP task that determines the emotional tone or opinion expressed in text.
Named Entity Recognition (NER)
An NLP task that identifies and classifies named entities (persons, locations, organizations, etc.) in text.
Machine Translation
An NLP task that automatically translates text from one language to another.
Question Answering (QA)
An NLP task that automatically answers questions based on provided context or knowledge.
Text Summarization
An NLP task that automatically generates concise summaries of longer documents.
Speech Recognition
A task that converts spoken audio into written text.
Text-to-Speech (TTS)
A task that converts written text into spoken audio.
Code Generation
An AI task that automatically writes or completes code based on natural language or partial code.
Multimodal AI
AI systems that process and integrate information from multiple modalities (text, images, audio, video, etc.).
Knowledge Graph
A structured representation of knowledge using entities and relationships, enabling semantic understanding.
Semantic Search
Search that understands meaning and intent rather than just keyword matching.
Recommendation System
An AI system that predicts and suggests items (movies, products, content) that users will likely prefer.
Anomaly Detection
An unsupervised learning task that identifies abnormal or unusual data points.
Time Series Forecasting
A task that predicts future values based on historical sequential data.
"Attention is All You Need" Paper
Seminal 2017 paper introducing the Transformer architecture that revolutionized NLP and AI.
Scaling Laws
The observation that model performance improves predictably with scale (data, compute, parameters).
Emergence
The phenomenon where large models suddenly develop new capabilities not present in smaller models.
Interpretability
The ability to understand and explain AI model decisions and internal representations.
Fairness and Bias in AI
The challenge of building AI systems that treat all groups fairly without discriminatory bias.
Privacy and Security in AI
Protecting sensitive data and AI models from unauthorized access, leakage, and attacks.
Adversarial Attack
Intentional perturbations to inputs designed to fool AI models into making incorrect predictions.
Fundamentals
Artificial Intelligence (AI)
The simulation of human intelligence by machines, enabling computers to perform tasks that normally require human cognition.
Machine Learning (ML)
A subset of AI that enables systems to learn and improve from experience without being explicitly programmed.
Deep Learning
A subset of machine learning using artificial neural networks with multiple layers to learn hierarchical representations of data.
Neural Network
A computational model inspired by biological neural networks that consists of interconnected nodes (neurons) organized in layers.
Models
Transformer
A neural network architecture that uses self-attention mechanisms to process sequential data in parallel, enabling efficient processing of long sequences.
Large Language Model (LLM)
A deep learning model trained on vast amounts of text data to understand and generate human language.
Convolutional Neural Network (CNN)
A neural network architecture specialized for processing grid-structured data like images using convolutional layers.
Recurrent Neural Network (RNN)
A neural network architecture with loops that process sequential data by maintaining hidden state across time steps.
Random Forest
An ensemble learning method that builds many decision trees and combines their predictions.
Gradient Boosting
An ensemble method that sequentially builds trees, each correcting errors of previous trees.
Decision Tree
A tree-structured model that makes predictions by recursively splitting data based on feature thresholds.
Support Vector Machine (SVM)
A supervised learning algorithm that finds the optimal hyperplane to separate classes in high-dimensional space.
K-Means Clustering
An unsupervised learning algorithm that partitions data into k clusters by minimizing within-cluster variance.
Generative Adversarial Network (GAN)
A framework with two neural networks (generator and discriminator) that compete to generate realistic data.
Variational Autoencoder (VAE)
A generative model that learns to encode data into latent space and decode to reconstruct data.
Diffusion Model
A generative model that gradually removes noise from data to generate new samples.
BERT (Bidirectional Encoder Representations from Transformers)
A pre-trained language model that uses bidirectional context for understanding text.
GPT (Generative Pre-trained Transformer)
A series of large language models by OpenAI using left-to-right generation.
Techniques
Supervised Learning
Machine learning approach where the model learns from labeled training data (inputs with known outputs).
Unsupervised Learning
Machine learning approach where the model learns patterns from unlabeled data without predefined target outputs.
Reinforcement Learning (RL)
Machine learning approach where agents learn optimal behavior by interacting with environments and receiving rewards or penalties.
Prompt Engineering
The practice of crafting effective input prompts to guide LLMs toward desired outputs.
Fine-Tuning
The process of adapting a pre-trained model to a specific task by training it on task-specific data.
Transfer Learning
Machine learning technique that reuses knowledge from one task to improve learning on another related task.
Embedding
A numerical representation of text, images, or other data that captures semantic meaning in a continuous vector space.
Attention Mechanism
A neural network component that allows models to focus on relevant parts of input by computing weighted combinations.
Backpropagation
The algorithm used to compute gradients in neural networks, enabling efficient training through gradient descent.
Loss Function
A mathematical function that measures the difference between predicted and actual values, guiding model training.
Optimization
The process of adjusting model parameters to minimize loss and improve performance.
Regularization
Techniques that add constraints to model training to prevent overfitting and improve generalization.
Validation
The process of assessing model performance on held-out data to estimate generalization ability.
Retrieval Augmented Generation (RAG)
A technique that combines document retrieval with LLM generation to answer questions using external knowledge.
Few-Shot Learning
The ability to learn from very few examples, enabling quick adaptation to new tasks.
Chain-of-Thought (CoT)
A prompting technique that improves LLM reasoning by encouraging step-by-step explanation.
Gradient Descent
An optimization algorithm that iteratively updates model weights by moving in the direction of steepest loss decrease.
Dropout
A regularization technique that randomly disables neurons during training to prevent co-adaptation and reduce overfitting.
Batch Normalization
A technique that normalizes layer inputs across a batch to accelerate training and improve stability.
Residual Connection (Skip Connection)
A neural network connection that bypasses layers, adding the input directly to output, enabling deeper networks.
Data Augmentation
Techniques that artificially expand training data by applying transformations while preserving labels.
Cross-Validation
A technique that splits data into multiple folds to get more robust performance estimates.
Hyperparameter Tuning
The process of finding optimal hyperparameter values to maximize model performance.
Neural Architecture Search (NAS)
Automated methods for designing optimal neural network architectures rather than manual design.
Ensemble Learning
Combining multiple models to make predictions, typically improving performance over individual models.
Feature Engineering
The process of creating, selecting, and transforming features to improve model performance.
Feature Selection
The process of selecting the most relevant features to improve model performance and reduce complexity.
Dimensionality Reduction
Techniques for reducing the number of features while preserving important information.
Principal Component Analysis (PCA)
A linear dimensionality reduction technique that finds principal components capturing maximum variance.
Transfer Learning in NLP
Adapting pre-trained language models to specific NLP tasks by fine-tuning.
Tools
Ready to Build Production AI Systems?
Join a small cohort learning directly from an engineer who's shipped AI at Atlassian and PhonePe scale.
20 weeks · Weekend live classes · 1:1 mentorship · 20+ projects.
20 Weeks
Live Weekend Classes
< 30 Seats
Small Batch
Next Soon
Get priority access
7-Day
Money-Back Guarantee
Taught by Debasish Maji · Senior AI Engineer · Ex-Atlassian (Rovo Agent) · Ex-PhonePe (550M+ users)
Get notified when the next batch opens + free AI resources