AI & ML Glossary

The simulation of human intelligence processes by machines, especially computer systems, including learning, reasoning, and self-correction.

A subset of AI that enables systems to learn and improve from experience without being explicitly programmed.

A subset of machine learning using neural networks with multiple layers to learn hierarchical representations of data.

A computing system inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers.

A step-by-step procedure or formula for solving a problem or performing a computation, especially by a computer.

A mathematical representation of a system or phenomenon trained on data to make predictions or decisions.

The process of feeding data to a machine learning model so it can learn patterns and improve its performance.

The process of using a trained model to make predictions on new, unseen data.

A machine learning approach where models are trained on labeled data with known inputs and outputs.

A machine learning approach where models find patterns in unlabeled data without predefined outputs.

A deep learning architecture designed for image processing, using convolutional layers to automatically learn spatial features.

A neural network architecture with connections forming cycles, designed for sequential data processing.

An advanced RNN variant that can learn long-term dependencies using memory cells and gating mechanisms.

A state-of-the-art neural architecture using self-attention mechanisms to process sequential data in parallel.

A neural architecture with an encoder that processes input and a decoder that generates output, used in seq2seq tasks.

A mechanism that allows models to focus on relevant parts of input when processing, improving performance on complex tasks.

An attention mechanism where the model attends to different positions within the same sequence.

An attention mechanism using multiple parallel attention operations to capture different types of relationships.

A neural network that learns to compress data into a lower-dimensional representation and then reconstruct it.

A framework with two networks (generator and discriminator) that compete to generate realistic synthetic data.

A field of AI focused on enabling computers to understand, interpret, and generate human language.

The process of breaking text into individual tokens (words, subwords, or characters) for processing.

A numerical representation of text where words/sentences are mapped to dense vectors capturing semantic meaning.

A technique for learning word embeddings using shallow neural networks, popular for its efficiency.

A method for learning word embeddings that combines global matrix factorization with local context windows.

Bidirectional Encoder Representations from Transformers, a pre-trained language model using masked language modeling.

Generative Pre-trained Transformer, an autoregressive language model trained to predict the next token in a sequence.

A model that predicts the probability of a sequence of words or tokens based on previous context.

A numerical value indicating how much focus a model should place on different input elements.

The maximum number of tokens a model can attend to simultaneously when making predictions.

A field of AI that enables machines to interpret and understand visual information from images and videos.

The task of assigning an image to one of several predefined categories or classes.

The task of identifying and locating objects within an image, providing bounding boxes and class labels.

The task of classifying each pixel in an image to a specific semantic class.

The task of detecting and segmenting individual instances of objects in an image.

A layer in CNNs that applies filters to input to learn local features like edges and textures.

An operation that reduces spatial dimensions by aggregating values in local regions.

The output of a convolutional or pooling layer, representing learned visual features.

A mathematical function applied to neuron outputs to introduce non-linearity, enabling learning of complex patterns.

A technique that normalizes layer inputs to have zero mean and unit variance, improving training stability.

A machine learning paradigm where agents learn by interacting with an environment and receiving rewards or penalties.

An autonomous entity that learns to make decisions and take actions in an environment.

The external system with which an agent interacts, providing states, actions, and rewards.

A signal provided by the environment indicating the quality or success of an agent's action.

A strategy that maps states to actions, defining how an agent should behave in different situations.

A function that estimates the expected cumulative reward from a given state or state-action pair.

An off-policy RL algorithm that learns the value of actions in states without requiring a model of the environment.

A family of RL algorithms that learn by directly optimizing the policy parameters using gradient ascent.

The tradeoff between exploring new actions to gather information and exploiting known good actions.

A mathematical framework for modeling decision-making where outcomes depend only on the current state.

A function that measures the difference between predicted and actual values, guiding model optimization.

An optimization algorithm that iteratively updates model parameters to minimize the loss function.

An algorithm for computing gradients of the loss function with respect to model parameters using the chain rule.

A hyperparameter controlling the step size in gradient descent, affecting convergence speed and stability.

The number of samples processed before updating model parameters, affecting training stability and speed.

A complete pass through the entire training dataset during model training.

The process of evaluating a model on data it hasn't seen during training to assess generalization.

A held-out dataset used to evaluate final model performance after training and hyperparameter tuning.

A condition where a model learns training data too well, including noise, and performs poorly on new data.

A condition where a model is too simple to capture underlying patterns in the data.

An adaptive learning rate optimization algorithm combining momentum and RMSprop for efficient training.

A gradient descent variant using randomly selected samples, faster and more memory-efficient than batch GD.

A technique that accelerates gradient descent by accumulating gradients over time.

A technique that prevents overfitting by adding penalty terms to the loss function.

A regularization technique that randomly deactivates neurons during training to prevent co-adaptation.

A technique that stops training when validation performance plateaus, preventing overfitting.

A strategy that adjusts the learning rate during training, often decreasing it over time.

A regularization technique that penalizes large weights, encouraging simpler models.

A loss function commonly used for classification that measures the difference between predicted and true distributions.

A loss function commonly used for regression that measures average squared differences between predictions and targets.

The proportion of correct predictions among all predictions made by a model.

The proportion of true positives among all positive predictions, measuring false alarm rate.

The proportion of true positives among all actual positives, measuring detection rate.

The harmonic mean of precision and recall, providing a balanced performance metric.

A curve plotting true positive rate against false positive rate at different classification thresholds.

The area under the ROC curve, measuring overall classifier performance (0.5 = random, 1.0 = perfect).

A table showing true positives, true negatives, false positives, and false negatives for a classification model.

The average absolute differences between predicted and actual values, measuring prediction error magnitude.

The square root of MSE, providing error measurement in original data units.

A metric for language models measuring how well the model predicts a test dataset.

A collection of data samples used for training, validating, or testing machine learning models.

An individual measurable property or characteristic used as input to a machine learning model.

The target or ground truth value for a sample in supervised learning.

The process of creating new features or transforming existing ones to improve model performance.

Techniques that artificially expand datasets by creating modified copies of existing samples.

The process of scaling features to a similar range, improving training stability and convergence.

The process of transforming features to have zero mean and unit variance.

A dataset where classes have significantly different frequencies, complicating model training.

A situation where information from outside the training set influences model training, causing inflated performance.

A configuration parameter set before training, such as learning rate or number of layers.

A technique where a model trained on one task is adapted for another task, leveraging learned representations.

The process of adapting a pre-trained model to a specific task by training on task-specific data.

The ability to recognize or classify unseen classes without any training examples.

The ability to learn from very few training examples, often using meta-learning techniques.

Learning how to learn, enabling models to quickly adapt to new tasks with limited data.

Techniques for training models on data from one domain and deploying them in another domain.

A technique for training a small student model to mimic a larger teacher model.

Techniques that combine predictions from multiple models to improve overall performance.

A training technique that improves model robustness by training on adversarial examples.

The art of crafting effective prompts to elicit desired outputs from large language models.

A popular deep learning framework known for dynamic computation graphs and ease of use.

An open-source machine learning framework developed by Google for building and training models at scale.

A high-level neural networks API that can run on top of TensorFlow, designed for rapid prototyping.

A Python library for numerical computing, providing support for arrays and mathematical functions.

A Python library for data manipulation and analysis, providing dataframe structures.

A Python library providing simple tools for data mining and machine learning.

A parallel computing platform by NVIDIA enabling GPU acceleration for machine learning.

A containerization platform for packaging and deploying machine learning applications.

An open-source tool for creating and sharing documents with live code, equations, and visualizations.

A version control system for tracking changes in code, essential for collaborative development.

Error from overly simplistic assumptions in learning algorithms, causing systematic underfitting.

Error from sensitivity to fluctuations in training data, causing overfitting to noise.

The fundamental principle that reducing one source of error often increases the other.

A model's ability to perform well on unseen data, the ultimate goal of machine learning.

A model's ability to maintain performance under distribution shifts or adversarial inputs.

The degree to which a model's predictions can be understood and explained by humans.

The ability to provide clear reasons for a model's decisions or predictions.

A technique that divides data into multiple folds for robust model evaluation.

Cross-validation that divides data into k equal folds, training and testing k times.

Adjusting model outputs to match true probability distributions, improving prediction reliability.