100 AI Glossary Terms Explained to The Rest of Us
| # | Term | Definition |
|---|---|---|
| 1 | Activation function | A function used in neural networks to introduce nonlinearity, transforming the input signal to the output signal. |
| 2 | Adaboost | Adaboost is a popular ensemble learning algorithm that combines weak classifiers into a strong classifier by weighting them according to their accuracy. |
| 3 | Adversarial attack | An adversarial attack is a type of attack on a machine learning model, where an attacker intentionally perturbs the input data to cause misclassification or other undesired behavior. |
| 4 | AlphaGo | AlphaGo is a computer program developed by DeepMind that defeated the world champion in the ancient Chinese board game Go, using a combination of deep reinforcement learning and Monte Carlo tree search. |
| 5 | Artificial general intelligence | Artificial general intelligence refers to the hypothetical ability of a machine to understand or learn any intellectual task that a human being can. |
| 6 | Artificial intelligence | Artificial intelligence refers to the field of computer science and engineering that deals with the creation of intelligent machines that can perform tasks requiring human-like intelligence. |
| 7 | Attention mechanism | An attention mechanism is a component used in neural networks to selectively focus on parts of the input data that are most relevant to the task, improving performance and interpretability. |
| 8 | Autoencoder | An autoencoder is a type of neural network used for unsupervised learning, where the network learns to encode and decode data, typically by mapping it to a lower-dimensional latent space. |
| 9 | Backpropagation | Backpropagation is a common algorithm used for training neural networks, which involves computing the gradients of the loss function with respect to the weights and updating them using gradient descent. |
| 10 | Bag of words | The bag of words model is a simple representation of text data used in natural language processing, where the text is represented as a vector of word counts or frequencies. |
| 11 | Bayesian optimization | Bayesian optimization is a technique used to tune the hyperparameters of a machine learning model, by modeling the performance of the model as a probability distribution and iteratively selecting the hyperparameters with the highest expected improvement. |
| 12 | Capsule network | A capsule network is a type of neural network that is designed to capture the hierarchical relationships between parts of an object, enabling it to recognize objects from different viewpoints and deformations. |
| 13 | Clustering | Clustering is a type of unsupervised learning where the algorithm groups similar data points into clusters based on the similarity of their features. |
| 14 | Convolutional neural network | A convolutional neural network is a type of neural network commonly used for image and video recognition, which uses convolutional layers to extract features from the input data. |
| 15 | Cross-validation | Cross-validation is a technique used to evaluate the performance of a machine learning model by partitioning the data into training and testing sets, and repeating the process with different splits. |
| 16 | Decision tree | A decision tree is a type of machine learning algorithm that involves recursively partitioning the input space based on the values of the input features, to make predictions based on the resulting regions. |
| 17 | Deep learning | Deep learning refers to a subset of machine learning that uses neural networks with multiple layers to learn complex representations of the input data. |
| 18 | Dropout | Dropout is a regularization technique used in neural networks to prevent overfitting, by randomly dropping out some neurons during training. |
| # | Term | Definition |
|---|---|---|
| 19 | Ensemble learning | Ensemble learning is a technique used in machine learning where multiple models are combined to improve overall performance or reduce overfitting. |
| 20 | Evolutionary algorithm | An evolutionary algorithm is a type of optimization algorithm inspired by natural selection, where a population of candidate solutions evolves over generations through mutation, selection, and reproduction. |
| 21 | Feature engineering | Feature engineering is the process of selecting and transforming the input features of a machine learning model to improve its performance. |
| 22 | Feedforward neural network | A feedforward neural network is a type of neural network where the information flows from the input layer to the output layer, without any feedback loops. |
| 23 | Fine-tuning | Fine-tuning is a technique used in transfer learning, where a pre-trained model is further trained on a new dataset with a similar task, to improve its performance on the new task. |
| 24 | Generative adversarial network | A generative adversarial network is a type of neural network used for generating new data, by training two networks that compete with each other: a generator network that creates new samples, and a discriminator network that tries to distinguish between real and fake samples. |
| 25 | Gradient descent | Gradient descent is a common optimization algorithm used in machine learning to minimize the loss function by iteratively adjusting the weights in the direction of the steepest descent of the gradient. |
| 26 | Hyperparameter | A hyperparameter is a parameter of a machine learning model that is set before training and cannot be learned from the data, such as the learning rate or regularization strength. |
| 27 | Image segmentation | Image segmentation is the process of partitioning an image into multiple regions or objects based on their visual characteristics, often used in computer vision tasks. |
| 28 | Inference | Inference is the process of using a trained machine learning model to make predictions on new, unseen data. |
| 29 | K-nearest neighbors | K-nearest neighbors is a type of machine learning algorithm used for classification or regression, where the output for a new input is determined by the labels of its K nearest neighbors in the training data. |
| 30 | Long short-term memory | Long short-term memory is a type of recurrent neural network used for sequence prediction or generation, where the network can selectively remember or forget previous inputs or outputs. |
| 31 | Machine learning | Machine learning is a subfield of artificial intelligence that focuses on creating algorithms that can learn from and make predictions on data, without being explicitly programmed. |
| 32 | Mean squared error | Mean squared error is a commonly used loss function in regression problems, which measures the average squared difference between the predicted values and the true values. |
| 33 | Neural network | A neural network is a type of machine learning algorithm that is inspired by the structure and function of the human brain, consisting of interconnected nodes or neurons that can learn from data. |
| 34 | Object detection | Object detection is the process of identifying and locating objects in an image or video, often used in computer vision applications. |
| 35 | One-hot encoding | One-hot encoding is a technique used to represent categorical data as binary vectors, where each dimension corresponds to a possible value of the category. |
| 36 | Overfitting | Overfitting is a common problem in machine learning where a model performs well on the training data but poorly on new, unseen data, due to memorizing noise or spurious correlations in the training set. |
| # | Term | Definition |
|---|---|---|
| 37 | Precision and recall | Precision and recall are commonly used metrics for evaluating the performance of a machine learning model in binary classification problems, where precision measures the proportion of true positive predictions out of all positive predictions, and recall measures the proportion of true positive predictions out of all true positives. |
| 38 | Principal component analysis | Principal component analysis is a technique used for reducing the dimensionality of a dataset, by transforming the data into a new coordinate system that captures the most variance in the data. |
| 39 | Reinforcement learning | Reinforcement learning is a type of machine learning algorithm inspired by behavioral psychology, where an agent learns to take actions in an environment to maximize a reward signal. |
| 40 | Regularization | Regularization is a technique used in machine learning to prevent overfitting by adding a penalty term to the loss function, which discourages the model from fitting to noise or irrelevant features. |
| 41 | Regression | Regression is a type of machine learning problem where the goal is to predict a continuous output variable, such as a numerical value. |
| 42 | Reinforcement learning | Reinforcement learning is a type of machine learning algorithm inspired by behavioral psychology, where an agent learns to take actions in an environment to maximize a reward signal. |
| 43 | Sampling | Sampling is the process of selecting a subset of data from a larger dataset, often used to train or validate machine learning models. |
| 44 | Semi-supervised learning | Semi-supervised learning is a type of machine learning where only a small portion of the training data is labeled, and the model must also learn from the unlabeled data to improve its performance. |
| 45 | Singular value decomposition | Singular value decomposition is a technique used for decomposing a matrix into its singular values and vectors, often used in data compression or dimensionality reduction. |
| 46 | Softmax function | Softmax function is a mathematical function used in machine learning for converting a vector of scores or logits into a probability distribution over multiple classes. |
| 47 | Support vector machine | Support vector machine is a type of machine learning algorithm used for classification or regression, where the goal is to find the hyperplane that maximally separates the data into different classes or predicts the output variable. |
| 48 | Tensor | A tensor is a multi-dimensional array used for representing data in machine learning, often used as inputs or outputs of neural networks. |
| 49 | Time series analysis | Time series analysis is a branch of statistics and machine learning that deals with data collected over time, often used for forecasting or modeling temporal dependencies. |
| 50 | Transfer learning | Transfer learning is a technique used in machine learning where a pre-trained model is used as a starting point for a new, related task, often with less data or computation. |
| 51 | Unsupervised learning | Unsupervised learning is a type of machine learning where the goal is to find patterns or structure in the data, without explicit supervision or labels. |
| 52 | Variational autoencoder | Variational autoencoder is a type of generative model used for learning latent representations of high-dimensional data, by combining neural networks with probabilistic inference. |
| 53 | Weight initialization | Weight initialization is the process of setting the initial values of the weights in a neural network, often used to prevent vanishing or exploding gradients during training. |
| 54 | Zero-shot learning | Zero-shot learning is a type of machine learning where the model is able to recognize and classify new objects or categories that were not seen during training, by leveraging existing knowledge or attributes. |
| # | Term | Definition |
|---|---|---|
| 55 | Active learning | Active learning is a type of machine learning where the model is able to select which samples to acquire or label for training, often used to improve efficiency or reduce labeling costs. |
| 56 | Adversarial example | An adversarial example is a small, intentionally crafted perturbation to an input that causes a machine learning model to misclassify the output. |
| 57 | Adversarial training | Adversarial training is a technique used in machine learning to improve the robustness of a model against adversarial examples, by augmenting the training data with examples that are intentionally designed to fool the model. |
| 58 | Attention mechanism | Attention mechanism is a component often used in neural networks for processing sequential or variable-length inputs, by selectively focusing on the most relevant parts of the input. |
| 59 | Autoencoder | An autoencoder is a type of neural network used for unsupervised learning, where the model is trained to encode and decode the input data, often used for data compression or anomaly detection. |
| 60 | Backpropagation | Backpropagation is a mathematical algorithm used for training neural networks, where the gradients of the loss function with respect to the model parameters are propagated backwards from the output to the input. |
| 61 | Bayesian optimization | Bayesian optimization is a technique used for hyperparameter tuning in machine learning, by building a probabilistic model of the performance of different hyperparameters and iteratively selecting the most promising ones to evaluate. |
| 62 | Bias-variance tradeoff | The bias-variance tradeoff is a fundamental concept in machine learning, which refers to the tradeoff between the ability of a model to fit the training data well (low bias) and its ability to generalize to new data (low variance). |
| 63 | Capsule network | Capsule network is a type of neural network architecture proposed for image recognition, which uses capsules as the basic building blocks of the network to model hierarchical relationships between different parts of an object. |
| 64 | Clustering | Clustering is a type of unsupervised learning where the goal is to group similar data points into clusters or categories based on their similarity or proximity. |
| 65 | Convolutional neural network | Convolutional neural network is a type of neural network architecture often used for image recognition, which applies convolutional filters to the input data to extract local features and then aggregates them through pooling layers. |
| 66 | Cross-validation | Cross-validation is a technique used in machine learning for estimating the performance of a model on new, unseen data, by splitting the dataset into multiple folds and using them for training and testing. |
| 67 | Data augmentation | Data augmentation is a technique used in machine learning for artificially increasing the size of a dataset, by applying random transformations or perturbations to the input data. |
| 68 | Decision tree | A decision tree is a type of machine learning model used for classification or regression, which recursively splits the data into subsets based on the values of different features, and then predicts the output based on the majority or mean value of the samples in each leaf. |
| 69 | Deep learning | Deep learning is a subset of machine learning that uses neural networks with multiple layers to learn hierarchical representations of the input data. |
| 70 | Dropout | Dropout is a regularization technique used in neural networks, where some of the neurons are randomly dropped out or deactivated during training, to prevent overfitting and encourage the network to learn more robust representations. |
| # | Term | Definition |
|---|---|---|
| 71 | Embedding | An embedding is a low-dimensional representation of a high-dimensional object, often used in natural language processing for mapping words or sentences to a continuous vector space. |
| 72 | Ensemble learning | Ensemble learning is a technique used in machine learning for combining multiple models into a single, more accurate model, often used to improve the performance or robustness of the system. |
| 73 | Feature engineering | Feature engineering is the process of selecting or creating relevant features from the input data to improve the performance of a machine learning model, often used to extract domain-specific knowledge or reduce the dimensionality of the input. |
| 74 | Federated learning | Federated learning is a type of distributed learning where the model is trained on multiple decentralized devices, and the updates are aggregated without sharing the raw data, often used for privacy-preserving or resource-constrained scenarios. |
| 75 | Generative adversarial network | Generative adversarial network is a type of neural network architecture used for generating synthetic data, which consists of two parts: a generator that generates the samples and a discriminator that tries to distinguish them from the real data. |
| 76 | Gradient descent | Gradient descent is a first-order optimization algorithm used for finding the optimal values of the model parameters that minimize the loss function, by iteratively updating the parameters in the direction of the negative gradient. |
| 77 | Hyperparameter | A hyperparameter is a parameter that is set before training a machine learning model, which controls the behavior of the algorithm or the complexity of the model, and needs to be tuned based on the performance on the validation data. |
| 78 | Imbalanced data | Imbalanced data is a type of dataset where the number of samples in different classes or categories is significantly different, often causing problems for machine learning models that are biased towards the majority class. |
| 79 | Inference | Inference is the process of using a trained machine learning model to make predictions or decisions on new, unseen data. |
| 80 | Instance-based learning | Instance-based learning is a type of machine learning where the model is trained to store the training data and uses it for making predictions on new, similar data, often used for classification or regression. |
| 81 | K-nearest neighbors | K-nearest neighbors is a type of instance-based learning algorithm used for classification or regression, which predicts the output based on the majority or mean value of the k closest samples in the training data, often used for simple or non-parametric models. |
| 82 | Label | A label is a categorical or numerical value that represents the output or target variable in a machine learning task, often used for supervised learning. |
| 83 | Learning rate | Learning rate is a hyperparameter used in optimization algorithms, which controls the step size of the parameter updates during training, and needs to be set based on the characteristics of the problem and the model. |
| 84 | Linear regression | Linear regression is a type of machine learning model used for regression, which models the output as a linear combination of the input features and a bias term, often used for simple or interpretable models. |
| 85 | Loss function | A loss function is a function used to measure the difference between the predicted output of a machine learning model and the true output, often used for training and evaluating the performance of the model. |
| 86 | Machine learning | Machine learning is a field of artificial intelligence that focuses on developing algorithms and models that can learn from data and make predictions or decisions, without being explicitly programmed. |
| # | Term | Definition |
|---|---|---|
| 87 | Multi-task learning | Multi-task learning is a type of machine learning where the model is trained to perform multiple related tasks simultaneously, often sharing the learned representations or features. |
| 88 | Neural network | A neural network is a type of machine learning model that is inspired by the structure and function of biological neurons, consisting of multiple layers of interconnected nodes or units that process the input data and generate the output. |
| 89 | Object detection | Object detection is a computer vision task of detecting and localizing objects of interest in an image or a video, often using deep learning models or feature-based methods. |
| 90 | Overfitting | Overfitting is a common problem in machine learning where the model is too complex or over-parameterized and fits the training data too well, resulting in poor performance on the new, unseen data. |
| 91 | Preprocessing | Preprocessing is a step in data preparation where the raw data is transformed, cleaned, or normalized to make it suitable for machine learning models, often including feature scaling, dimensionality reduction, or data augmentation. |
| 92 | Principal component analysis | Principal component analysis is a technique for dimensionality reduction that finds a linear transformation of the input data that maximizes the variance or information content, often used to reduce the complexity or noise of the data. |
| 93 | Reinforcement learning | Reinforcement learning is a type of machine learning where the agent learns to take actions in an environment to maximize a cumulative reward signal, often used for decision-making or game playing. |
| 94 | Regression | Regression is a type of machine learning task where the model predicts a continuous numerical output based on the input features, often used for forecasting or modeling. |
| 95 | Regularization | Regularization is a technique used in machine learning to prevent overfitting by adding a penalty term to the loss function that encourages the model to have smaller or sparser parameter values, often used to improve the generalization performance. |
| 96 | Semi-supervised learning | Semi-supervised learning is a type of machine learning where the model is trained on a combination of labeled and unlabeled data, often used to leverage the unlabeled data for improving the performance or reducing the annotation cost. |
| 97 | Support vector machine | Support vector machine is a type of machine learning model used for classification or regression, which finds a hyperplane that separates the different classes or predicts the output based on the closest samples, often used for binary classification or non-linear models. |
| 98 | Transfer learning | Transfer learning is a technique used in machine learning where the knowledge or parameters learned from one task or domain are transferred or adapted to another related task or domain, often used to improve the performance or reduce the training cost. |
| 99 | Unsupervised learning | Unsupervised learning is a type of machine learning where the model is trained on unlabeled data without any explicit output or target variable, often used for clustering, dimensionality reduction, or feature learning. |
| 100 | Validation | Validation is a step in machine learning where the performance of the model is evaluated on a held-out validation set or through cross-validation, often used to tune the hyperparameters or compare the performance of different models. |
