https://www.victorexplore.com/Recent content on VictorHugo -- 0.169.0en-usSun, 29 Jun 2025 00:00:00 +0000https://www.victorexplore.com/posts/how-to-live-a-good-life/descion-making-framework-using-probability-theory/Sun, 29 Jun 2025 00:00:00 +0000https://www.victorexplore.com/posts/how-to-live-a-good-life/descion-making-framework-using-probability-theory/A framework for making decisions under uncertainty with probability theory: actions, outcomes and how to choose between them.https://www.victorexplore.com/posts/how-to-live-a-good-life/you-are-rl-agent/Sun, 25 May 2025 00:00:00 +0000https://www.victorexplore.com/posts/how-to-live-a-good-life/you-are-rl-agent/Human behaviour modelled as a reinforcement learning problem: the world as the environment, the senses as observations and the brain as the policy.https://www.victorexplore.com/posts/how-x-algo-works/Sat, 24 May 2025 00:00:00 +0000https://www.victorexplore.com/posts/how-x-algo-works/A technical breakdown of how X's recommendation algorithm actually works, based on the open-source code. Not speculation—just what the code says.https://www.victorexplore.com/posts/dump/rag/Sat, 24 May 2025 00:00:00 +0000https://www.victorexplore.com/posts/dump/rag/What retrieval augmented generation is, which limits of large language models it addresses, and how it works.https://www.victorexplore.com/posts/reinforcement-learning/0-cheatsheat/Fri, 09 May 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/0-cheatsheat/An overview of reinforcement learning: the basic terms, the Markov decision process, the interaction loop and the aim, in maths.https://www.victorexplore.com/posts/how-reinforcement-learning-works-in-unity/Sat, 01 Mar 2025 00:00:00 +0000https://www.victorexplore.com/posts/how-reinforcement-learning-works-in-unity/How reinforcement learning works in Unity: the packages, how Unity communicates with Python, and what to know before PPO.https://www.victorexplore.com/posts/dump/ai-roadmap/Sat, 01 Feb 2025 00:00:00 +0000https://www.victorexplore.com/posts/dump/ai-roadmap/The step-by-step roadmap I made for my younger brother to learn AI: maths, coding, machine learning and what to do after.https://www.victorexplore.com/posts/dump/virtual-environment/Sat, 01 Feb 2025 00:00:00 +0000https://www.victorexplore.com/posts/dump/virtual-environment/Why Python projects need a virtual environment, built up from what a module and a package are.https://www.victorexplore.com/posts/prnn/33--adaboost/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/33--adaboost/AdaBoost derived from the general boosting form: the problem setting, how the step size is found and the full algorithm.https://www.victorexplore.com/posts/adrl/13---aggregated-posterior-mismatch-/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/13---aggregated-posterior-mismatch-/Why the aggregated posterior of a VAE does not match the prior, and why the EM algorithm does not have this problem.https://www.victorexplore.com/posts/prnn/5---bayes-classifier/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/5---bayes-classifier/The Bayes classifier, its notation, and why it is the best classifier for the 0-1 loss.https://www.victorexplore.com/posts/prnn/19--bias-variance-tradeoff/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/19--bias-variance-tradeoff/How the average squared-error risk splits into bias, variance and irreducible noise, and what the tradeoff means.https://www.victorexplore.com/posts/prnn/31---boosting/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/31---boosting/How boosting trains models one after another, each one correcting the errors of the ones before, with the maths.https://www.victorexplore.com/posts/adrl/cheatsheet/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/cheatsheet/A formula sheet for deep generative models: divergences, common probability distributions and the key results from the notes.https://www.victorexplore.com/posts/adrl/10---conditional-gans/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/10---conditional-gans/How a conditional GAN feeds extra information such as class labels to the generator and the discriminator, with the objective and the training steps.https://www.victorexplore.com/posts/adrl/24---conditional-score-matching/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/24---conditional-score-matching/Score matching with labelled data: classifier guidance and classifier-free guidance.https://www.victorexplore.com/posts/adrl/4---convex-conjugate/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/4---convex-conjugate/The convex conjugate (Fenchel conjugate): definition, properties, and why the conjugate of the conjugate gives the function back.https://www.victorexplore.com/posts/adrl/30---ddim/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/30---ddim/Why DDPM inference needs many sequential steps, how DDIM gives up the Markov assumption, the derivation and how inference works.https://www.victorexplore.com/posts/adrl/27-ddpm-as-sdes/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/27-ddpm-as-sdes/How the DDPM forward process becomes a continuous SDE, how the SDE is reversed, and how the reverse SDE is made discrete again.https://www.victorexplore.com/posts/prnn/28---descion-trees/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/28---descion-trees/How a decision tree splits data, how it is grown with Gini impurity or mean squared error, and how it is pruned.https://www.victorexplore.com/posts/adrl/0---index/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/0---index/A list of my deep learning notes in order: divergences, GANs, VAEs, diffusion models, score matching, SDEs and sequence models.https://www.victorexplore.com/posts/adrl/16---denoising-diffusion-probabilistic-models-ddpm---part-1/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/16---denoising-diffusion-probabilistic-models-ddpm---part-1/DDPM seen as a VAE with many latent variables: notation, the latent posterior and how the ELBO breaks into parts.https://www.victorexplore.com/posts/adrl/17---denoising-diffusion-probabilistic-models-ddpm---part-2/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/17---denoising-diffusion-probabilistic-models-ddpm---part-2/The three parts of the DDPM ELBO explained: the reconstruction term, the prior matching term and the denoising matching term.https://www.victorexplore.com/posts/adrl/18---denoising-diffusion-probabilistic-modelsddpm---part-3/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/18---denoising-diffusion-probabilistic-modelsddpm---part-3/The DDPM reparameterisation, the architecture, and the training and inference steps.https://www.victorexplore.com/posts/adrl/23---difference-between-smld-and-ddpm/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/23---difference-between-smld-and-ddpm/How SMLD (noise conditional score networks) and DDPM differ in the forward process and in inference.https://www.victorexplore.com/posts/adrl/11---domain-adversarial-networks/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/11---domain-adversarial-networks/How domain adversarial networks learn features that work across domains: the feature extractor, the domain discriminator and the training objective.https://www.victorexplore.com/posts/prnn/8---em-algorithm/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/8---em-algorithm/The EM algorithm from latent variable models and variational inference: the ELBO, how it is made tight, and EM for Gaussian mixtures.https://www.victorexplore.com/posts/adrl/3---f-divergence/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/3---f-divergence/F-divergence as a general measure of the difference between two probability distributions, with its main properties.https://www.victorexplore.com/posts/adrl/9--frechet-inception-distance/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/9--frechet-inception-distance/Why generative models cannot be evaluated like discriminative models, and what the Frechet Inception Distance measures.https://www.victorexplore.com/posts/adrl/5---gans/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/5---gans/GANs derived from F-divergence minimisation, and why the network is called adversarial.https://www.victorexplore.com/posts/adrl/1--what-is-the-difference-between-generative-model-and-discriminative-model/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/1--what-is-the-difference-between-generative-model-and-discriminative-model/What a generative model learns, what a discriminative model learns, how the two differ, and what a conditional generative model is.https://www.victorexplore.com/posts/reinforcement-learning/11-feb-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/11-feb-2025/Infinite horizon discounted cost problems: the setting, the optimal cost function, Bellman operators and the Bellman equation.https://www.victorexplore.com/posts/adrl/14.1-info-vae/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/14.1-info-vae/How InfoVAE changes the VAE objective with a mutual information term to fix the aggregated posterior mismatch, and how it is trained.https://www.victorexplore.com/posts/reinforcement-learning/16-jan-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/16-jan-2025/Controlled Markov chains and Markov decision processes: policies, optimal policies and the principle of optimality.https://www.victorexplore.com/posts/reinforcement-learning/1-introduction-to-reinforcement-learning/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/1-introduction-to-reinforcement-learning/The core idea of reinforcement learning, an agent that learns from reward, and the core problems of the field.https://www.victorexplore.com/posts/adrl/36---jepa/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/36---jepa/The JEPA framework: prediction in embedding space, the formulation, training and downstream use.https://www.victorexplore.com/posts/prnn/35---k-means-clustering/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/35---k-means-clustering/K-means as hard clustering, derived from mixture model clustering, with the algorithm.https://www.victorexplore.com/posts/adrl/2---kl-divergence---kullback-leibler-divergence/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/2---kl-divergence---kullback-leibler-divergence/KL divergence with intuition and properties, and why minimising it is the same as maximising likelihood.https://www.victorexplore.com/posts/reinforcement-learning/19-feb-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/19-feb-2025/How to learn the system model from data, and Monte Carlo methods for value estimation: first-visit and every-visit.https://www.victorexplore.com/posts/prnn/9---linear-discriminant-analysis/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/9---linear-discriminant-analysis/Linear discriminant analysis derived from Gaussian class-conditional densities for two classes.https://www.victorexplore.com/posts/prnn/12---linear-regression/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/12---linear-regression/Linear regression from the ideal regressor: its derivation, how to read it, and empirical risk minimisation.https://www.victorexplore.com/posts/prnn/0---index/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/0---index/A list of my machine learning notes in order: classifiers, regression, density estimation, SVMs, neural networks, trees, boosting and clustering.https://www.victorexplore.com/posts/prnn/27---miscellaneous-terms/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/27---miscellaneous-terms/Short definitions of common machine learning terms: epoch, batch size and the variants of gradient descent.https://www.victorexplore.com/posts/reinforcement-learning/2-multi-armed-bandit-problem/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/2-multi-armed-bandit-problem/The multi-armed bandit problem: action-value methods, greedy and epsilon-greedy policies and incremental updates.https://www.victorexplore.com/posts/adrl/6---naive-gans/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/6---naive-gans/The original GAN: derivation from the general F-divergence form, how to read the objective, and how it is trained.https://www.victorexplore.com/posts/prnn/25---neural-network/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/25---neural-network/A neural network defined as a composition of functions, with the common non-linear activation functions.https://www.victorexplore.com/posts/adrl/33---noise-contrastive-estimation/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/33---noise-contrastive-estimation/The idea behind noise contrastive estimation: learn a distribution by learning to tell its samples from noise, with the formulation.https://www.victorexplore.com/posts/prnn/17---parzen-window-estimate/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/17---parzen-window-estimate/The Parzen window (kernel density) estimate: the idea, the uniform and Gaussian kernels, and the algorithm.https://www.victorexplore.com/posts/reinforcement-learning/4-policy-iteration/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/4-policy-iteration/Policy iteration and modified policy iteration: the algorithm, the key proposition and convergence.https://www.victorexplore.com/posts/reinforcement-learning/18-feb-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/18-feb-2025/The optimality conditions of policy iteration in discounted cost problems, and the algorithm step by step.https://www.victorexplore.com/posts/adrl/25--precursor-to-stochastic-differential-equations-sdes/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/25--precursor-to-stochastic-differential-equations-sdes/Why the sampling steps of Langevin dynamics, DDPM and SMLD are discrete versions of a continuous process, with worked examples.https://www.victorexplore.com/posts/reinforcement-learning/25-feb-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/25-feb-2025/Q-learning explained: model-based against model-free methods, the Q-function and the Q-Bellman equation.https://www.victorexplore.com/posts/adrl/20---score-matching-part-1/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/20---score-matching-part-1/Sampling seen as optimisation, the score function, and explicit and implicit score matching.https://www.victorexplore.com/posts/adrl/21---score-matching-part-2/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/21---score-matching-part-2/Why the trace term in implicit score matching is a problem, and how projected and sliced score matching avoid it.https://www.victorexplore.com/posts/adrl/22---score-matching-part-3/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/22---score-matching-part-3/Denoising score matching, the reparameterisation that makes training efficient, and the data coverage problem.https://www.victorexplore.com/posts/adrl/transformers/31.5/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/transformers/31.5/Sequence models from hidden Markov models and RNNs to transformers, with attention heads and multi-head attention.https://www.victorexplore.com/posts/adrl/28---smld-as-sde/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/28---smld-as-sde/How the SMLD forward process becomes a continuous SDE, how the SDE is reversed, and how the reverse SDE is made discrete again.https://www.victorexplore.com/posts/adrl/32-state-space-models/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/32-state-space-models/State space models from the linear form to Mamba: Euler's method, unrolling the equations and how the A matrix is initialised.https://www.victorexplore.com/posts/adrl/26---stochastic-differential-equations-sdes/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/26---stochastic-differential-equations-sdes/An introduction to stochastic differential equations for diffusion models: the formal definition and what Brownian motion is and is not.https://www.victorexplore.com/posts/prnn/21---support-vector-machine---data-linearly-separable/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/21---support-vector-machine---data-linearly-separable/The support vector machine for linearly separable data: the optimisation problem, the dual form, and how to find w and b.https://www.victorexplore.com/posts/prnn/22---svm---dataset-is-not-linearly-separable/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/22---svm---dataset-is-not-linearly-separable/The soft-margin support vector machine for data that is not linearly separable: slack variables, the dual form, and how to find w and b.https://www.victorexplore.com/posts/prnn/23---svm-with-kernel/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/23---svm-with-kernel/The kernel trick and how it lets a support vector machine work in a higher-dimensional space without computing the feature map.https://www.victorexplore.com/posts/reinforcement-learning/20-feb-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/20-feb-2025/TD learning from incomplete episodes: the TD(lambda) algorithm and its special cases TD(0) and TD(1).https://www.victorexplore.com/posts/reinforcement-learning/13-mar-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/13-mar-2025/Temporal difference learning for policy evaluation: the Bellman equation behind it and the approaches to value estimation.https://www.victorexplore.com/posts/reinforcement-learning/30-jan-2025/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/30-jan-2025/Stochastic shortest path problems: expected stage cost, the cost-to-go function and the Bellman operators.https://www.victorexplore.com/posts/adrl/14---vae-as-regularized-autoencoder/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/14---vae-as-regularized-autoencoder/How the VAE objective splits into a reconstruction error and a regulariser, the link to MAP estimation, and beta-VAEs.https://www.victorexplore.com/posts/reinforcement-learning/3-value-iteration/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/reinforcement-learning/3-value-iteration/Value iteration for stochastic shortest path problems: the dynamic programming operator, its properties and the Gauss-Seidel variant.https://www.victorexplore.com/posts/adrl/12---variational-encoding/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/12---variational-encoding/Variational autoencoders from the start: the problem setting, the model, and why and how the reparameterisation trick is used.https://www.victorexplore.com/posts/adrl/8---wasserstein-gans-/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/8---wasserstein-gans-/The perfect discriminator theorem, Wasserstein GANs, what k-Lipschitz means and how to enforce the 1-Lipschitz constraint.https://www.victorexplore.com/posts/adrl/7--wasserstein-metric/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/adrl/7--wasserstein-metric/The Wasserstein distance between two distributions: definition, properties and how it relates to KL divergence.https://www.victorexplore.com/posts/prnn/32---xg-boost/Wed, 01 Jan 2025 00:00:00 +0000https://www.victorexplore.com/posts/prnn/32---xg-boost/How XGBoost builds decision trees one after another, each one correcting the errors of the trees before, step by step.https://www.victorexplore.com/about/Mon, 01 Jan 0001 00:00:00 +0000https://www.victorexplore.com/about/Who Victor is and what this site is for.https://www.victorexplore.com/contact/Mon, 01 Jan 0001 00:00:00 +0000https://www.victorexplore.com/contact/How to reach Victor.https://www.victorexplore.com/lastcig/Mon, 01 Jan 0001 00:00:00 +0000https://www.victorexplore.com/lastcig/LastCig is a quit-smoking app for Android. Track cigarettes, resist cravings and watch your body recover.https://www.victorexplore.com/lastcig/privacy/Mon, 01 Jan 0001 00:00:00 +0000https://www.victorexplore.com/lastcig/privacy/How the LastCig app and the LastCig video publisher handle your data.https://www.victorexplore.com/privacy/Mon, 01 Jan 0001 00:00:00 +0000https://www.victorexplore.com/privacy/What victorexplore.com and the 1000 Cuts game collect, and how ads use cookies.