Lihan Lian

Lihan Lian

Robotics enthusiast. Passionate about optimal control and deep learning.

Implementation Details of Cartoon-VAE-Diffusion

1 minute read

Published:

Variational Autoencoder (VAE) learns to compress data into a latent Gaussian space and reconstruct it in a single shot. Denoising Diffusion Probabilistic Model (DDPM) tackles the same evidence-lower-bound objective from another direction: it begins with pure noise and iteratively denoise through hundreds of steps, exchanging speed for high-fidelity, stable synthesis. Both frameworks connect random noise to data, yet VAE rely on an explicit encoder–decoder pair, whereas DDPM use a learned Markov chain that inverts a forward noising process. This blog traces the progression from VAE to DDPM, clarifying their shared principles, with code examples available at this repository.

Problem Formulation for Image Generation Model

We posit that every image \(x \in \mathbb{R}^n\) is generated by first sampling a low-dimensional latent \(z \in \mathbb{R}^d\) and then “decoding” it into pixel-space:

  • Prior on latents:

    \[P_\theta(z)\]

    (usually \(\mathcal{N}(0,I)\), no dependence on \(x\)).

  • Decoder likelihood (a Gaussian around a neural-net mean):

    \[P_\theta(x \mid z) \;=\; \mathcal{N}\bigl(x;\;G_\theta(z),\;\sigma^2 I\bigr),\]

    where \(G_\theta(z)\) is the deterministic network output (the mean).

    From these we get the joint,

    \[P_\theta(x,z) \;=\; P_\theta(z)\,P_\theta(x\mid z),\]

    and the marginal (aka evidence),

    \[P_\theta(x) \;=\; \int P_\theta(x,z)\,dz \;=\; \int P_\theta(z)\,P_\theta(x\mid z)\,dz.\]
    • \(P_\theta(z)\): “How we expect latents to be distributed, before seeing any image.”
    • \(P_\theta(x\mid z)\): “If the latent was \(z\), how likely is image \(x\)?”
    • \(P_\theta(x,z)\): joint chance of drawing \(z\) then generating \(x\).
    • \(P_\theta(x)\): overall likelihood of observing image \(x\) under our model.

Variational Autoencoders (VAE)

  • Evidence Lower Bound (ELBO)

  • Final Objective Function

  • Code Example

    VAE sampling result. Trained with z_dim = 512, epochs = 100.

Diffusion Models

DDPM

  • Results

References

  1. Variational Autoencoder - Model, ELBO, loss function and maths explained easily!
  2. Understanding Variational Autoencoders (VAEs)
  3. The Breakthrough Behind Modern AI Image Generators - Diffusion Models Part 1
  4. Denoising Diffusion Probabilistic Models (DDPM Paper)
  5. Denoising Diffusion Implicit Models (DDIM Paper)
  6. Diffusion Model Paper Explanation, PyTorch Implementation Walk Through and corresponding github repo
  7. diffusion-DDPM-pytorch & diffusion-DDIM-pytorch
  8. An Optimal Control Perspective on Diffusion-Based Generative Modeling & SDE/ODE Interpretation of Diffusion Model

You May Also Enjoy

Taxonomy of Robotic Manipulation: Inverse Kinematics (IK), Operational Space Control (OSC), Impedance Control, Riemannian Motion Policy (RMP) and Geometric Fabrics Permalink

less than 1 minute read

Published:

Inverse Kinematics (IK) converts a desired end-effector pose into joint angles and is fundamental for basic motion planning. Operational Space Control (OSC) layers dynamics on top of IK, commanding Cartesian forces/velocities so the robot can track trajectories while rejecting disturbances. Impedance Control then shapes the robot’s apparent mass-spring-damper behavior, enabling compliant contact and human-robot interaction. Riemannian Motion Policy (RMP) generalizes impedance ideas across multiple task maps, blending goal-reaching, joint limits, and obstacle avoidance on curved manifolds. Finally, Geometric Fabrics build on RMP to sculpt global energy landscapes, yielding provably safe navigation around complex environments. This post aims to provide brief explanation for each concepts.

Shed Some Light on Proximal Policy Optimization (PPO) and Its Application Permalink

15 minute read

Published:

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm that refines policy gradient methods like REINFORCE using importance sampling and a clipped surrogate objective to stabilize updates. PPO-Penalty explicitly penalizes KL divergence in the objective function, and PPO-Clip instead uses clipping to prevent large policy updates. In many robotics tasks, PPO is first used to train a base policy (potentially with privileged information). Then, a deployable controller is learned from this base policy using imitation learning, distillation, or other techniques. This blog explores PPO’s core principle, with code available at this repository.

From Q-Learning to Deep Q-Learning and Deep Deterministic Policy Gradient (DDPG) Permalink

16 minute read

Published:

Q-learning, an off-policy reinforcement learning algorithm, uses the Bellman equation to iteratively update state-action values, helping an agent determine the best actions to maximize cumulative rewards. Deep Q-learning improves upon Q-learning by leveraging deep Q network (DQN) to approximate Q-values, enabling it to handle continuous state spaces but it is still only suitable for discrete action spaces. Further advancement, Deep Deterministic Policy Gradient (DDPG), combines Q-learning’s principles with policy gradients, making it also suitable for continuous action spaces. This blog starts by discussing the basic components of reinforcement learning and gradually explore how Q-learning evolves into DQN and DDPG, with application for solving the cartpole environment in Isaac Gym simulator. Corresponding code can be found at this repository.

Dwell on Differential Dynamic Programming (DDP) and Iterative Linear Quadratic Regulator (iLQR) Permalink

6 minute read

Published:

Although optimal control and reinforcement learning appear to be distinct field, they are, in fact closely related. Differential Dynamic Programming (DDP) and Iterative Linear Quadratic Regulator (iLQR), two powerful algorithms commonly utilized in trajectory optimizations, exemplify how model-based reinforcement learning can bridge the gap between these domains. This blog begins by discussing the fundational principles, including Newton’s method and Bellman Equation. It then delves into the specifics of the DDP and iLQR algorithms, illustrating their application through the classical problem of double pendulum swing-up control.