AI Assignment – Detailed Explanation of Python
Codes
Question 1: KL-Divergence Demonstration
The first program demonstrates KL-Divergence between two probability distributions using Python
libraries such as NumPy, SciPy, and Matplotlib. 1. Two normal distributions are defined: - P(x):
Standard Normal Distribution (mean=0, std=1) - Q(x): Shifted and wider Normal Distribution
(mean=2, std=1.5) 2. KL-Divergence Formula Used: D(P||Q) = Σ P(x) * log(P(x)/Q(x)) 3. The
function kl_divergence(p, q) computes the divergence while avoiding division-by-zero errors using
NumPy's conditional operations. 4. Results: KL Divergence P||Q = 50.7780 KL Divergence Q||P =
110.8657 This shows that KL-Divergence is not symmetric. D(P||Q) ≠ D(Q||P). The graphical plot
visually compares both distributions and displays divergence values on the figure.
