The Importance Sampling-Computational Physics-Lecture Slides, Slides of Computational Physics

Download The Importance Sampling-Computational Physics-Lecture Slides and more Slides Computational Physics in PDF only on Docsity!

Stochastic Methods

Topic:

The Importance Sampling

Dr. Nasir M Mirza

Computational Physics Computational Physics

Email: [email protected]

Docsity.com

Lecture: Importance Sampling

Monte Carlo Methods

Monte Carlo Methods

Docsity.com

The quantity G

N

is an estimate for G and the fundamental theorem of

Monte Carlo guarantees that

G

G

N

=

<

Since G

N

is an estimate for G, we can say

.

)

(

)

(

2

2

2

1

G

dX

X

f

X

g

−

=

∫

σ

.

error

G

G

N

=

If the variance exists, the error is generally given as

,

1

N

error

σ

ε ≅

=

Where,

The Direct Method for Integrals

Docsity.com

There are three major classes of techniques that are used to reduce the variance in Monte Carlo calculations. 1.

Importance Sampling can be used to increases the likelihood ofsampling variable where the function is large or rapidly varying.

The expected value of the random variable can be used ratherthan the variable itself.

Correlations between succeeding samples may be exploited tothe advantage.

In control variates, an easily evaluated approximation to theintegrand is used to reduce the variance.

How to Reduce Variance

IMPORTANCE SAMPLING may be the answer to this all !!!

Docsity.com

Then the variance of G when h(X) is used becomes And we know that G

2

is fixed, so we aim at finding a proper h(X) that

will minimized the quantity

The h(X) function that satisfies these criterion, will be the best choice. Lagrange multiplier method is used to deduce the function. OR a hit an trial method can also be employed.

.

)

(

)

(

)

(

)

(

}

var{

2

2

2

2

G

dX

X

h

X

h

X

f

X

g

G

−

⎤ ⎥ ⎦

⎡ ⎢ ⎣

=

∫

∫

dX

X

h

X

h

X

f

X

g

2

2

2

Importance Sampling

Docsity.com

A Monte Carlo method to evaluate the integral would be to sample a series of X values from h(X) and then construct the following sum:

∑

=

=

N

i

i

i

i

N

X

h

X f X g N G

1

)

(

) ( ) ( 1 ~

This clearly corresponds to the minimum variance calculation. As a first example, let us consider one integral to be estimated.

Importance Sampling

Docsity.com

Then we can let

2

x

x

h

The estimator for G is now

)

1

(

)

2

/

cos(

2 3

)

(

)

(

~

2

x

x

x

h

x

g

g

−

=

=

π

The variance associated with a single sample is about 0.000990 and

Example 1: Importance Samplingit is about 100 times smaller with this choice of sampling function.

Docsity.com

%

Program name; fg-isampling.m

%

Sampling from a Function g(x) = 1/x and

%

find area under f(x)g(x), where,

%

f(x) = x^2 in interval [0, 1]

const = pi/2; c1 = 1./const; x(1)=0; N(1) = 1000; exact(1)= c1; area(1)=c1; Max_bins = 40; nmax = 40;

for jj=1: nmax %%% loop for change in N

rand('state', 0) % initialize

jj N(jj+1) = N(jj) + 1000;

sum1 = 0.0; sum2 = 0.0;

for i=1:

%%%%% loop begins for 5 bins

rand('state', i) % initialize icount = 0; sum = 0.0;

for k = 1:N(jj+1)

x = rand; y = rand; % rejection technique % g = cos(const*x); % simple way % sum = sum + g; icount = N(jj+1); % simple way

gg = 2cos(constx)/(3(1-xx)); %importance sampling if(y <=(1-x^2)) sum = sum + gg;

%importance sampling

icount = icount + 1; end

%importance sampling

end

area(i) = sum/icount; sum1 = sum1 + area(i); sum2 = sum2 + area(i)^2;

end %%%%%%%%% loop ends for 5 bins

avarea(jj+1) = sum1/10.0 ; ava2 = sum2/10.0; var = ava2 - avarea(jj+1)^2; stdev(jj+1) = sqrt(var) ;

exact(jj+1) = c1;

axis([1 50000 0.63 0.65])

errorbar(N,avarea,stdev) plot(N, exact, 'r:.') hold on m(jj)= getframe;

end %%% loop for change in N

MATLAB Program for

Importance

Sampling

Example 1:

Docsity.com

N

Av. Area

St. Deviation

ExactValue

2000

0.

8.4476E-

0.

10000

0.

2.5389E-

0.

15000

0.

2.4953E-

0.

20000

0.

2.1742E-

0.

25000

0.

1.6991E-

0.

30000

0.

1.5901E-

0.

35000

0.

1.7786E-

0.

40000

0.

1.6464E-

0.

N

Av. Area

St.

Deviation

Exact Value

2000

0.

0.

0.

10000

0.

0.

0.

15000

0.

0.

0.

20000

0.

0.

0.

25000

0.

0.

0.

30000

0.

0.

0.

35000

0.

0.

0.

40000

0.

0.

0.

Results using simple MC method

Results using Importance Sampling

Example 1: Results of MATLAB Program

Docsity.com

Another example on importance sampling:

The variance associated with this procedure is

Again a straightforward Monte Carlo method would be to sample xuniformly on (0, 1) and form the estimator:

dx

x

G

∫

1 0

2

2

1

1

x

g

−

=

050

.

0

}

var{

1

=

g

To improve the calculations and reduce the variance, the integrandcan be expanded in a power series about its maximum:

L

−

=

2

2 1

1

x

g

Example 2 Find this integral:

: Importance Sampling

Docsity.com

Numerically minimizing the variance with respect to

β

gives the

variance{g} = 0.0029 when

β

If we choose b to be ½ as implied by the power series, the minimumvariance is not achieved. The var{g} = 0.011, but is still substantiallylower than the variance from simple MC.

Reduce Variance

Just introduction of importance sampling scheme and a good choiceof beta can lower the variance dramatically. An experimental method of obtaining the parameter like

β

in an

importance function is to try MC calculations with different choices of β

and estimate the value that minimizes the variance of g/h.

Example 2

: Importance Sampling

Docsity.com

17

Choice of Importance Function

In a global sense, we want an importance function that matches the general behavior of the integrand at its maximum but also is similarover the whole range of integration. •

In the figure, g is the integrandwe wish to approximate.

The h

1

is importance function

derived from the Taylor seriesexpansion.

It is always greater than g.

The h

2

is a better importance

function because it describe gbetter over the entire range ofintegration.

5

10

15

26 24 22 20 18 16 14 12

g

h

2

h

1

Example 2

: Importance Sampling

Docsity.com

%

Program name; fg-isampling.m

%

Sampling from a Function f(x)

& find area under f(x)g(x), where,

%

f(x) = 1 in interval [0, 1]

const = pi/4.

; x(1)=0; c1 = const; beta = 0.

N(1) = 1000; exact(1)= const; area(1)=c1; Max_bins = 40; nmax = 40;

for jj=1: nmax %%% loop for change in N

rand('state', 0) % initialize

jj N(jj+1) = N(jj) + 1000;

sum1 = 0.0; sum2 = 0.0;

for i=1:

%%%%% loop begins for 10 bins

rand('state', i) % initialize icount = 0; sum = 0.0;

for k = 1:N(jj+1)

x = rand; y = rand ;

g = sqrt(1.0-x*x);

% direct sampling

sum = sum + g; icount = N(jj+1);

% direct sampling

% xh = (1.0 - betaxx)/(1-0.33333*beta); %importance sampling %

gg = sqrt(1.0-x*x)/xh;

%importance sampling

%

if(y <= xh ) sum = sum + gg;

%importance sampling

% icount = icount + 1; end

%importance sampling

end

area(i) = sum/icount; sum1 = sum1 + area(i); sum2 = sum2 + area(i)^2;

end %%%%%%%%% loop ends for 5 bins

avarea(jj+1) = sum1/10.0 ; ava2 = sum2/10.0; var = ava2 - avarea(jj+1)^2; stdev(jj+1) = sqrt(var) ;

exact(jj+1) = c1;

axis([1 50000 0.78 0.8])

errorbar(N,avarea,stdev) plot(N, exact, 'r:.') hold on m(jj)= getframe;

end %%% loop for change in N

MATLAB

Program for

Importance

Sampling

Example 2:

Docsity.com

Results of MATLAB Program

1 .0 x 1 0

2 .0 x 1 0

3 .0 x 1 0

m in [v a r{g }] a t

β

v a ria n c e {g }

β

Example 2

: Importance Sampling

Docsity.com