butools.fitting.EmpiricalRelativeEntropy¶

butools.fitting.EmpiricalRelativeEntropy()¶

Matlab:	re = EmpiricalRelativeEntropy(f1, f2, intBounds)
Mathematica:	re = EmpiricalRelativeEntropy[f1, f2, intBounds]
Python/Numpy:	re = EmpiricalRelativeEntropy(f1, f2, intBounds)

Returns the relative entropy (aka Kullback–Leibler divergence) of two continuous functions given by samples and the bounds of the corresponding intervalls.

This function can be used to characterize the distance between two density functions, distribution functions, etc.

Parameters:	f1 : vector, length M Samples of the first continuous function f2 : vector, length M Samples of the second continuous function intBounds : vector, length M+1 The bounds of the intervals. The ith sample corresponds to the interval (intbounds(i),intbounds(i+1))
Returns:	re : double The relative entropy

Examples

For Matlab:

>>> tr = dlmread('/home/gabor/github/butools/test/data/bctrace.iat');
>>> intBounds = linspace(0, MarginalMomentsFromTrace(tr, 1)*4, 50);
>>> [pdfTrX, pdfTrY] = PdfFromTrace(tr, intBounds);
>>> [cdfTrX, cdfTrY] = CdfFromTrace(tr);
>>> step = ceil(length(tr)/2000);
>>> cdfTrX = cdfTrX(1:step:length(tr));
>>> cdfTrY = cdfTrY(1:step:length(tr));
>>> [alpha, A] = APHFrom3Moments(MarginalMomentsFromTrace(tr, 3));
>>> [pdfPHX, pdfPHY] = IntervalPdfFromPH(alpha, A, intBounds);
>>> cdfPHY = CdfFromPH(alpha, A, cdfTrX);
>>> rePdf = EmpiricalRelativeEntropy(pdfTrY, pdfPHY, intBounds);
>>> disp(rePdf);
      0.43241
>>> reCdf = EmpiricalRelativeEntropy(cdfTrY(1:end-1), cdfPHY(1:end-1), cdfTrX);
>>> disp(reCdf);
   0.00040609

For Mathematica:

>>> tr = Flatten[Import["/home/gabor/github/butools/test/data/bctrace.iat","CSV"]];
>>> intBounds = Array[# &, 50, {0, MarginalMomentsFromTrace[tr, 1][[1]]*4}];
>>> {pdfTrX, pdfTrY} = PdfFromTrace[tr, intBounds];
>>> {cdfTrX, cdfTrY} = CdfFromTrace[tr];
>>> step = Ceiling[Length[tr]/2000];
>>> cdfTrX = cdfTrX[[1;;Length[tr];;step]];
>>> cdfTrY = cdfTrY[[1;;Length[tr];;step]];
>>> {alpha, A} = APHFrom3Moments[MarginalMomentsFromTrace[tr, 3]];
>>> {pdfPHX, pdfPHY} = IntervalPdfFromPH[alpha, A, intBounds];
>>> cdfPHY = CdfFromPH[alpha, A, cdfTrX];
>>> rePdf = EmpiricalRelativeEntropy[pdfTrY, pdfPHY, intBounds];
>>> Print[rePdf];
0.4324143797771531
>>> reCdf = EmpiricalRelativeEntropy[cdfTrY[[1;;-2]], cdfPHY[[1;;-2]], cdfTrX];
>>> Print[reCdf];
0.00040609487431599847

For Python/Numpy:

>>> tr = np.loadtxt("/home/gabor/github/butools/test/data/bctrace.iat")
>>> intBounds = np.linspace(0, MarginalMomentsFromTrace(tr, 1)[0]*4, 50)
>>> pdfTrX, pdfTrY = PdfFromTrace(tr, intBounds)
>>> cdfTrX, cdfTrY = CdfFromTrace(tr)
>>> step = math.ceil(Length(tr)/2000)
>>> cdfTrX = cdfTrX[0:Length(tr):step]
>>> cdfTrY = cdfTrY[0:Length(tr):step]
>>> alpha, A = APHFrom3Moments(MarginalMomentsFromTrace(tr, 3))
>>> pdfPHX, pdfPHY = IntervalPdfFromPH(alpha, A, intBounds)
>>> cdfPHY = CdfFromPH(alpha, A, cdfTrX)
>>> rePdf = EmpiricalRelativeEntropy(pdfTrY, pdfPHY, intBounds)
>>> print(rePdf)
0.432414379777
>>> reCdf = EmpiricalRelativeEntropy(cdfTrY[0:-1], cdfPHY[0:-1], cdfTrX)
>>> print(reCdf)
0.000406094874315