( xm x +1
AM41PB Probabilistic Modelling – Coursework
(deadline – 8/4/2022)
A total of 10% will be given for presentation and quality of the exposition. The submission should be readable, well organised and professionally looking.
1.Bayesian approach – The Pareto distribution is a power-law-like probability distribution that describes the distribution of wealth in a society, tting the trend that a large portion of wealth is held by a small fraction of the population. It takes the form
x xm
fX (x) =
0 x < xm
where it provides the probability of x above a threshold xm 0.
On BlackBoard, under the section assignments, please nd a dataset income distribution by percentile of the UK population for the tax years 2010-11 to 2018-9. Use the last digit of your student number to choose the tax year you will be working on (0 ! 2010 11; 1 ! 2011 2;:::;8;9 ! 2018 9).
(a) Show that the Pareto distribution is indeed a probability distribution.
[3 marks]
(b)Plot the data as the percentage of the population (y-axis) with income above certain values (x-axis); to simplify the presentation present the income in thousands of pounds.
[5 marks]
(c)Assuming the data is i.i.d. and represented by the Pareto distribution, nd the maximum likelihood expression for the parameter ML. Obtain the numerical value of ML given data D and xm = $15; 000. Specify the data you are working with and the number of data points used.
[6 marks]
(d) For data set D = fx1; : : : ; xN g, show that the Gamma probability distribution function
1 for > 0;
p( j ; ) = ( 0 ) e
for 0:
is the conjugate prior for the Pareto distribution. Derive the corresponding posterior
p( jD).
[8 marks]
(e) Assuming the data is represented by the Pareto distribution, nd the Maximum A
Posteriori expression of the parameter MAP. Use the prior values = 1 and = 1 to
obtain the numerical value of MAP given data D. [6 marks]
(f) Use the values obtained for the parameters ML and MAP to compare the estimated
percentage of the population with income above $20; 000. [7 marks]
(35 marks)
2. Decision theory- Large Diamonds (D) are exponentially rare and their weight follows the exponential distribution
p(gjD) = De Dg; g 0 ;
where g is the weight of a diamond in grams and D is a constant. In the same mine there are also White Sapphire (WS) crystals that look like diamonds but are very regular in weight, following a Gaussian distribution
p(gjWS) = 1 exp (g S)2
(2 S2)1=2 2 S2
of mean S and variance S2. The average size of diamonds D is much smaller than that of White Sapphire D S. Additionally, diamonds are much rarer, so that p(WS) p(D).
The mine owner wants to automate the sorting of Diamonds vs White Sapphire by weight, setting a threshold weight gT .
(a)Write an expression for the misclassi cation error of Diamonds as White Sapphire p(DjWS) and of White Sapphire as Diamonds p(WSjD) as functions of the threshold
weight gT . [5 marks]
(b) Minimise the total misclassi cation error with respect to the threshold weight gT . What
is the expression obtained for gT (you can use a simpli ed notation)? [8 marks]
(c)Calculate the value of gT for the parameters D = 1; S = 5; S2 = 1 and prior probabil-ities p(WS) = 0:8 and p(D) = 0:2. Sketch the two distributions, point to the threshold value gT and to the area that contributes to the misclassi cation error of Diamonds as White Sapphire.
[8 marks]
(d)The mine is loosing money and the owner realises that he has not taken into account that large diamonds are exponentially more expensive and the misclassi cation cost p(WSjD) with respect to p(DjWS) (which for simplicity we will attribute cost 1 to) is
CostWSjD = e C g :
Assume C < D. Find the threshold weight gT expression in this case (not numerical value). [9 marks] (30 marks) 3.Graphical models - James is concerned that he has contracted a new disease (J = 1). He shows suspicious symptoms: he has a sore throat (S = 1) and an elevated fever (F = 1). He has been in touch only with Ruth (R) over the last week and thinks she has infected him. The following probabilities are given: •the prior probability of Ruth having the disease is P (R = 1) = 1=4; •Ruth spent a lot of time with James recently and is likely to have infected him if she is infectious P (J = 1jR = 1) = 3=4; clearly, she has not infected him if she does not have the disease; •a person (say James) who has contracted the disease, always experiences a sore throat P (S = 1jJ = 1) = 1 but has elevated fever only in half of the cases P (F = 1jJ = 1) = 1=2; not having contracted the disease, James can still have elevated fever P (F = 1jJ = 0) = 1=4 and a sore throat P (S = 1jJ = 0) = 1=4 (a) Plot the directed acyclic graphical model that represents the problem. [2 marks] (b) Write an expression for the joint probability of all variables in the problem as well as the probabilities given in the question. [3 marks] (c) What is the probability that Ruth has the disease given that James has a sore throat and elevated fever P (R = 1jS = 1; F = 1)? [8 marks] (d) After a while, James drinks some water and realises that his sore throat was a result of not drinking enough so that S = 0 . What is the probability that Ruth has the disease now? [7 marks] (e) Later in the day James is again unclear whether he has a sore throat or not. A lateral ow test (T ) shows that Ruth is positive for the disease (T = 1). However, these tests are inaccurate and the probability of a person having the disease testing positive is 3=4, but also healthy individuals may test positive with probability 1=4. Given that James has elevated fever (F = 1), what is the probability that he is ill? [5 marks] Note that all probabilities are fractions, it is easier not to convert them to decimals. (25 marks)