Essential Maths for ML – Part 3

In my last blog [Essential Maths for ML - Part 2]  we have discussed about addition rule, multiplication rule of probability and conditional probability  . In this blog, we will discuss the Bayes Theorem which plays an important role in most of the machine learning algorithms.

So let’s consider A1, A2, A3, and A4 be the mutually exclusive and exhaustive event of a random experiment. Let B be the common event i.e. the event B is made-up of 4- mutually exclusive and exhaustive events.

 P(B) = P(A1B) + P(A2⋂B) + P(A3⋂B) + P(A4⋂B)

P(B) = Σ P(AiB)…………(1)

We already know from the concept of the conditional probability that

P(A1⋂B) = P(B) * P(A1/B)

P(A1/B) = P(A1⋂B) / P(B)………(2)

Replacing the value of P(B) from the eq2 we can say that

P(A1/B) = P(A1B) / Σ P(AiB)

P(A1/B) = P(A1) * P(B/A1) / Σ P(AiB)

So the Bayes Theorem states that if A1, A2, A3……….An, are n mutually exclusive and exhaustive events with prior probabilities P(A1), P(A2), P(A3)……………P(An) and B be the common event for which the conditional probabilities of the probability of occurrence of B given A1, B given A2 ………….B given An will be P(B/A1), P(B/A2), P(B/A3)………..P(B/An) respectively then the posterior probability of occurrence of A1 given that B has already occurred is given by :

P(A1/B) = P(A1/B) * P(B) / Σ P(Ai) * P(B/Ai)……..i=1 to n

Let’s understand this theorem with an example.

The probabilities that Mr. A, Mr. B and Mr. C will get the promotion are 0.40, 0.35 and 0.25 respectively.  The probabilities that they will introduce new business are 0.10, 0.15, and 0.20 respectively. What is the probability that Mr B introduces a new business by getting the promotion?

·       Let A1 be the event for Mr. A getting the promotion

·       Let A2 be the event for Mr. B getting the promotion

·       Let A3 be the event for Mr. C getting the promotion

·       Let B be the event that a new business was introduced

So according to the above definitions we can say that

P(A1) = 0.40, P(A2) = 0.35, P(A3) = 0.25,

P(B/A1) = 0.10, P(B/A2) = 0.15 and P(B/A3) = 0.20

 

We need to find out the value of P(A2/B)

Event

Ai

Prior Probability

P(Ai)

Conditional Probability P(B/Ai)

Joint Probability P(Ai⋂B) = P(Ai)* P(B/Ai)

Posterior probability P(Ai/B) = P(Ai⋂B) / Σ P(Ai) * P(B/Ai)

A1

0.40

0.10

0.0400

0.0400/0.1425 = 0.2807

A2

0.35

0.15

0.0525

0.0525/0.1425 = 0.3684

A3

0.25

0.20

0.0500

0.0500/0.1425 =0.3509

Sum

1.0

0.45

0.1425

 

 Now our required probability is P(A2/B) = 0.3684

I hope you will find this blog useful and informative.  In my next blog we will continue discussing the mathematical concepts required to understanding the machine learning models.  Please do not forget to follow me on my blog page.

Comments

Popular posts from this blog

Levenshtein distance concept in NLP

Lexical Processing

Creating Bag-of-Words using Python