MMIS 671: Solutions to Homework 2. Decision Theory and Bayesian Inference


An organization uses a spam filtering software to block email messages that may potentially be spam messages. The spam filter can be set to one of two security modes: High-Security-Mode (H) or Low-Security-Mode (L).
Extensive evaluation using a benchmark corpus consisting exclusively of spam messages yields the following performance statistics for the spam filter:
96% of the spam messages are blocked in the High-Security-Mode.
90% of the spam messages are blocked in the Low-Security-Mode.
Extensive evaluation using a benchmark corpus consisting exclusively of non-spam (legitimate) messages yields the following performance statistics for the spam filter:
10% of the non-spam messages are blocked in the High-Security-Mode
4% of the non-spam messages are blocked in the Low-Security-Mode
The organization estimates that 80% of the messages that it receives are spam messages.

Define: S: Event that a message is a spam. B: Event that a message is blocked.

Given: P(S)= 0.80 ⇒P(S ̅ )= 0.20 
High-Security-Mode Low-Security-Mode
P(B|S)=0.96 ⇒P(B ̅|S)=0.04 P(B|S)=0.90 ⇒P(B ̅|S)=0.10
P(B|S ̅)=0.10 ⇒P(B ̅|S ̅)=0.90 P(B|S ̅)=0.04 ⇒P(B ̅|S ̅)=0.96

Let P_H (S|B ̅ ) denote the conditional probability that a message that is not blocked by the spam filter operating in the High-Security-Mode is actually a spam message. Estimate P_H (S|B ̅ ). 

P_H (S|B ̅ )=(P(B ̅|S)×P(S))/(P(B ̅│S)×P(S)+P(B ̅|S ̅)×P(S ̅))= 


Let P_H (S ̅|B) denote the conditional probability that a message that is blocked by the spam filter operating in the High-Security-Mode is actually not a spam message. Estimate P_H (S ̅|B). 

P_H (S ̅|B)=(P(B|S ̅)×P(S ̅))/(P(B|S ̅)×P(S ̅)+P(B|S)×P(S))= 


Let P_L (S|B ̅ ) denote the conditional probability that a message that is not blocked by the spam filter operating in the Low-Security-Mode is actually a spam message. Estimate P_L (S|B ̅ ). 



Let P_L (S ̅|B) denote the conditional probability that a message that is blocked by the spam filter operating in the Low-Security-Mode is actually not a spam message. Estimate P_L (S ̅|B).



If the cost of not blocking a spam message is $1 and the cost of blocking a non-spam message is $10, should the organization operate the spam filter in the High-Security-Mode? Why?

Expected Cost = $1×P(B ̅│S)×P(S)+$10×P(B|S ̅)×P(S ̅) 
Substituting the given probabilities, we compute the expected costs:
Expected Cost in High Security Mode= 
Expected Cost in Low Security Mode= 

The organization should operate the spam filter in the High-Security-Mode only if its expected cost is lower.

Recall that the cost of not blocking a spam message is $1. At least how high should the cost of blocking a non-spam message be for a risk-neutral rational decision maker to prefer operating the spam filter in the Low-Security-Mode? 

Let the cost of blocking a non-spam message = $ C.
Expected Cost = $1×P(B ̅│S)×P(S)+$C×P(B|S ̅)×P(S ̅) 

Equate the expected costs in the two modes to determine the breakeven point.

A risk-neutral rational decision maker to prefer operating the spam filter in the Low-Security-Mode when its expected cost is lower. 
That is when C≥ 


Let $ C be the amortized cost per message of operating the spam filter. Write a short memo the CIO of the organization explaining at most how high C could be for the organization to use the spam filter with the specified performance? 

What is the expected cost per message when operating the spam filter?
Without a spam filter in place we won’t be blocking any messages. 
Under our current assumptions, the resulting expected cost per message is $0.80 since 80% of the messages are spam and the cost for not blocking a spam is estimated to be a dollar. 
How much does the spam filter saves us per message?
That is the maximum we should be willing to invest in the spam filter per message.