where we have used a known fact from probability theory that P(xi, yj ) = P(yj |xi)×P(xi). Note that E(Y |X) is also known as the conditional entropy of Y given X. To answer this question, we need to show that E(Y |X) ? E(Y ). Let us com- pute the difference between the entropies after splitting and before splitting, i.e., E(Y |X) ? E(Y ), using Equations 4.1 and 4.3:

What will be an ideal response?