In decision tree algorithm, these 2 terms play major role and sometimes harder to grasp what it means. Lets decode.
Let's assume we have 3 classes (namely A,B,C) in our dataset. Entropy (E) = (prob. of randomly selecting an example in class A) +(prob. of randomly selecting an example in class B) + (prob. of randomly selecting an example in class C). i.e, sum of all the probabilities. (total uncertainty of data)
What do we get out of this?