Reason for lack of Google Hangout users:
Actually it was Google talk. In my first Android mobile it was Google talk and I was happy with that app. Suddenly Google introduced Google+ and they updated Talk to Hangout. It was very confusing because usually updates will add additional features but in this case they changed the whole user interface. In Google Talk I used to chat only if my friend is online. In Hangout there is no way to check whether the user is online or offline.
Hangout used online Gmail contacts plus Orkut contacts. So most of them will be unknown to you. Google try to implement offline chat like SMS. In SMS you don't bother whether the user mobile is on or off. But it confused users and many were not pleased with it.
Reason for WhatsApp success:
The main advantage of WhatsApp is it used only the phone contacts. All will have only close friends contacts in their phone book. I had only 30 to 40 contacts in my phone whereas I had 130 gmail contacts. So WhatsApp created connection only with necessary people and you will frequently contact them. In WhatsApp you can see the user available status, send offline message, create groups. On the whole it filtered and provided you with necessary people.
All other social network allowed you to add unknown person. Even Facebook allowed you to add unknown contacts but with warning. It is the key for WhatsApp success.