Prophet Hud - 11:116
Should not the righteous of the generations before you have prevented evil in the earth? Except a few whom We saved from them. As for the wrongdoers, they indulged in the prosperity given to them, so they joined the criminals.
The believers were always a minority:
The Scatterers/51:31
"(Abraham said): "O messengers, what is your mission?"
The Scatterers/51:32
"They said, "We have been sent to a guilty people."
The Scatterers/51:33
To send upon them stones of clay...
The Scatterers/51:34
Marked by your Lord for the transgressors.
The Scatterers/51:35
Then We drove out whoever therein was of the believers.
The Scatterers/51:36
We did not find therein any Muslim except in a house.
Prophet Jonah/10:83
Only a few of his people believed in Moses because they feared the torture and oppression of Pharaoh and his men. Pharaoh was a tyrant in the earth who exceeded the limits.
Prophet Hud/11:40
"Finally, when Our command came and the sky boiled and overflowed, We said to him: "Load on him a pair of every kind, his children and those who believe, except those who were condemned before."Those who believed with him were already a few."