wikipedia article showing dates when certain groups received voting rights.

Before 1856, only white men who paid taxes and were property owners had the right to vote. This was about 6% of the total population. Universal suffrage for white men didn't come about until 80 years after the foundation of the United States. Other countries around that time were similarly limited in their voting rights, and even more restrictive than the US.

So why do we only hear about women's suffrage and how men oppressed women's rights? White men were just as oppressed by wealthy land owners as everyone else.