It's already making it tough for them to save for retirement, start businesses, and even stay current on daily expenses.

On average, the study finds, people graduate college with nearly $40,000 in debt.

Men carry higher debt levels than women, and when broken down by race, white and Asian grads have higher debt levels than black and Hispanic/Latino grads.

Among study respondents, all of whom were at least 3 years out of school, the average amount of debt remaining to be paid is approximately $30,000—meaning that, on average, grads still have about 75% of their debt still to pay years after they've left school.

White male students, with the highest level of debt among subgroups, have about 33% higher debt levels than white females—who tend to graduate with the second lowest levels of debt.

However, it's not because women are getting help in paying their tuition. Just 6% of women say their parents paid for the majority of the tuition, and 50% say they received no help at all.

Among male students, it was a different story, with only 43% of male respondents saying their parents did not contribute financially.

Overall, just 10% of grads got tuition help from their families, while close to half got through college without any family bailouts.

Ten percent of male respondents, in fact, say their parents paid the majority of their college tuition; that's almost twice as many as the women who say the same.

Yet women graduate with less debt than men. The only exception is black women, who had $272 more in debt than black males.

Asian males graduated with the second highest level of debt, only an average of $253 less than white males. But they also got the largest amount of help from their parents, with nearly double the number of Asians reporting the majority of their tuition paid by parents in comparison with other races.

And the lowest debt levels of all belong to Hispanic/Latino women, who not only graduated with the least amount of debt but had $564 less to pay off than white women.

More than half the respondents of both genders say their debt has negatively impacted their current lives, with nearly two thirds saying that current loans have made it harder to save for retirement; more than 50% say the debt has made starting a small business more difficult.

When it comes to keeping up with daily expenses, the men have an easier time than the women, with only about half the men saying their debt has made day-to-day finances more burdensome; 10% more women feel that way.

And while 13% more white college graduates than Asian graduates believe their loans have negatively impacted their retirement savings, Asian respondents believe their debt weighs more heavily on their ability to launch their own business—with nearly 10% more Asian graduates saying so than those of any other race.