“We determined that we couldn't back away from that market because if we did, it was going to be a substantial portion of our business,” he said.

Templeton said the credit union had to stop working with people who were on the fringe. To a certain point, SRP was willing take a risk on doing so.

“[Up] until then, our track record had been near perfect for 25 years. I think for 25 years we lost maybe two times on a loan like that,” he noted.

Templeton said that track record showed those borrowers were willing to work with the credit union.

“Now, all of a sudden, they've got a brand on the mortgage that says non-qualified. It's a target in the event of any type of catastrophic event in that person's life and the outlier presents just too much of a risk today where 10 years ago that was bread and butter to us,” Templeton said.

John Buhrmaster, president of First National Bank of Scotia and chairman of the Independent Community Bankers of America, said if loans held in portfolio by community banks were exempt from the QM rule, his institution could provide more capital to businesses.

“We would not be restricted,” he said. “The hardest loan to make right now is to a small business, a self-employed individual.”

Buhrmaster also said his community bank is losing capital on development loans.

Murray, who is also chairman of the Nevada Credit Union League, urged Congress look at credit union capital requirements, increase the member business lending cap and

streamline field of membership requirements. He also asked Congress to ensure the CFPB uses its exemption authority to a much greater extent than it has to date.

Sen. Pat Toomey (R-Pa.) and Joe Donnelly (D-Ind.) of the Senate Banking Committee introduced a bill that would raise the asset-size threshold for financial institutions subject to CFPB oversight from $10 billion to $50 billion.