"One of the comments I've heard at times from smaller institutions is that they're going to consider getting out of the mortgage business because they don't want to be making mortgages that are not qualified mortgages," said Cordray, referring to the Ability-To-Repay/QM rule.

"I don't think the rule is actually pinching you to that extent – and I've had this conversation with a number of chief executives who were a little bit confused about the rule and therefore making a decision to get out of mortgages when they really didn't need to do so," he added.

Cordray stressed that the CFPB does not want its rules leading to "unexpected disruptions or unintended consequences" for credit unions.

"I don't want any institution on this phone to be scared out of mortgage lending by consultants and others who say, 'oh, there's all kinds of risk involved in the Ability-To-Repay rule.' If you do good solid lending, you're used to doing it, you've made those kind of hard-headed decisions year in and year out, about who's a good credit risk and what the properties look like in your community…keep doing what you do and what you do well," Cordray said.

"Don't let people get in the way of you serving your members by helping them finance homes which is something we need credit unions to do and again that you tend to do very well," he added.

Cordray said the CFPB has confidence in the credit union model. He applauded the NCUA for releasing a great deal of information on the rules to help credit unions.

"We encourage you to continue writing mortgages according to your accepted settled underwriting standards," he said.

"You've had very few foreclosures even during the worst financial crisis of our lifetime, almost certainly we will not see market conditions like that any time in the foreseeable future," he added.