"It allows us to do our job, it allows us to protect the people who deserve to be protected, and by law (they) are going to be protected," Mulvaney said. "It also allows you to do your jobs. It allows you to provide the (financial) services. We recognize that there is value in what you do, and we thank you for it."

Mulvaney received a round of applause when he said that the CFPB recognizes that credit unions did not cause the financial crisis and that one size does not fit all when it comes to regulations.

While the acting director said the CFPB will continue to go after the bad actors and that the agency will continue to faithfully enforce the laws to protect consumers, it will be done differently.

"Yes, we are there to help protect people who use credit cards, but we are also there to protect people who provide that credit because that is an important service to provide consumers," he noted. "We are there to help people who borrow money, but we are mindful and respectful of the people who make those loans."

He pledged to use cost analysis models to determine the true cost of regulations.

"As I have told the folks at work, there is going to be a lot more math in our future," he said.

Mulvaney also said the CFPB is not going to bend over backwards to find creative ways to sue people.

"We are not going to regulate by enforcement by sending a message to people by suing them," he said. "You absolutely have the right to know what the rules are. You can't do your job without it. And it's up to us to make sure you know what the rules are. So in the future, you will know what the rules are before we accuse you of breaking them."

Mulvaney note that everyone deserves be treated fairly.

"I don't ever want to go to bed at night wondering whether or not we put out a business (that) deserved to be put out of business or not because that means that people lost their jobs, people lost their reputations and communities have lost their access to financial services," he said. "And if we have to do that, we have to be absolutely 100% sure that we are acting appropriately. I think that is the biggest difference between the old leadership and the new leadership."