“It is impossible at this time, based on the information we have, to draw firm conclusions about the performance of our regulators, but all this does raise serious questions,” he continued. “Maybe our federal regulators deserve a pat on the back; but maybe they deserve a swift kick on the backside. We'll find out which.”

However, Waters said that the CFPB discovered the problem at Wells Fargo.

Some members of the committee said they might want the panel to investigate whether the practice of cross selling products by financial institutions might encourage abuses.

Stumpf again apologized for the crisis, which has resulted in federal and state regulators fining Wells Fargo some $185 million for bank employees opening approximately two million unauthorized credit card and bank accounts. Earlier this week, the bank took back $41 million in stock bonuses awarded to Stumpf.

That didn't satisfy some members of the committee, who said Stumpf should resign and that Wells Fargo was a criminal enterprise.

Stumpf defended cross selling of products, saying that it deepens the relationship between a financial institution and its account holders.

However, Stumpf announced that the company's regional banks will end the practice of setting product sales goals at the end of this week.

Still, some members called for Stumpf's resignation.

Republicans made it clear that the discovery of the Wells Fargo problems should not be considered a victory for the CFPB—one of the agencies involved in the probe and one that GOP members want to abolish.

“The CFPB was asleep at the wheel,” said Rep. Scott Garrett (R-N.J.), who said the agency did not take action for several years even though The Los Angeles Times had reported on some of the abuses.

“They completely blew it,” Garrett said.