Businesses in Canada with fewer than five employees pay 45% more per employee at $5,942 to comply with government regulation than their U.S counterparts at $4,084, according to the CFIB. 

One of the most startling findings was that 31% of business owners surveyed in Canada and 23% in the U.S. said if they had known the burden of complying with regulations, they may not have gone into business. 

"Red tape is not just a nuisance," said Laura Jones, CFIB executive vice president. "It's a hidden tax that holds back our economy."

The report gathered feedback from March to May 2012 from 8,562 small  and medium-sized business owners across Canada.

The total cost of regulation in Canada is estimated at $31 billion a year, according to the CFIB. Since the group began its analysis in 2005, those costs have been relatively stable, the federation added, saying this is encouraging because it suggests that work at both the federal and provincial levels of government to control the growth of regulation is paying off. 

Small businesses in the U.S. are coping with regulations in much high proportions. The total cost of regulation is estimated to be $198 billion a year. Still, given that the U.S. has roughly ten times the population of Canada, its total costs of regulation per capita are considerably lower, the CFIB said. 

"Not all regulation is red tape, but businesses in both countries tell us regulatory costs could be reduced by about 30% without harming the important health and safety objectives of regulation," Jones said. "That's the equivalent of a $9 billion stimulus package each year in Canada with no downside."

The $149.7 billion Credit Union Central of Canada in Toronto said credit unions are strong supporters of local businesses and are familiar with the costly burdens that regulations create for small business owners.  

"Canadian credit unions agree that a strong regulatory environment is important to protecting the savings and financial assets of Canadians. However, it is important that the regulations be clear in their application and tailored appropriately to the circumstances of small financial institutions," said David Phillips, president/CEO of Canadian Central.

The credit union said it has also encouraged the federal government to consistently apply the small business lens concept introduced last year in the government's Red Tape Reduction Action Plan to all new regulations. 

"A small business lens requiring regulators to consider the impact of regulations on small business is a very important perspective to bring to regulation-making so that regulations can be designed to achieve their objectives without imposing excessive burdens on small businesses," Phillips said.

The CFIB cited data from the SBA that showed in 2010, the agency said small businesses with fewer than 20 employees faced the largest burden in federal regulations, which was 36% higher than the costs incurred by larger firms with 500 or more employees. 

Indeed, while efforts have been made by various levels of government to reduce the burden of regulation, the smallest businesses are still the hardest hit, according to the CFIB. Based on 2012 data, regulations cost Canadian businesses with fewer than five employees $5,942 per employee per year. This is over five times as much as the regulation cost of $1,146 per employee per year in businesses with at least 100 employees.

Among the reasons why smaller businesses pay more than their larger counterparts, is they tend to have fewer resources to devote to regulatory compliance. Larger businesses often have in-house resources devoted solely to regulatory monitoring and compliance.

The second reason why regulation hits the smallest firms hardest is that these businesses often operate in much more competitive industries than their larger counterparts, the CFIB said. In competitive business environments, there are fewer opportunities to pass along costs to customers.

While the CFIB's report reveals that costs are still high to comply with regulations, Canada is listening to the frustrations from small business owners. 

"Across the country, entrepreneurs have told me they're very happy the federal government is making important and necessary changes to help Canada's job creators," Jones said. "The Prime Minister and his government get this issue, and deserve credit for their work that will make Canada more competitive in the long run."