Participants were asked to describe the strengths and weaknesses of their core system. Additionally, they were queried on topics such as general functionality, system openness and also quality of support, Brownell said.

While there were some big winners and losers in the data, some industry experts raised questions about the survey's methodology and validity.

According to the survey, the Brookfield, Wis.-based Fiserv had four of the lowest scores for its Cleartouch, Advantage, CUBE and Charlotte cores. The other “below average” core was for INTECH in Omaha, Neb. Electronic Recordkeeping Services in Fishers, Ind., received the only “poor” rating, the data showed.

In a written statement, Fiserv said, “Based on the platforms cited, it doesn't appear to be representative of the satisfaction of our overall client base. For instance, the CUBE platform is slated for end-of-life in 2015, an announcement made over four years ago to the small number of clients on that platform. Overall, there are less than 100 customers on all the platforms mentioned in that category, representing 5% of our install base.”

Fiserv said one-third of all financial institutions use the company for account processing, including more than 2,000 credit unions.

“We conduct monthly client 'pulse' surveys for our product lines, including those mentioned, and feel this survey is not representative of the satisfaction of our overall client base,” Fiserv wrote.

Other Fiserv cores systems, including DNA, Portico and XP2, scored “above average” in the rankings, according to the CUCollaborate survey.

“I'm not surprised to see the frustration of some Fiserv customers that are using core systems that aren't currently receiving the development or marketing attention that Fiserv DNA is enjoying now,” Scott Hodgins, senior director with Cornerstone Advisors, a research firm in Scottsdale, Ariz., said.

Fiserv's rival, the Monett, Mo.-based Symitar, by contrast, received a “good” rating for its flagship Episys core, the survey revealed. Fiserv's CUnify core system also earned a “good” ranking from the survey's participants.

In a statement, Symitar President Ted Bilke wrote, “Symitar understands customer support is a key factor in client satisfaction and retention. We request feedback from our customers via individual cases as they close and annual surveys. Our corporate goal is to always score between 'exceeds expectations' and 'far exceeds expectations.' I'm happy to share that we consistently meet and surpass our goal. This focus on providing consistent quality customer service across all our products and services is reflected in our survey scores.”

Read more: Some question the survey's statistic validity …

CUProdigy was another winner, according to the survey's participants. The Layton, Utah-based core developer notched a “great” rating, making it one of six core systems to receive that accolade.

Participants also gave a “great” ranking to Systronics, an Olathe, Kan., firm that primarily serves small credit unions; Corelation's Keystone in San Diego; Commercial Business Systems in Goldsboro, N.C.; FedComp in Fairfax, Va., and Cu Centric in Columbus, Ohio.

“We felt it was necessary for our member/owners to participate in the survey,” Craig Peterson, CUProdigy director of client services, said. “As a CUSO, we are focused on the requirements of our user group and the feedback from the survey was insightful.”

Still, some experts raised concerns about the survey including Preston Packer, director of sales and marketing at Computer Marketing Corp.'s FLEX in Sandy, Utah. The firm earned a “good” rating among participants.

“I believe that reports such as what CUCollaborate is providing hold a great deal of potential for credit unions when reviewing core systems. However, it is imperative that Sam Brownell and CUCollaborate remain transparent in their reporting and work to improve their sample size,” Packer said. “Otherwise, these reports will fall by the wayside, just as others have done in the past.”

David Gibbard, formerly a SVP at the Birmingham, Ala.-based core developer EPL, and now an independent consultant in Atlanta, offered a more broadly critical assessment of the survey. Participants gave EPL a “good” rating.

“I don't think this survey is statistically valid. It's people who are very happy or very unhappy,” he said.

Gibbard claimed most credit union executives would not take the time to fill out the questionnaire. Only raving fans and those who are grievously unhappy would bother. He believed the survey had value for what he called “its trending information.” For instance, many credit unions on legacy Fiserv cores are dissatisfied, which is a trend, he said. However, Gibbard did not think valid quantitative conclusions could be drawn from the data.

He may have a point. With Fiserv's maligned CUBE, just one respondent filled out a survey. With its Charlotte core, four out of 98 on the platform, responded. The sole “poor” score for Electronic Recordkeeping Services came from one response out of some 68 core installations at credit unions.

On the other end, Systronics, which had the highest rating in the survey, snared it on the basis of two responses out of 73 clients.

Do those low response rates invalidate the CUCollaborate effort? Brownell claimed an overall survey response rate of 4.5%.

“While we make no claims about that response rate providing a statistical significance, we are very proud of the high participation rate that we have received as a new company,” he said.

Brownell said CUCollaborate believes the response rate will increase over the next few years.

“Our goal is to achieve over a 10% participation rate both industry-wide and for each platform,” he added. “We also very clearly show the response rate that we received for each platform in our report so readers can decide for themselves how much weight they feel like they should give the ratings we gathered through our survey.”

Can the results still be trusted?

Thad Moore, vice president at the $546 million Self-Help Credit Union in Durham N.C., chose his words carefully when asked for his opinion of the report.

“I like the promise of it and I got nuggets of information out of it. There's not enough data points, however,” he said.

Moore indicated he would have more confidence if the response numbers increased.

“It's a good first effort,” he pointed out.