Righting Course: The Credit Union Fix for AI’s Unintended Consequences

Among the top priorities credit unions face in the year ahead, we can find a desire to identify the tools and techniques necessary to ensure members have a frictionless experience. Among those tools, investments in machine learning and artificial intelligence are often part of strategic planning session discussions across our industry.

AI makes us think about an array of possible alternatives, all pointing to operational efficiency gains and a responsive experience for our members. It is at times such as this, where a lesson from Facebook’s startup years and its old mantra of “move fast and break things” can offer insights to credit unions.

Yes, to be disruptive, fast-prototyping, failing-fast and agile methodology are at the core of an innovative culture. Breaking things was meant to challenge the status quo, but is also used as a culture shelter for “because we move fast, mistakes are made, and that is OK.” That last part may not be a strategy that credit unions want to emulate. When it comes to our role of being stewards of our members’ money and information, accuracy, security and scrutiny must trump speed.

The AI Iceberg

The number of credit unions launching AI initiatives is increasing year after year. But for many, it is still more like an iceberg. Most of the implications of such technology are below the waterline. After all, algorithms for AI are heavily dependent on the data they are being fed for its machine learning. Those tech companies at the bleeding edge of AI may still be operating in the fast lane, and as credit unions interact with them, those stewardship characteristics are critical to the success of any future practical applications of IA.

When it comes to AI, the iceberg metaphor serves us well to begin understanding two critical issues.

First, the output of arguments is unsound if the premises built in the algorithm are flawed. It is possible that current data being used for machine learning could be biased, resulting in favoring certain behaviors that are prevalent among only some segments of our membership.

Second, the possibility that even with “clean,” unbiased data, the output may inadvertently identify characteristics that may put some members at risk of being marginalized.

For example, let’s say the aggregate effect (regardless of intent) of the interpretation of the data by an algorithm yields a discriminatory effect. We cannot wave our hands and say, “the algorithm did it,” when the outcome of the use of machine learning in credit scoring is to exclude or discriminate against people based on race or disability, even if demographic data was not in the mix at the input stage.

Another example that made pop culture news is of “Tay,” a Twitter bot designed by Microsoft and launched on March 23, 2016 to engage with people ages 18 to 24. Within 16 hours, Tay, shaped by the conversations it established with other users, began to post inflammatory and offensive tweets through its Twitter account, forcing Microsoft to shut it down. Tay was programmed to learn from the behaviors of other Twitter users, and in that regard, the bot was a success. The difference between Tay and many AI-based solutions is that Tay’s results were transparent, and its human watchers recognized that even though it was operating “properly,” the result was not socially desirable.

Who Benefits From AI?

This is in essence at the root of our dilemma. Algorithms built to maximize profit and reduce risk are both good things that make our case to invest on AI in the first place, but may inadvertently be leaving vulnerable populations, literally, out of the equation.

It is here where we may find a role for credit unions to harness the power of AI to advance member-centered values of credit unions. In adopting AI deliberately, and harnessing it to their own mission and principles rather than the other way around, credit unions can become the trusted leaders of financial tools and services that work for everyone. This topic is studied in detail in an upcoming Filene report titled “Ethical and Legal Concerns of Using Artificial Intelligence.”

A Better AI Outcome

Here is where credit unions can excel: By playing a role that allows members to understand how their financial institution uses their information, and serving as an advocate for transparency and inclusiveness so that this promising technology is used ethically and improves the lives of many. With this goal in mind, here’s how credit unions can lead the charge in fair practices of AI:

• Know your data and understand how it may impact the machine learning process before AI initiatives are launched. If we use past data to make autonomous AI decisions, and these could range from historical lending results to hiring and executive selection practices, we may infiltrate biases.
• Develop an evaluation process for the adoption of AI-based initiatives and choose vendors carefully by discussing openly how the data is being used, and what the unintended consequences of the algorithms may be.
• Be transparent. Educate credit union members about what AI means to them as consumers and leverage this information to protect and inform the larger credit union system through partnerships and collaboration.
• Contribute to the development of regulations around the use of AI by financial institutions. As credit unions represent the interests of all people, help ensure that regulations protect the needs and rights of everyone in your community.

In the words of Ed Filene, in his 1931 publication, “Successful Living in This Machine Age”: “… in times of great social change, long experience is the very thing on which we can not rely.”  Almost 90 years later, this warning resonates deeply and shines light on the nimble characteristic that credit unions must possess in embracing our role of helping all people through the tools and technology offered to enhance members’ financial well-being.

Elry Armaza Impact Director & Analyst for the Filene Research Institute. He can be reached at 608-661-3750 or elrya@filene.org.