Implementing an ethical approach to data

AA: I’ve also been seeing lately that a lot of organizations want to ensure they are responsible when it comes to AI: implementing ethics frameworks and having people accountable for the ethical and responsible use of AI and data. How do you typically guide them on this journey in regard to ethics and governance around AI? Do you have any particular frameworks or processes that you would recommend (across different organizations) in terms of what is really needed to become ethical in this space?

CA: This is more about culture. It’s also about going back to the main high-stakes usages of AI.

People are going to hate to hear me say this, but not everybody needs to be concerned about ethics and AI. If they’re just doing A/B testing on steroids, they don’t necessarily need to be concerned.

When I talk about the ethical use of AI, the starting point is typically an impact assessment of the use case that you want to move forward based on.

Let’s say it’s some C-suite person who wants to make sure AI is being used ethically. Typically, the way you want to start with ethics is you want to look at the most impactful, high-stakes, high-competitive-advantage types of strategies that you’re using AI for inside your company. You can determine that without going through some big, lengthy process. You can pretty much just tell. There’s a red-yellow-green situation here. If it’s going to be life and death – if a person steps into your self-driving car and can die, or someone outside of that car can die by being hit by that car – that’s a high impact. That’s a red-level threat. If someone’s life, liberty, and safety can be affected, that’s red. Health diagnostics, weaponry, or anything automated is going to fall into this category. Automated with humans involved is probably going to fall into the category at some point.

If you take a step down, the orange level would be along the lines of whether we are going to affect anyone’s rights or happiness. Are we going to keep them from getting jobs, or anything that would fall into the UN framework of human rights? If we’re going to limit their ability to have shelter, food, water, or education, or the pursuit of happiness, that’s high stakes. We have to go through a much more rigorous process of evaluation: is the use case even an appropriate use of AI?

We can argue that some people in this field think more like computers than people. If you plotted a spectrum from computer to person, there are a lot of people who can fall into the thinking-like-a-computer part of the spectrum. You have to evaluate the use case considering that spectrum. Some people are OK with receiving, for example, a chatbot version of, “Hey, you’ve got cancer.” That, in my mind, is never going to be an acceptable use case for AI – never. But in other people’s minds elsewhere on that spectrum, they’d say, “It’d keep doctors from having to deliver that information, which would be hard on them.” But what about the person on the receiving end of this news? Did you think about them?

I know it seems ridiculous to think a chatbot might deliver such horrific news to someone, but during COVID times, when doctors were overwhelmed and going down with COVID themselves, the idea was becoming more of a possibility. I’m so glad we didn’t fully get there, but we were on the cusp. If you read my book, you’ll see how close the UK National Health Service (NHS) was to implementing this.

AA: That’s a great example. How effective do you think AI ethics boards are in helping organizations implement responsible AI?

CA: This one’s a tough one. It’s not because of the AI ethics boards. It’s because of the way that the AI ethics boards are set up for failure or success by the group of people who is the most powerful inside the organization. We call this the HIPPO: the highest-paid person’s opinion inside the organization. The HIPPO will be the one that is adhered to whenever there is a conflict between the AI ethics board opinion and the HIPPO. And this causes a natural contention that can make it hard for external AI ethics boards to function properly.

I just commented on this in Wired about the Axon CEO who openly announced he was thinking about developing taser-enabled drones for the inside of grade schools. It was a response to the Uvalde grade school shooting (https://www.wired.com/story/taser-drone-axon-ai-ethics-board/). The announcement caused his AI ethics board to resign because they had already discussed it with the CEO and disagreed on the development of taser-enabled drones for schools. They didn’t want to be associated with that – rightfully so, in my opinion.

You need to consider how you set these boards up, how much power and visibility they will be given, and what you will use them for. In my mind, the best use of AI ethics boards is to bring in diverse sets of opinions, especially if you’re a smaller start-up or a mid-sized firm that doesn’t have access to people who are regularly researching in the field of ethics. Typically, you want to bring in an AI ethics board that’s going to lend you their diversity in some way, shape, or form.

Diversity can exist in many different ways. It can be socioeconomic. I think at one point, Starbucks had considered doing away with cash, and what they found out real quick was that if you do away with cash, there’s a whole group of people in America, about 40 million people, who don’t have credit. They can’t whip out a piece of plastic and pay for stuff because they get paid in cash. There’s a whole cash world out there that’s not necessarily criminal. The data scientists working on the project had never been without credit, and didn’t realize this isn’t the case for everyone.

You need those very diverse opinions to remind you that there are people out there that don’t think like you. You need to figure out how to accommodate them or you may lose them, you may have public backlash, or you may face some kind of societal impact that you weren’t expecting. I think that’s where ethics boards are really strong, especially when you’re moving into a new space that you haven’t been in before.

Google’s external AI ethics board was short-lived, but let’s say Google had wanted that board because they were moving into weaponry, which they had not been doing. Let’s say they wanted to think through fully what that decision entailed. They had a member of the board that was supposed to help them think through the ins and outs of that. I think those are really good ways to use a board.

Unfortunately, a lot of companies want to just virtue signal by having and announcing the board publicly and announcing when they take the board’s advice, but not announcing when they don’t take its advice. That’s the rub right now: it’s trying to figure out that balance, and that’s hard.

AA: Speaking of ethics more broadly, there are so many different definitions around bias, fairness, and explainability. How do we reach common ground? Will we have a situation where different organizations are implementing different interpretations, and how does that affect the citizen or the consumer?

CA: That assumes that the decisions will be transparent at all, which is one of the things that I’m working on right now.

I’ve been promoting the 12 Tenets of Trust, which is on the AI Truth website right now. I think you need to have some way for people to have a sense of agency and transparency in the process of what you’re building. Otherwise, you will have bias that you can’t account for, because we’re all biased. We’re only human, so we’re biased. Unfortunately, we’re also the ones who build all of the information, and the information comes from us, which is also biased. We also can’t find data for every single aspect of our lives. As much as we produce data constantly, sometimes there really are areas of our lives where we can’t provide data that will back up a decision about us that affects us.

12 Tenets of Trust

In Cortnie’s fantastic new book, What You Don’t Know: AI’s Unseen Influence on Your Life and How to Take Back Control, she shares her insights and expertise to help everyone understand AI beyond all the hype. She also provides 12 tenets, or principles, to help creators develop trusted AI, which you can also find online: https://www.aitruth.org/aitrustpledge.

You can read more about What You Don’t Know on Cortnie’s website: https://www.cortnieabercrombie.com/.

You really have to set up your high-impact machine learning capabilities with transparency in mind. Again, not A/B testing stuff, but the high-impact stuff that incorporates feedback from those who will be affected, if you can do that.

In the financial business, we know how to implement identity and verification. There should be no reason why we can’t allow someone to verify their identity and then respond with, “Hey! You are munging five different sets of data. I see that you had this one thing in my financial score. I see it here: you gave me access. I can actually see it online. I know where to go to get to it, and I can see the explanation of what was weighted, why I was given this score, and why it might have gone down or changed, and I want to dig into that because it’s currently affecting my ability to get an education loan or a house loan.” I should be able to click into the system, and then it should open up a way for me to answer questions such as, “Where did that data come from? What was the sourcing of that? What was the lineage of that?” Then, I can participate. Was that data correct or wrong? Did it have me living in some part of town that was maybe risky according to the model, even though I have a really nice house there?

That feedback loop and the ability to have some personal agency where we can weigh in is important. Even our music choices give us that ability. So why don’t the most major decisions in our life? For music, I can say, “I like that song; I don’t like that song.” On Netflix, I can say, “I like that movie; I don’t like that movie.” Why can’t we do that for the data that’s affecting us the most? Why can’t we have the explainability and transparency in place? Somebody on the backend should have an automated capability to compare, an ability to do this thumbs up/thumbs down per piece of information that’s affecting our outcomes. They should have some sort of automation that can then affect another system to change data permanently.

Then there are notifications.

If something’s high-impact, we owe it to people to give them notifications. It is the least we can do.

If Netflix can do it, come on! If we’re going to sabotage someone financially, I think that’s the least we can do for them. Let them know and let them have the ability to weigh in, “Yeah, that’s my data. No, that’s not my data. Oh, this person turned in a bad piece of information. It was a different Cortnie Abercrombie over here in this other part of town!”

AA: That’s one of the best arguments I’ve heard for explainable AI.

A lot of organizations fail in their endeavors in AI. I’ve seen statistics of around 85% failure rates. In your experience working with large organizations, do you think this is realistic, and if so, why is it happening like this? So much money is being spent on AI. It’s relatively cheap and easy these days to develop machine learning models. Why are so many failing? Where are they getting it wrong?

CA: This is a hot point for me. This is a Cortnie Abercrombie statistic with absolutely no grounding, so I shouldn’t even say it, but I think 90% of what goes wrong is in the data, and people just don’t want to investigate the data because it takes time. It takes energy to come up with the right data that’s fit for purpose. I think that we use a lot of scraped data. We beg, borrow, and steal whatever we have to because of the way that we have set up our processes. When we look at what is at the root of AI and machine learning models, there are three aspects. I’m sure every data scientist out there is going to say, “You can’t boil my whole job down to three things!” I’m going to try anyway. It’s data, algorithms, and training or implementation. I’m including the training of the algorithm in implementation. You could probably argue that it could go outside of the data side, or you could say it’s part of the actual algorithm itself. But I think algorithm selection is its own beast, along with iterating constantly on that until you get the level of accuracy that you’re looking for.

Additionally, we have another problem that nobody even acknowledges: what are the users going to do with this stuff? I was talking to someone who just bought a Tesla and had no instruction on how to use it. 20 to 30 years ago, even when we were just getting pivot tables in Excel, that was new stuff. People trained us on all those new things. Nowadays, we just hand stuff over and say, “Here you go,” and we don’t tell people anything about it. We don’t tell them how it works or what it’s been trained on. This friend of mine who bought the Tesla won’t even use the self-parking feature on the car because she’s like, “What if a toddler runs out? Has this even been trained on toddlers? Can it even see a toddler with its camera?” I think that’s a legitimate question.

If you don’t give people some level of training and understanding of something, they just won’t use it, and that’s how we get these failure rates. There’s no trust. First of all, what data did you use to train it? That’s probably the most basic question that everybody’s going to have in their minds. The first question my friend had was, “Has it been trained on toddlers?” These are erratic little beings that can just dart right out behind a car. Someone else may be parking while we’re parallel parking, and there may be a van approaching. Do I really trust this thing to take into account some erratic little being that’s only 2 feet tall running out into the path? That question is legitimate in all cases of AI and data science. Where did you get the data, how did you train this thing, and do I trust that?

Think about scraping all those “labeled faces in the wild,” which has been the most used open-data source for facial recognition – something like 35% of the pictures were of George Bush. Did you know that? That’s ridiculous! Because it was a dataset created in the late 90s. It was mostly his face because we didn’t have as much social media participation as we do now. Even the frequency of updates is so important. How many data scientists do you hear these days saying, “Well, all of my 8,000 APIs are updated on this date”? We don’t know! We’re just pulling this crap together, managing it 50 ways to Sunday. There are 8,000 APIs coming in. I don’t know when they all come in! I don’t know where they came from! I don’t know who put that together!

And yet you have the Cambridge Analytica situation where a company pulled Facebook user data, did a personality test on everybody, and used that data to target specific people with political campaigns.

You’ve got to know where this stuff comes from and you’ve got to investigate and interrogate it, especially if it’s one of the major features driving your model. If it’s actually something that’s making a big difference, you owe it to yourself to know everything about those bits of data that are coming in. I’m not expecting people to know all of the 8,000 different APIs that they use, but they should know the major things that are affecting their models, and that’s where I think things are going wrong: it’s the data – and not understanding it. All of this leads to not having trust in the AI product or service. Lack of trust leads to non-use, and that leads to the failure rates we see today.