We were thrilled to have Avi Goldfarb, one of the world's leading experts on AI, kick off our 2023 Eye of the Executive In-Person Series in partnership with Aird & Berlis LLP, on March 23rd, 2023. Avi is the co-author of the bestselling book Power and Prediction, on how artificial intelligence is changing business, economics, and finance. In his keynote, he provided key insights on how AI is transforming the global economy and how companies can best leverage its capabilities.
Avi Goldfarb is a renowned expert in AI and its impact on business, economics, and finance. He has written several books on the topic, including Prediction Machines: The Simple Economics of Artificial Intelligence, which was named a best business book of 2018 by strategy+business magazine. He is also a frequent speaker and commentator on the topic, having been featured in publications such as The New York Times, The Wall Street Journal, and Forbes.
PEO Leadership provides its business community the ability to leverage its collective knowledge, experience, and network; to challenge and be challenged in a high-disclosure, objective, and trusted environment through a combination of Peer Advisory Boards, One-on-One Coaching, and Thought Leadership Executive Networking Events - all to enhance the personal and professional lives of its members.
We were thrilled to have Avi Goldfarb, one of the world's leading experts on AI, kick off our 2023 Eye of the Executive In-Person Series in partnership with Aird & Berlis LLP, on March 23rd, 2023. Avi is the co-author of the bestselling book Power and Prediction, on how artificial intelligence is changing business, economics, and finance. In his keynote, he provided key insights on how AI is transforming the global economy and how companies can best leverage its capabilities.
Avi Goldfarb is a renowned expert in AI and its impact on business, economics, and finance. He has written several books on the topic, including Prediction Machines: The Simple Economics of Artificial Intelligence, which was named a best business book of 2018 by strategy+business magazine. He is also a frequent speaker and commentator on the topic, having been featured in publications such as The New York Times, The Wall Street Journal, and Forbes.
PEO Leadership provides its business community the ability to leverage its collective knowledge, experience, and network; to challenge and be challenged in a high-disclosure, objective, and trusted environment through a combination of Peer Advisory Boards, One-on-One Coaching, and Thought Leadership Executive Networking Events - all to enhance the personal and professional lives of its members.
Hi, I'm Leon Goren, president of PEO Leadership, a peer-to-peer leadership advisory firm. We're an amazing community of CEOs, presidents, and senior executives. Ask yourself, are you learning as fast as the world is changing? It's time for Ontario business leaders to band together for counsel and support. It's time for you to tap into the business wisdom of our peer groups and unlock new ways to grow. I want you to come out of this COVID crisis a better leader and your organization ready for what's next, take the first step at PEO-leadership.com. Special thanks to Aird & Berlis for helping us bring you today's PEO Leadership's Way Forward podcast.
Thank you all for joining us today. I think many of you who are running your business today are thinking about AI. But what I'm hoping at the end of the session today is, you'll start to look at it a little differently. You'll start to think about it from a perspective of what can I start doing today in reinventing my business and creating new processes and workflows to actually start to pull AI into your business. It's very early today in terms where AI is. The question is, and I think we all believe that we're actually moving somewhere down the road with AI, but we're almost in that in-between state. And in between stages actually, I think that's where the opportunity exists, for those that are willing to sort of dig into it and look into it a little bit more.
So Avi Goldfarb is the Rotman Chair in Artificial Intelligence and Healthcare, and professor of marketing at the Rotman School of Management, which is the University of Toronto. He is also the Chief Data Scientist at the Creative Destruction Lab, and a Research Associate at the National Bureau of Economic Research. His research focuses on the opportunities and challenges of the digital economy. Since receiving his PhD in Economics from Northwestern University, he has published academic articles in marketing, computing, law, management, medicine, physics, political science, public health, statistics, and economics. His work on online advertising won the INFORMS Society of Marketing Science Long Term Impact Award, and he testified before the US Senate Judiciary Committee on competition and privacy in digital advertising. His keynote speeches include the NATO Defense College’s 70th anniversary event, the IAEE Aerospace Conference, and the European Commission's workshop on the economics of AI. And of course, obvious recent research on artificial intelligence includes the best selling books: Prediction Machines, and Power and Prediction. Thank you.
Just to give you a sense of how I came into AI. I'm marketing professor at the University of Toronto, I made the early part of my career focusing on trying to understand how digitization how the Internet affected the practice of marketing, online advertising and privacy in the economy more generally. And then in 2012, we started this this organization called the Creative Destruction Lab, which is the program that helps science based startups scale at the University of Toronto. In that very first cohort in 2012, there was this company called Atomwise, they're using AI to predict which molecules bind with its proteins like AI for drug discovery. And the next year, we had a couple more companies saying they were AI companies in our lab. And then a couple of years later, we had this flood of companies coming out of the computer science labs, a Toronto and Waterloo saying they are AI companies, to the point where as far as we know, at the Creative Destruction Lab, as of 2015, we had more AI startups than anywhere else in the world. And so my co authors and I- Ajay, Joshua and I said, Well, this, this seems like it's a thing. And no one else knows yet. Because we're in Toronto, and it's coming out of Toronto, it's coming out of Waterloo, and the rest of the world hasn't figured it out. And so we put our heads together and tried to start to do research on trying to understand how AI was going to affect business was going to affect the economy was going to affect society. That eventually led to our first book Prediction Machines, which in turn got us insights into companies trying to build AI into their processes. And there was all sorts of excitement. And a little disappointment, for many saying, hey, you know what? You started talking about this way back in 2015, 2016. Here we are in 2021. How come the revolution hasn't happened yet? Maybe in 2023, it's happening. How come the revolution hasn't happened yet? And so then, we get our heads together and said, Well, now we have something new to say. That's something new to say is this book Power and Prediction, and wondering, are we talking about with you for the next half an hour to an hour? Okay, so plenty of hype around AI. Chat, GBT was released late November. Since then we've heard things like generative AI is a game changer that society and industry need to be ready for. Maybe we should be afraid of it because of the AI overlords coming our way. It's a talk of Davos, it seems to be the talk of the world. The excitement around AI seems to generate incredible opportunities. Daniel Kahneman who won the Nobel Prize, at least psychologist won the Nobel Prize in Economics at UCLA Nearly AI is going to win. And how people adjust is a fascinating problem. Well, it's a fascinating problem if you've already won the Nobel Prize. For most of us, it's a terrifying problem, right? If the machines are better than us at everything, what does? What does that mean for us what's left for us humans to do? A lot of the hype, and a lot of that anxiety is driven by confusion about what we're talking about when we're talking about artificial intelligence, we tend to jump to science fiction. And in science fiction, the machines can do just about everything we humans can do. The only real difference is that they're not carbon based. Otherwise, it seems like there are machines who are artificial general intelligence. Sometimes they listen to us, like they do for the most part in Star Wars. And sometimes they don't listen to us, like is true in the rest of science fiction. But this idea of an artificial general intelligence is exciting, it is possible. But it is not the technology we're talking about in 2023. The technology we're talking about today, is a particular branch of artificial intelligence, called machine learning. And machine learning has gotten much, much better over the past two decades. And we should think about that as prediction technology, using information you have to generate information you don't have. So I know there is at least one person in this audience who took a stats class with me. And that this is a little bit of a reminder of that. Okay, that AI seems like it's so easy, it seems like it's so exciting. But ultimately, it's computational statistics. It is a branch of computational stats, using the information you have to generate information you don't have. Computer scientists are brilliant marketers, they call it AI. But it's computational stats. Computational stats is nevertheless a big deal, because prediction helps us make better decisions. And decisions are a big deal. Decisions are everywhere. So the opportunity in AI, is to recognize that what's happened is that predictions gotten cheaper. We can and because predictions gotten cheaper, predictions gotten better and better and better and better. We're gonna see more and more machine prediction, embedded in everything we do. That was the theme of our first book Prediction Machines, written around 2018, saying we're just on the cusp of this massive drop in the cost of production. Remember, we're economists. So we think about technological changes a drop in the cost of something fundamental. And this is what you may or may not remember from econ 101, demand curve sloped downward. If something is cheap, you're going to do it more. When the price of coffee falls, people buy more coffee. When the price of machine prediction falls, we're going to see more and more and more machine prediction embedded in everything we do. Point number one, today's AI is prediction technology. I don't think AI is crazy. Someday there are good reasons to think it will happen. But it has been 20 to 50 years away. Since the early AI researchers were talking about it in the 1960s. And it continues to be 20 to 50 years away. This is not the technology we're talking about today. And so what we're seeing is machine prediction embedded into more and more processes. Go to a bank and you want a loan. And the old days, the loan officer would look you up and down and make some kind of prediction about whether you're going to pay back that loan. Maybe just a gut feel maybe there was some data behind it or some paper. Increasingly, we're using machine learning tools. We're using AI we're using prediction machines to predict whether someone's going to pay back that loan. The insurance industry, they're in the business of prediction. That's what they do. They're pricing risk, and increasingly, insurance companies are using AI to help them with underwriting. But as machine predictions getting cheaper, we're starting to realize that some things that you might not have thought of as prediction. Our prediction, it turns out medical diagnosis can be reframed as prediction. What is your doctor do when they diagnose you? They take in data about your symptoms, and they fill in the missing information of the cause of those symptoms. That's prediction. image recognition is prediction. How do you recognize a familiar face your eyes take in these light signals and they fill in missing information, have a label and a context. Turns out that can be solved with prediction machines. Even language in image generation, turns out can be solved with prediction. That's all open AI is doing. It's taking data about what people have written in the past, in response to a particular kind of prompt and filling in Okay, in response to this new prompt, what kind of information? Is that person looking for? Based on the corpus of data out there, given this question, what's the word? And then what's the word that's most likely to come beside that word? Similarly, do you ask Dali, also from open AI, like an image of astronaut on a horse in the style of Andy Warhol? It predicts what you're looking for in provides that image in his still prediction technology using data you have to generate data you don't have that the creative destruction lab, we've seen lots of companies unpack lots of startups uncap, pack workflows and existing industries, identify some prediction tasks within that workflow, human process and drop in the machine, keeping the workflow the same embedding prediction in that workflow. And so we've started to see increasing excitement around the technology to the point where people said this is this is the new general purpose technology. This is the new steam engine. This is the new electricity. How does that play out? Well, imagine there's some dial, okay, and the dials making the predictions better and better and better over time. One particular example is an Amazon because Amazon makes recommendations of what they think you might want to buy. These recommendations are pretty amazing. The right something like 5% of the time, they have hundreds of millions of items in their catalog, you're laughing Wait, wait, I don't know you had hundreds of millions of things you would you be right file, you know, one of every 20 times, probably not, it's incredible, right? Hundreds of millions of items in the catalog the right one at every 20 times. But obviously, given the laughter, 1 in 20 times isn't transformative. Amazon's business model and their workflow is in many ways, the exact same business model and workflow when they started way back in 1995. It's also pretty similar to what Sears was doing way back 100 years ago with the Sears catalog, they're a catalog company, you go to their catalog, it is a fantastic catalog, it has a lot more stuff than Sears, did you order something, they send that information to their warehouse and they ship it to your door. Imagine if instead of 5%, it was 20% or 40% or 60%? Well, at some point, Amazon could change their business model, they wouldn't have to wait for you to order it. So you know what I'm, I'm pretty sure I know what you want, I'll just send it to your door. And you don't have to want it. Or you don't have to want it for sure. You just have to be confident enough, they have to get enough of your share of wallet, that it's more than for them to bother to ship it to you. Okay, and to deal with an infrastructure for returns be clear Amazon's thought of this. This is a patent for anticipatory shipping from way back. So from Amazon, from way back in 2013. You think, wow, they've thought of this for a while. But then do you think from way back in 2013? How come it still isn't there, the prediction technology seems to be getting better and better. And yet, this idea of a disparate shipping hasn't quite happened yet. More generally, when you look at AI investments, the vast majority of companies who invest in AI are disappointed. They say they still haven't seen a financial return from our AI investments. And so even more companies aren't even bothering to adopt AI because it's expensive, and 90% of the time you don't get anything out of it. So why would we even bother? And how does it make sense that all these smart people are saying AI is the new general purpose technology, it's the new electricity. If the vast majority of companies aren't getting much out of AI, tell you a little bit about electricity. Edison's patent for the electric light bulb was an 1880. Tesla's patent for the alternating current motor 1890. If you were paying attention in the 1880s, it was clear that electricity was going to be a big deal. It was clear that it was going to change the way we lived and the way we worked. But it wasn't until the 1920s that the median household, the median factory, had adopted electricity. It took 40 years from recognizing the potential of the technology until it actually had an impact on most people's lives at home and at work. What took so long, this is what a factory might have looked like in the 1880s. You can see that there's there's a steam engine at the center of the factory and all the machines are connected by these belts to the steam engine at the center. In a factory, and if you remember your high school physics, you may or may not, the longer those belts are, the more energy is going to dissipate. And so the organization of the factory was such that you put the most power hungry machines near the power source. And you put the least power hungry machines as far away from the power source as possible, you organize the factory around the needs of power. And so what happened in the 1880s and 1890s, electricity, a handful of innovators said, Well, you know what, for where I am, from my factory, I can save 5, 10, 15%, maybe, on my energy costs. And so I'm going to take out that steam engine drop in an electric motor, but I don't want to mess with my workflow, because that'll make me have to figure out all sorts of other things. So take out the steam engine, dropping the electric motor keep everything else the same. And they did, they saved 510, maybe even 15% of their energy costs, for the most part, that wasn't worth the bother. And so for most companies, most factories, never even adopted by 1900, we had well under 10% of us factories, adopting mechanical drives. But then around 1900, some people started to realize that electricity wasn't just cheap power, electricity decoupled the power source for the machines. And once you decouple the power source of the machines, you could do things differently, you could create a different kind of factory. And that's what we imagined, you think about the quintessential 20th century factory, Henry Ford and others. That is a direct consequence of this decoupling of the machine from the power source, because now what you can have is inputs going in one end, and outputs coming out the other with modular production and the modern factory system. It required a reimagining of what a factory could be, it wasn't just taking out the old way, dropping in the new way, at the same point, it was designing a new system. And at that point, once people started to realize what the new factory looked like, then we started to see an acceleration of adoption. With respect to AI, if AI is going to be like electricity, the transformational opportunities are thinking about new systems, not point solutions. Yes, we are kind of in the 1890s. We're in the between times where we see the potential the technology, but it hasn't happened yet. Because we haven't figured out what the organization of the future looks like. We know it's going to be different, we know better prediction is going to transform the way we live and the way we work. We just don't know exactly how. And that's why we see lots of people hesitating to adopt. And even among the adopters, they haven't quite figured out how to make money off this thing. The vast majority of the AI, as I just described are point solutions, there is an existing workflow, they identify a prediction within that existing workflow, they take out a human process, drop in a machine and leave everything else the same, because that's easier. But if you're leaving everything else the same, the upside potential of what you're doing is limited to the fraction of the overall value you're already creating. And so even Amazon with the recommendation engine, well, if you were in 2003, they would have recommended the Davinci Code. They didn't really have an AI based system, but they knew, Look, everyone wanted to read the Davinci Code in 2003. And so you just recommend the Davinci Code, and it works pretty well. Their AI system, yeah, 5, 10, maybe even 50% better than that. But it's all incremental. The real potential is imagining a new kind of work. These 10% of firms that have reported a benefit are imagining new systems. Almost everybody else is saying I don't want to imagine a new system. I want an easy when I'm going to look at my existing workflow, take out some process dropping a new one. But I don't want to tell anybody that I have to do anything differently, because that's hard. So here's an example of a system solution. If you want to drive a taxi in the City of London, it is three years of school. It is a very difficult city to get around one way streets, all sorts of crazy stuff. And so in the first year of school, you're it's all maps, you're like studying in a classroom with maps. The second year you get to go on a moped and start to test yourself around the city and your third year. It's apprenticeship like around the city, still studying, learning your way around. because it is that hard to get from point A to point B, not really, it was that hard to get from point A to point B in London. But now, we have GPS systems that allow us to get around and figure out the best route at different times a day. And as the systems were originally deploying, about 15 years ago, some enterprising entrepreneurs said, Well, no who's gonna really benefit from this? professional drivers. If you're a cab driver, you can save 5, 10, 15% On time from get from point A to point B. And that's going to increase your income and make you better off. And so we had navigational API's used for professional drivers applied to the taxi industry, it helped. But there was a system solution there too. What AI could also do is make everybody as good at getting around the city as London as a professional driver, amateur drivers professional, that's a new way to imagine an industry and that is exactly what Uber and Lyft ended up doing. They reimagined what was possible, because of a combination of prediction technology, and the maps, and digital dispatch. How do you think through in your own industry where system solutions gonna come from? Well, one key starting point is digging into what is it that AI does, that creates opportunity, and also generate some disruption. Right now, when you make a decision, you have a prediction that you make in your head.
But you also have what we call judgment, the other part of a decision that you make in your head and they are combined. AI is going to separate the prediction from the judgment. And that is hard, it is uncomfortable. And the people who are good at doing prediction and judgment together might not be the same people who are good at doing judgment in the face of a machine prediction. Give you an example of what I mean by this decoupling of prediction and judgment. When you see the movie I Robot. This is a crab it looks like you've seen the movie I Robot. Okay, so you have it's not always the case when I teach my undergrads okay. But iRobot is it's a fine movie. It's excellent science fiction. Okay, here's why it's excellent science fiction. So Will Smith is the protagonist iRobot. And he hates robots. And you can kind of see where the movie is gonna go based on that. And there's this flashback scene about why he hates robots. Here's why. He and this little girl are in a car accident. And for whatever reason, they start sinking into a river. And it's pretty clear that both Will Smith and this girl are about to drown. And then a robot comes along and saves him and not the girl. And that's why he hates robots. Well, it was a robot so he could audit it. And he looked at why did the robot save me and not the girl. And the robot predicted that he had a 45% chance of survival. But the girl only had an 11% chance of survival. And that's why the robot saved him over the girl. 45 is more than 11. But then he says, Well, 11% was more than enough. And human being a human being would have known that. Well, that's a statement about judgment. That's the same, but what we care about our values our payoffs. I don't know that all we all humans would agree that that girl's life was worth more than four times his life. But understanding the relative value of things is not something that prediction machines can do. That remains inherently human. And that's judgment. But it's not easy. It's useful, it's convenient to have some ambiguity when you make one decision versus another that I don't know if was because I predicted that will be better, or because I think that person is more important than this person. It's just- it's nicer to just say, Well, you know, I don't know there's some ambiguity going on. But what AI does, it forces us into judgment, it forces us to decide what matters, what we value in the presence of a prediction. How does this example of this how does this play out? I don't know if you remember a few years ago in Flint, Michigan, there was this water crisis, they discovered massive quantities of lead in the drinking water in Flint. And it wasn't in every house. It wasn't about 20% of the houses in Flint, Michigan, that had lead pipes. The problem was they didn't know which 20% and there were no records of whether any given house had lead or copper pipes. And so the city started digging up house by house, and you know, cost 1000s of dollars and time to go you dig up somebody's pipes. And then if they're lead, you can replace them and if they're not lead, then you just put the dirt back on and go to the next house. It's a slow, expensive process. And so these two professors at the University of Michigan, built an AI to predict which houses had led and it worked there AI was 80% accurate. So the baseline was 20. So it was four times better than just digging randomly, which is what they were doing before. And it seemed like, Well, that was the end, hey, we finally found an application of AI. That makes life better. But then some people in the city started saying, Well, how come the people in the next neighborhood over there getting their pipes dug up, but my pipes aren't getting dug up? Or the people in the street, the next street over there getting their pipes checked, but my pipes haven't been checked. And they started to complain. They complain to the politician said, hey, hey, politician, how come? You know this neighborhood that voted for you? We're not getting our pipes dug up. But these other neighborhoods, they seem to be getting their pipes dug up? What did they do? This isn't fair, why should we listening to some professor from a different towns algorithm? That doesn't seem right. And so the politician said, You know what, you're right, that isn't fair. We're going to take the power away from these professors from the algorithms. And we're going to keep making the decision in the best interest of the people of Flint, which is to go the way we were systematically house by house, neighborhood by neighborhood potentially started with the people who voted for me, hard to really tell if that actually happened. And the success rate plummeted. It went from 80% to 20%. And this is consequential. It meant that 1000s of people in the city of Flint, were now drinking toxic water. And they should not have been drinking toxic water. But the story didn't end there. Because it's so clear, given the predictions, what the correct judgment is, it was clear that the politicians did not have the correct judgment. So people sued. And the people got the power back, a judge came along and said It is now the law in Flint, Michigan, to listen to Jacob and Eric's algorithm, that algorithm that AI based at the University of Michigan, it is law in Flint. And then the company grew, and it's now used in dozens of cities, around the country and around the world. And of course, once they're using the algorithm, again, the success rate shot right back up. So this is the case where AI because it decoupled the prediction from the judgment, the politicians ambiguity about we don't really know where the lead is. So we're gonna dig in the neighborhoods we want to dig in first. That power was taken away. And it was given to the people who ultimately were the ones who were drinking toxic water and needed their pipes replaced. Story of decoupling prediction and judgment. So AI helps make decisions that decouples the prediction from the other parts of the decision. I imagine the question most of you are thinking about is should I worry? Is this coming from my job in my industry, it's hard to tell on a one on one, whether we're going to exactly how that's gonna play out. But here's, here's a framework for thinking about it. Think about what you do. Think about the mission of your organization. And try to understand how much of your day to day activities serve the mission. And how much of your day to day activities exist, because you fail to deliver on the mission. For example, this this is by many accounts, the best airport in the world. This is Seoul, Incheon Airport. It is pretty spectacular. It has great shopping, great restaurants, this massage chairs is a fantastic Hotel. There's greenery is a multibillion dollar place, that if you're at an airport, it's about as wonderful as it could possibly be. But this is the airport all of you would rather fly through. This shed, I'm told is the private terminal at Boston Logan. This is how the super rich fly. They don't fly in multibillion dollar airports. They fly through sheds that don't have any services. They have a magazine rack with the same magazine over and over and over again. How does that make sense? Well, the reality is no one wants to spend time at an airport. The mission even the mission of Seoul, Incheon Airport, is to ensure smooth air transportation, and the vast majority of the billions of dollars they spent on building that airport are dealing with the fact that they fail, that they fail to deliver on smooth air transportation, you're stuck at the airport, because you don't know how long it's going to take to get there and through security into your gait. And because you're stuck at the airport, they try to make it as pleasant as possible. But their procedures are based on failure. And if you had a good prediction about how long it would take to get to the airport through security into your gate, you do it the superrich, do, right? You just step right up, walk in the plane and take off. This happens industry, by industry think through how many of your standard operating procedures are dedicated to the fact that you're not delivering good customer service. And whenever that's happening, you should worry. And then you think through Well, if we had better prediction, is someone else going to come along and serve our customers in a way that we overwhelmingly fail at? Imagine you go to your doctor, your doctor looks at your blood work, whatever your symptoms and said, Well, there's there's a 5% chance that something catastrophic is going to happen to your health over the next year. If that happens, we will will write you a check for half a million dollars. And okay, we'll see you next year. Well, that's not right. That's not the way things are supposed to work. You're supposed to tell me what's wrong with me and provide some kind of treatment plan. But okay, luckily, luckily, that's not how medicine works today. But it is how home insurance works, right? Your insurance company has all sorts of data about your house. And they use that data to predict the risk that something catastrophic is going to happen to your house. And then they say we're providing peace of mind against that catastrophic loss. Because when it happens, we'll write you a check.