MindSET podcast debuts with a dip into AI and ML

Ron Keesing: Don't buy into the hype, right? AI machine learning isn't going to solve all your problems. It's not going to come in and transform every aspect of your business, despite what it says on the cover of Time magazine. But it is a very powerful tool that you need to understand how to use in order to succeed and understand how to bring into your business.

Bridget Bell: Welcome to MindSET, a Leidos podcast. I'm your host, Bridget Bell.

Meghan Good: And I'm your host, Meghan Good. Join us as we talk with pioneers in science, engineering, and technology, to understand their creative mindset and share their stories of innovation.

Bridget Bell: We're diving in with a topic that's getting a lot of attention right now, artificial intelligence and machine learning. We sat down with Ron Keesing to discuss how AI and ML are solving really complex problems.

Meghan Good: He shared a lot of interesting examples of where AIML is being deployed today across sectors from defense and intelligence communities, to civilian agencies and regulated industries like electric utilities.

Bridget Bell: We also got to speak with him on degrees of intelligence and how human reasoning and processing fits into AI. Believe it or not, AI does have limitations. So we got him talking about those.

Meghan Good: I really enjoyed that his background is in neuroscience and mathematics, which seems a perfect mix for AIML, but also my background's in cybersecurity, so I was especially interested when the conversation turned to adversarial AI and what Ron predicts for the future.

Bridget Bell: We hope you enjoy listening to our conversation and hearing Ron's mindset of innovation. Keep listening until the end, when he offers advice on how to get away from the hype and be successful with AI.

Meghan Good: Okay, let's get started.

Bridget Bell: Welcome to MindSET. Today we are here with Ron Keesing, vice president and director of artificial intelligence and machine learning. Ron, thank you for being here with us.

Ron Keesing: Oh, it's a great opportunity. Thank you.

Bridget Bell: So before we dive in, tell us a little bit about your background and how you came to this role.

Ron Keesing: Sure. So I've actually been working in AI and machine learning for a very long time. I started my career back in the early nineties, working on autonomous spacecraft actually at NASA, and it's been a lifetime fascination of mine working on both natural human and artificial intelligence. And it's been exciting to watch the field develop over the last two decades, and especially over the last five years, move really quickly into places where we can actually apply machine learning to solve real world problems.

Bridget Bell: So what is artificial intelligence and machine learning? Let's start with the basics.

Ron Keesing: Artificial intelligence is the development of systems that use human-like intelligence to solve problems.

Ron Keesing: Now that's kind of a loose definition. It means many things to many people, but I like to think of it as really meaning that these systems can behave in flexible ways under uncertainty and really make decisions like humans make decisions. Machine learning is a subfield of artificial intelligence where rather than telling machines what to do explicitly, we let them learn how to behave in an intelligent manner by looking at data, by learning from data. And it's actually a very exciting technique that's led to rapid advancement in the field. Most of the systems that we're familiar with today that actually have that kind of intelligent behavior, whether it's self driving cars or Alexa, all those applications that are used to an artificial intelligence are all based on machine learning. They're based on having the machine learn to have intelligent behavior by learning from data.

Meghan Good: So are there limits to what artificial intelligence and machine learning can do? Because we're hearing a lot of hype right now.

Ron Keesing: Sure. We are. We're sort of in many ways at the peak of a hype cycle around artificial intelligence and machine learning. And it's important to recognize that because we use machine learning, which is based on data, really our ability to build powerful artificial intelligence systems is limited by problems where we have the right kind of data. And it's not just the right kind of data that's important, the other part is a clear ability to define objectively what good behavior looks like. Because what machine learning does is teach a machine to minimize the difference between what it does today and what the good behavior would look like. So for example, we've had tremendous success in fields like Go and chess where the objective quality of winning a game is really easily measured. It's actually much harder to build machine learning applications and problems where we can't so clearly define a single thing to measure that means the system is performing well.

Ron Keesing: Another real big limitation is that artificial intelligence systems in machine learning systems, as we build them today, don't tend to generalize well into other domains. I mean, it's actually very interesting. We humans take knowledge, we learn in one domain and we apply it into other domains very seamlessly, but today's artificial intelligence systems can't do this at all. They can't draw on context or experiences or data from a different domain to actually understand what's likely to happen in a new kind of problem and in a new kind of domain.

Ron Keesing: And finally, one of the other big limitations and challenges of today's systems is that because they learn from data, their performance can only be as good as the data that we give them, and it can actually learn bad things from the data as well as good. For example, one of the things that we're learning is that machine learning systems can learn to not only replicate, but actually amplify certain kinds of human biases.

Ron Keesing: So if machines are actually learning to say, make a decision about who to parole as a prisoner or who to identify as a job applicant, if the humans who create that data have some kinds of biases, cultural biases, racial biases, various kinds of prejudices, for example, the machine will not only learn to replicate those, but actually amplify them and make them worse. So these are all kind of key limitations in the way we do AI and machine learning today.

Bridget Bell: It's so interesting that humans can pass their bad behaviors to AI and ML.

Ron Keesing: Yeah, it's really a profound challenge, and it's something we're working on. Techniques, we're funding research on how to actually build systems that can not only detect when we build a system that has these biases, but automatically remove that bias from the system. But it's a challenge when you build a system like this.

Bridget Bell: So speaking of challenges, what are the other biggest opportunities and challenges with AIML for government agencies today?

Ron Keesing: That's a great question. And it's one that we get from many of our customers. The first place I like to start is to think about it from the perspective of a government customer, who's facing the onslaught of data that they face today. Almost all of our customers will tell us that they're getting far more data than they can possibly process or know what to do with. For example, I work with a government customer who, when we went in and helped them analyze, they realized they'd only looked at less than 5% of their data ever. And faced with that kind of volume of data, they had a fundamental challenge. Now, the beauty and the opportunity afforded by AI and machine learning is that you can turn that onslaught of data from a challenge into actually an opportunity because you can use all that data to actually train the machine how to improve.

Ron Keesing: So the machine can use all that data, actually learn to perform better and better if it's trained in the right ways. So, that's sort of the greatest opportunity afforded I think by machine learning is just turning the data challenge that many of our customers face on its head into an opportunity. But I think there are a lot of other things that we have the opportunity to do using AI and machine learning for our customers as well. For example, there are many, many tasks that humans perform today that can be automated using techniques like robotic process automation. So we can free up human time from doing really not very value adding tasks like copying data from one application to another, into really valuable tasks like doing higher cognitive processing on the same information. So there are great opportunities for automation.

Ron Keesing: Now, let me tell you about a little other interesting thing you can do, when you can apply automation into systems and it comes from the program we're doing called Sea Hunter for the Navy.

Ron Keesing: So See Hunter is a fully autonomous vessel we've built for the Navy that's actually transited the Pacific all the way from San Diego to Hawaii, completely without human intervention. One of the interesting things, it's a great, tremendous opportunity to reduce manpower on Naval vessels and operate more cheaply, but another really cool opportunity is that when you build a system like that, that doesn't have human aboard, you can actually build the vessel quite differently and more efficiently than a human operated vessel would be.

Ron Keesing: So there are lots of different opportunities afforded by inserting AI and machine learning into real systems, but there are challenges too. One of the things we hear from many of our customers who've made first forays into AI and machine learning is that they've tried it, they've built some models, they've got some initial successes, but they're not actually generating any real business value from AI and machine learning.

Ron Keesing: They struggle from getting past the conceptual stage where they've shown that they have a model that can do some useful thing, like predict when failures of a component are going to occur and actually putting that into operation. So we at Leidos work with a lot of our customers on moving beyond what we call just this simple data science models into supporting a whole life cycle of AI and machine learning. And this is a journey for many of our customers we have to kind of walk them through.

Ron Keesing: So it's not just about, I can do a little bit of data science or build a model in a lab, it's about collecting the data, actually building something in a lab, but then moving it into production and then sustaining it through its entire life cycle. That's a challenge, I think, to any government agency that wants to deploy AI and machine learning. And it's typically a very large hurdle to go from what they can do in a data science environment where they just test out a concept to actually putting something out into the field, deploying it and sustaining it, right? That's the challenge that every government customer has and it's something that we work with them very closely on.

Meghan Good: So with that, Ron, you have talked about how you interact a lot with different customers, and I know within the company, you interact with people who want to apply ML or AI more broadly to some of their projects. We have AI-Palooza where there's the showcase of those across the company. What in your mind is some of the most exciting uses of AI that you've heard about recently?

Ron Keesing: Well, I'll tell you one of my favorite projects that we're doing over in our civil group, in the energy sector. So this is a really interesting problem that they brought to me. I didn't really understand until they described it to me, but you know, if you go to your local utility company and you ask them about how all of their power poles, what all the equipment is on all their power poles, they actually don't quite know the exact distribution of all of their equipment because when line workers go out, they move lines around and so on. So the typical electrical utility company doesn't actually know the location of all their physical assets. So some super smart folks in our energy practice came up with the idea of using AI and machine learning to actually be able to help automatically learn the whole structure of a local electrical grid.

Ron Keesing: How do they do that? Well, first we fly drones over an area and use LIDAR to actually collect data on where the power poles are. When we've got that data, we then take available surface level images of power poles from things like Google street view and other sources, look on those pictures of the power poles and actually classify all the electrical equipment on those power poles. And using these kinds of techniques with very high accuracy, we can locate where all the equipment is in a local electrical utility distribution network. And it's a great application of AI and machine learning to solve a really hard problem that would be incredibly human intensive to try to solve using just people driving around in trucks.

Meghan Good: I love that example because our electric utility infrastructure is aging, and I think like you said, utilities also don't know where they are because they're getting older and they were installed 70 years ago. And so to be able to bring new technology to infrastructure that is decades old and find that new solution, that's a very cool example.

Ron Keesing: Yeah. And it's very current because you know, we're looking at how the state of California right now is having to shut down huge areas of the electrical grid because they can't really monitor what's happening effectively. So if we can use AI and machine learning to automate our understanding of a grid like that, we actually could potentially be able to help alleviate some of the problems that a state like California is facing.

Meghan Good: And even further, I mean, being someone who lives out kind of in the boonies, when the power goes out, we're out for a few days. Right? So if there is more knowledge that those utilities can pull from and say, "Oh, well, the power is out here because of these kinds of pieces of equipment." I mean the faster time to being back up would be such a better value for me as a customer too.

Ron Keesing: Yeah, no, it's a great, it's a great application and we're really excited about it.

Meghan Good: Really cool.

Bridget Bell: Even with these interesting applications, AI is not a magic wand and like we talked about, there's a lot of hype around it. So what do you see are the key elements of a successful AI system or project? What can you do to make it more successful, move beyond the hype?

Ron Keesing: That's again, a great question and something that we really focus on here at Leidos, typically when we get together with a customer and work on an engagement, the key first step is defining the right problem that you want to deploy AI and machine learning against. A lot of the engagements that I and my team have are actually going out and sitting down with customers and really defining the right AI machine learning problem. Because a lot of folks will come in and tell us, we want to use AI and machine learning, but we don't know what to do. Right? And the first thing is defining a good problem where you have good data available and where, and this is the most crucial part, you're going to be able to measure the value that AI and machine learning brings. Because a lot of folks just want to come in and try things that don't actually deliver business value.

Ron Keesing: And at Leidos, we really want to focus on making sure that the AI machine learning solution we bring is going to be ultimately delivering business value to the customer. So once you've defined that kind of right, clear objective, a business objective you want to maximize, and you figured out how you're going to deliver a real return on that AI machine learning investment. Obviously the next step is you've got to make sure you've got a problem where data is available and where you've got the right to answer the problem. Sometimes this can be really creative. For example, figuring out ways to use maybe proxy data that's available from another source. At Leidos, we've developed a lot of solutions where we can use a combination of real and simulated data. So there are a lot of clever ways to use data to build models, but that is really a crucial next step of the problem.

Ron Keesing: One other real challenge for everyone in this industry is making sure you've got the right talent. I think people are aware for example of the fact that it's pretty tough to hire great AI machine learning talent. Depending on who you ask, it's near impossible. On the other hand, what we find is we've got actually a lot of great machine learning talent within Leidos, both folks who've been doing it for a long time and really enjoy doing it at Leidos because we give them the opportunity to solve really interesting problems, but also we've got a tremendous set of people with all the fundamental mathematical and statistical skills necessary and computer programming skills necessary, and we work on up skilling them to become expert practitioners of AI and machine learning because they were an applied mathematician or a statistician whose fundamental knowledge makes them eminently qualified to take up the field.

Ron Keesing: And then we help them get up-skilled, we give them problems and projects to work on to really increase their skill level. And then we've brought them together into an AI machine learning accelerator, so we can kind of make them available to our customers across the company and to work on the company's most important problems using AI and machine learning. Because, I will say, that access to talent to actually build the solutions is a big bottleneck for many organizations.

Ron Keesing: Then finally, as I talked about the ability to understand how to support the whole AI machine learning life cycle, having kind of models for how you're going to do that, that's critical to delivering the ultimate return on investment to the customer. Because again, there's this huge hurdle between saying I built a model for you and saying, I actually put this into production. I figured out how you're going to sustain, and I figured out how you're going to update your models over their whole life cycle so that you're ultimately getting the real cost savings, getting the more greater efficiencies, getting the higher performance that you've actually asked for in the first place.

Meghan Good: So switching gears a little, but kind of carrying on your thought of operating and maintaining these models over time, you know that I'm a cybersecurity person and I see AI as a new attack surface, really within any deployed environment. What are your thoughts on protecting the models once they're out there, once they're actually manipulating data that's coming across and actually probably the data itself, how are you thinking that we can really protect?

Ron Keesing: Yeah, it's such a critical insight that you bring there because it is absolutely another attack surface. Once you deploy AI machine learning models out into the field, there's a whole new growing area of research around how you can attack these models for different ways, by attacking the data by attacking kind of the inputs to the model, and trying to get the system to do things you might want to do. We call these different kinds of attacks spoofing. For example, if you try and give the system data that might fool it, poisoning by actually giving the model during training some data that will actually make it behave incorrectly. And now there are even these interesting attacks people use to try to, if you've trained a model, to actually try to reverse engineer the model and steal the data you use to train your model, which for example, in the world of, of health, where you might use, you know, privacy protected information or intelligence where you might be using classified data to train a model suddenly becomes this new security vulnerability.

Ron Keesing: So we work closely with partners and we fund our own internal research and development on ways to build certifiably robust AI machine learning solutions that can detect when they're attacked and defend themselves either by changing their behavior or by diversifying the way that they behave so that they kind of have security built in from the beginning. But it is absolutely critical when you think about deploying into any safety critical situation, any situation where a model is going to be exposed out into the environment, that you consider the fact that people can attack these models and that there's a whole growing body of literature that makes it very easy for people to do this. They don't even have to be great scientists or researchers. There are kits you can pull right off the web that explain how to attack a machine learning model. And again, at Leidos, we're working on how we secure those models so that once they're deployed, they're not going to be vulnerable to those kinds of attacks.

Meghan Good: That's very cool. And I love seeing that all the traditional terms, spoofing, poisoning, from the cybersecurity field are really playing out again in this new way for a different vulnerability, too. I'm looking forward to seeing how that goes, this machine to machine kind of combat. The responses being taken based on the data, based on the model. And then for all of us as almost observers of that activity happening and trying to shape it the best way that we can.

Ron Keesing: Well, I hope, and I am encouraged by the fact that in the field of machine learning we're taking on these challenges relatively early in the growth of the field. I think, we all have seen around the world of especially network security, that we all deployed out all of these models for communication and networking, that really were never secured upfront. And I think in the world of AI machine learning, people are talking about how can we build in the security layer early in the deployment of these solutions instead of leaving it as a bolt-on for later, with all the inherent problems we've seen in the world of network protection, where we've never really achieved full protection against the range of threats.

Meghan Good: Right. And I think that's probably something it's good to consider early and you can't be perfect and maybe we never will be, but it's a brilliant thing to consider at this point.

Ron Keesing: Yeah.

Bridget Bell: So AI is all about making predictions. So what are your predictions for the trends that we'll see for integration of AI and machine learning?

Ron Keesing: One of the trends I think we're going to see, and we already are starting to see it emerge, is a much greater focus on building systems that help humans and machines interact effectively. In the commercial world we've already gotten used to things like talking to our Alexa and so on, but in those voice driven systems, I think we'll see a much greater use of dialogue. Right now, current systems you say one thing to your Alexa and it's almost like stateless, like it can't remember the last thing you said. So I think we're seeing a much greater growth in the opportunity to have conversational technology. It will allow us to interact in a much more seamless and conversational way with visual interfaces.

Ron Keesing: People are playing a lot with how humans can interact with AI and machine learning, and even to some extent, explain how AI and machine learning are making decisions, or at least kind of have enough insight into how they operate, that they can interact more effectively with machines. So I think that focus on human machine interaction is going to be really big over the next several years. Again, I think we're going to see a lot of focus on as I talked about, going that journey from hype, I got some AI machine learning, to actually value to how does AI machine learning really deliver business value that transcends just saying I did it and it's cool, to actually what is the real business case?

Ron Keesing: And then sadly, I think it's probably not too long before we see some high profile failures associated with AI and machine learning, and we're getting these systems out there. We've already started to see accidents from self-driving cars and other kinds of things. And I talked about the fact that Leidos is working on ways to protect systems against attack and so on, but frankly, there are a lot of systems being put out there that are not adequately protected. So I think it's probably just a matter of time before we see a high profile attack or we see a failure of some system that's been put into a critical situation where AI machine learning is blamed for that failure.

Ron Keesing: So I think in the short term, those are some challenges I think we're going to see or some trends we'll see in the field. I think in the medium term, one of the things we're going to see, and this is Megan, towards your interest in the cyber world, is the integration of AI machine learning and cyber. I mean, we already see AI and machine learning used to try to detect attacks or find patterns in the data. I think what we're going to see though is essentially AI machine learning brought into the world of actually controlling our cyber defenses, because we're going to be dealing with adversaries who are using AI machine learning to attack us actively, not just to figure out what our vulnerabilities are and constantly be attacking. And the only way we'll be able to defend ourselves is by having AI machine learning actually be on the defensive side because things will happen far too quickly for a human to control. I really think that's a big trend we'll see.

Ron Keesing: And I also think we're going to see in the medium term kind of an intersection between the world of AI machine learning and the world of virtual reality in some really interesting ways. Over the long-term, it's really interesting, people talk about this problem of general AI, artificial intelligence that could potentially solve any kind of problem. And it's a really interesting idea. What's cool is that if you talk to researchers in AI machine learning, they'll say, "Well, that's 10 years away." And when they say 10 years away, what they mean is they have no idea how we're going to solve this problem. And it's still the case that if you talk to the smartest researchers in the field, they say it's 10 years away. Again, if you hear anybody say that, that just means we don't know yet. That's exciting, right? It means there are hard problems, less left to solve in this field. And it's a great field to be kind of a part of and working on because those general intelligence problems are just fascinating.

Bridget Bell: I want to go back to one of the earlier points you made. You talked about it a couple of times about how AI and machine learning is really a journey that from finding the data and implementation through actually finding the business value. So where are you seeing government agencies fall in that journey? Are there a lot that are close to finding that business value and coming through the journey or are they all still at the beginning?

Ron Keesing: I think it really depends on the customer and how they've chosen to approach the problem. Some customers have sort of divided off AI machine learning as a little problem area. They've set up a lab or set up some kind of discrete activity that's AI machine learning focused. And typically what we find there is when you find some really smart people and it's really exciting, and they're doing really cool work that gets published, but there's no way to flow the results of their work into real deployed solutions for their customers or for themselves. Right?

Ron Keesing: On the other hand, we have programs that are really exciting. We have one program we do for an intelligence community customer, where they asked us to integrate AI machine learning as part of a whole system. And in that program, we've actually deployed about 20 different machine learning algorithms as machine learning microservices, that process petabytes of data in a dynamic data pipeline. And that is delivering tremendous business value that's turned kind of a data challenge that they had from petabytes of heterogeneous data into this powerful discovery platform where analysts can actually find any data that they're looking for using a really simple and intuitive interface, all powered by AI machine learning.

Ron Keesing: So I think it really depends. We're seeing a lot of different results from our customers. Now the good news is because we have such a broad reach across customers and markets, we, as Leidos, are able to bring kind of best practices from where we're seeing it being done best for government customers to, as many of our government customers are ready to kind of take this journey with us and kind of talk about best practices, about what's worked in really successful deployments, and about what some of the pitfalls are that kind of keep AI machine learning from being deployed successfully as well.

Meghan Good: So you've been involved in this field, as you said for a very long time, and I'm sure have gone through your share of the hype cycles, but also your share of what the customer challenges are and where they are. To Bridget's question, are you still finding that they're the same kind of fears or the scariness of AI? Is that coming through or are people more in the ready to try stage or are they past that? You know, I would just think that as you're bringing any new technology, there's always the naysayers, right? So where do you see where we are right now?

Ron Keesing: I think we talked to people along every stage of their feeling of threat about AI machine learning. For some people, they look at it as a threat that's going to take their jobs away. Right? And for others, we've got early adopters who are leaning forward and coming up with incredibly exciting and innovative ideas for how they can transform business using AI and machine learning. So we work with people at every stage of that journey.

Ron Keesing: One of the things we find is that there's a model, I think, for introducing AI machine learning that really works well when you're having an organization that has a lot of people across every stage, right? And the first is when you bring in AI machine learning for a lot of reasons, it's really good to start with a model where the AI machine learning is just assisting humans do their jobs better, whether it's helping them find the data they're looking for.

Ron Keesing: You know, we don't even think about the fact that we use AI every time we use Google to go search the internet. Right? That's a model where we're using the computer to help do our jobs better. Right? But if you start kind of at the simplest where AI machine learning helps me communicate more easily than having to pull down my phone and search for a song, because I just told Alexa what I wanted. That's pure assistance. To a model where now maybe the machine learning is maybe helping me do a job differently than I could have before at a scale I couldn't before, because it augments my intelligence. So I'm still really driving the system, but now I can do tasks maybe I couldn't do. Like I'm searching through hundreds of hours of video, but the machine's helping me find all the segments where interesting happen so I can look at them and make sense of them. And instead of processing eight hours of video a day, I can process hundreds of hours of the important video because that's only eight hours of interesting material for me to review.

Ron Keesing: And then finally into modes where systems can become truly autonomous. Where we can trust that we can put a Navy vessel out into the ocean and conduct operations at sea, completely unmanned. It's really a journey. And if you try to jump right to the end, then those people who are concerned and feel threatened or rightly raise concerns about the safety and security of putting these solutions out into the field immediately, you fail kind of tripping up and not succeeding. So we've got this multi-step process of building an AI machine learning gradually, we find is really the best model that brings everyone along and actually kind of makes everyone's lives better through the application of AI and machine learning.

Bridget Bell: So it's so interesting because I think like we've talked about, some people do feel threatened by AI and ML, but as you talked through that, there's just such great opportunity for humans to be able to do their jobs better and more effectively and efficiently through these uses of technology. So I think those examples are really interesting.

Bridget Bell: Switching gears a little bit, I feel like we could spend hours talking with you on this subject. It's such an exciting topic and your passion for it is very clear, but let's wrap up with one final question of what advice do you have for the market?

Ron Keesing: We've talked a little bit about this already. Don't buy into the hype, right? AI machine learning isn't going to solve all your problems. It's not going to come in and transform every aspect of your business, despite what it says on the cover of Time magazine. But it is a very powerful tool that you need to understand how to use in order to succeed and understand how to bring into your business. So, my advice is choose a partner who understands how to guide you through this journey. Don't believe everything you read. Don't go to conferences and believe people who give you glossy marketing materials. Work with a partner who can show you that they've walked this journey with others and help them actually not just build something that looks cool, but actually get measurable business value from it because that's the hardest part of succeeding in AI and machine learning.

Bridget Bell: Very good. Well, thank you so much for your time. It was great speaking with you today, Ron.

Ron Keesing: And great speaking to you guys. Thanks.

Meghan Good: And thank you to our audience for listening to MindSET. If you enjoyed this episode, please share with your colleagues and visit Leidos.com/MindSET. And if you are specifically interested in AIML, please visit Leidos.com/AI.