Innovation in infrastructure

Cheryl (0:00:06): Go to the CNCF website and read the white paper, it will take you 15 minutes and it is really worth it.

Marc (0:00:21): Welcome to DevOps Sauna Season 4, the podcast where technology meets culture and security is the bridge that connects them. Welcome back to the sauna. I'm thrilled today to be doing another post game from the DevOps Conference Global that we had in London in March. And we have one of our lightning talk speakers today, Cheryl Hung, the Senior Director of Infrastructure and Ecosystem at Arm and founder of Cloud Native London. Hello, Cheryl. Welcome back to the sauna.

Cheryl (0:01:05): Hello, Marc. Thank you so much for having me. And I'm so happy to be here. 

Marc (0:01:09): It's always a delight to speak with you. And once again, I've got my usual cohort. Are you a usual or unusual cohort actually, Darren? 

Darren (0:01:18): I like to think of a little bit of both, but definitely leaning towards unusual. 

Marc (0:01:22): The right amount of unusual, I think. So Cheryl, we had you on a lightning talk talking about key trends shaping the future of infrastructure. And one of the things that was brought up was, I believe is a Cloud Native AI white paper. I think it was in pre-release during the DevOps Conference. Is there anything you can tell us about that? 

Cheryl (0:01:43): That is right. The Cloud Native AI white paper is out now. It's available through the CNCF website. I really strongly recommend that everyone go and take a read of this. I think it's a really comprehensive guide to AI as well as Cloud Native. And ironically, I did my undergraduate thesis quite a few years ago using AI. And at the time, AI was still not a serious thing. Do you know what I mean? You know, it was like this thing that was big in the 70s, but, oh, AI will never get real, never be useful. And then suddenly now, you know, AI seems to really have just been such a hype mode. I think I said during the lightning talk, if you're not on the AI hype train, now's the time to get on. So if you really want to start from scratch reading about Cloud Native AI, I really strongly recommend checking out this white paper. 

Darren (0:02:38): Can you give us something like a preview? What can we expect if we go and look into this white paper? We saw that the nice chart already that you stole from it for your lightning talk, but what else can we expect to see? 

Cheryl (0:02:51): Yeah, it goes into a lot of detail about the challenges that you're going to run into if you're trying to run a large-scale AI system. So the training, which is perhaps the easier part of it in some ways, and the inference, and there's loads and loads of different aspects that you can take from this. So just to give you an example that I had in my head, often you want to run inferencing as close to the users as possible for better user experience. So that means you want to try and run things close to the edge, like on edge devices. And edge itself is not a solved problem. So there's all these layers of how do we get to this place where we want to be with Cloud Native AI? And something else that stuck in my head was the data governance and policy side of it. So AI requires a huge amount of data. That data is subject to a lot of different rules in lots of different places. None of these things are already built out, but companies are legally responsible for them. So all of this stuff is brand new, not solved yet. Very, very interesting, in my opinion. 

Darren (0:04:02): And it sounds like it might actually be quite interesting from a European compliance standpoint with the new EU AI Act that's being put into force in the not-too-distant future. So it sounds like something that could provide some useful, at least, guidance towards that. 

Cheryl (0:04:18): Yeah, there was a guy that I met at KubeCon a couple of weeks after. They come from a French company, and they're really focused on AI within Europe, really. Because the base models right now, like GPT, are generally owned by the big US tech companies. And so there's this feeling that, you know, in Europe, perhaps we should also maintain some level of control over our AI infrastructure if it's going to be such a big driver of the future. So this is definitely, yeah, something that is really important for all of us. 

Darren (0:04:53): It means we've got this interesting divide, though, where we have the large models being run by the US companies and all the legislation in the EU countries. So we need to find some way of getting those things together. 

Cheryl (0:05:07): Yes. One of my friends said, slightly sarcastically, “The EU's a little too happy about bureaucracy and regulation, you know?” Yeah. But it is up to the companies to figure out a way to adhere to this regulation. I think that's the reality of it. And of course, things like GDPR have been overall a net positive. 

Marc (0:05:28): I think it's interesting as well that, like I see, especially in the Nordics, where the languages are a little bit smaller, building LLMs in the Nordics with Nordic languages has been a huge advantage and a huge business here. So I'm kind of just a little curious if having more emphasis on LLMs in the EU might enable better access for European-based languages. 

Cheryl (0:05:51): I'm curious, like, can you elaborate what you mean by Nordic languages?

Marc (0:05:55): So for example, Finland, five million people. So the best LLMs in Finnish are built in Finland. So Sweden, Norway, similar types of cases. Iceland, even a more difficult type of situation. So when I think about LLMs, you know, human lexical language, I'm curious if doing more work on LLMs in the EU might be a big benefit compared to the US, which is going to have a little bit more focus on American English, maybe British English. 

Cheryl (0:06:28): I think it's definitely good for the languages. It's good for the people in the countries, but it's also difficult to always be the small, you know, the minority, right? You're not going to have as many people working on it. 

Darren (0:06:42): To be fair, though, I think we're having this conversation a little bit late. I was in a lecture the other day, and the lecturer played a AI-generated rap about a city in Finland in Finnish. So these models are being effective on smaller local languages even right now. So, yeah, while it will be better, but it's something that's already being developed. I remember having this conversation in 2018 about Finnish language models just not existing. But in 2024, they definitely do. 

Marc (0:07:18): Yeah, we're kind of off on a tangent, but I mean, Apple Translate still doesn't have Finnish. It doesn't get any bigger than that. But you brought up something interesting that I was thinking of, and maybe you can help change some perceptions, including mine. I'm a little, perhaps, late. So when I think of Arm, I think of the greatest technology on the edge, right? So Arm has always kind of, for me, been, you know, every mobile, everything that I've worked on for the last 30 years, you know, has been around, or I forget how old Arm is, honestly, but for a very long time has been based upon Arm architectures. So then I kind of made a note, like Arm and infrastructure, isn't Arm mostly on the edge? 

Cheryl (0:08:01): Arm's main strength right now is definitely mobile devices, now perhaps expanding onto other kinds of edge devices, you know, laptops as well. But Arm has always been about energy efficiency. And it turns out that that's very good if you have a small device with battery life, but also very good if you have a massive data centre using huge amounts of electricity and energy to run it. So although the context might be different, actually the benefits from it kind of translate very well from edge devices to mass infrastructure. 

Darren (0:08:36): On that subject, do you find you see a kind of switch in adoption? Because, I mean, I have on my desk a load of many devices with Arm processors built in. So I'm always struggling with software that's been built for AMD architectures. But do you think now that you're shifting over to these data centres, we'll start seeing higher adoption for Arm processors among popular software packages? 

Cheryl (0:09:05): Absolutely. That's actually one of the main challenges that my team, the ecosystem team, is looking to solve. And we've made a lot of progress over the last few years. Five years ago, as you said, a lot of software just didn't support Arm at all. There was no option. But a lot of the work that my team has been on, has been on things like compilers to make it really easy for popular software packages to offer as a support as well as X86 support. I don't have a good stat for you off the top of my head, but I can definitely say that, you know, if you look at the top 50 most used software, then most of them will support nowadays. And we're trying to expand that number as wide as possible. And that's part of the reason that I'm really interested in this ecosystem challenge. The launch of, say, Graviton, I think that was four years ago now. But that's just the first stage. You know, getting those Arm processors into a data centre is one thing. Getting developers to adopt it is another thing entirely. And that means you have to have support for all the popular open source and commercial software that they're going to use. I'll point out one other trend that I thought might be helpful here. And I've been vaguely aware of Wasm for a few years now, WebAssembly, but it's always felt to me like a solution in search of a problem. And then this last KubeCon, I went around and I talked to a bunch of folks and Wasm now is really this enabling technology that makes it easy to move software from, you know, inside the data centre to the edge and to move it from Windows to Mac to Linux and to move it from x86 to Arm and back and forth. So I think Wasm is going to be a super interesting trend. I think it's going to make migrations far easier in the future.

Darren (0:10:57): I think you might be right with that. My thoughts on Wasm come from the security side of things where it does have some advantages in things like hiding evidence of malware, phishing attacks, this kind of thing. I've seen some interesting use cases on that side. That's going to be very interesting to see how, I mean, I don't think it's a bad language or a bad evolution. It's just, we have this trend to where security tends to follow along with the change. So there are these like kind of these instances where the reactive way we follow along with these things doesn't always catch up. So it's going to be interesting to see if we can handle that this time. 

Cheryl (0:11:36): I have great respect for security folks. I think what you do is so challenging. Things change so fast in this industry. Yes, security folks just have to be knowledgeable about everything, be really smart about how they think about things. 

Darren (0:11:50): There was once an instance where the rules had actually changed by the time I finished giving a presentation on the rules I was talking about. It was actually regarding the AI Act, which I was saying that the EU had been passing about for so long that, no, it wasn't going to come out soon. It was delayed and then they passed it. And I was wrong by the time I finished speaking. 

Cheryl (0:12:14): The irony. Did you find out at the end of the talk or did you find that out later? 

Darren (0:12:18): It was it was pointed out to me at the end of the talk. 

Cheryl (0:12:25): Wow. Wow, can’t get more timey than that.

Marc (0:12:27): It's funny. We often talk about people who ask questions to tell you how smart they are rather than to actually, you know, question or provoke a discussion. But in this case, actually, I guess that could be a good one. One of the things, Cheryl, that I've seen you speak a couple of times and I don't know, maybe I embarrass you a little bit. But the first time I saw you speak, we had a difficulty in the venue and there was a PC that was hidden under the stage that the stage crew wasn't even aware of. And it was a relay from your laptop to the AV system. And you had to go with no slides. And then eventually, I think they took one of the cameras and they focused on your machine on the screen of your laptop so that you could give the presentation. And then when I saw you speak at the DevOps conference in London, you did a lightning talk. And not only did you nail it, but people remembered you and they remembered, you know, the confidence of the way that you spoke, as well as, of course, the content. And I think it's so cool not only to do this type of work, but also to get up and to tell other people about, you know, what it is that you're working on and to kind of inspire them even as a human. Is there anything about, you know, like preparing for a talk or especially like a lightning talk, which can be really tricky? You do it in five minutes and you're kind of bumped off the stage. Is there anything you'd like to tell our listeners about, you know, stepping up or preparing for this type of thing or how does it make you feel or these kinds of things? 

Cheryl (0:13:58): Well, first of all, thank you very much for the compliments. It's nice to know that people remembered me and what I spoke about. I would say that the first time that I did any kind of speaking on technical topics, I was a hot mess. I'll be honest. I was so nervous and I stumbled over my words. I had no idea what I was doing. But I think like anything else, like I think it's a skill that anyone can get better at. I spent probably two years watching videos back of myself, studying other speakers that I thought were good public speakers in order to develop that. Now, I wouldn't necessarily say I get on stage and feel confident. I would say I get on stage and feel joy. You know, for me, it is such a rush of adrenaline. And it's fun to get up there on stage and it's fun to tell people this is what we're looking at. These are the things that I find interesting. I don't think you have to be a world class expert to get on stage. I think you just have to get up there and say something that you think is interesting. And other people will usually embrace it and feel that energy that comes from you. But I really feel strongly about it because I run Cloud Native London, which is a meetup with over 8000 members in London. And I've always strongly encouraged first time speakers, women, minorities, just to get up there and get in front of people and start talking and say, you know, you don't have to be perfect first off. You just have to get up there and talk about something that you enjoy and other people enjoy it as well. 

Marc (0:15:25): People want you to succeed. And the thing is that it's difficult to kind of judge the value of our own talks sometimes because we think, oh, well, everybody ought to know this. Or, you know, I might be just saying something that people don't understand. But chances are, if it's interesting to you and if it brings you joy to get up and speak about it, then it's going to also bring someone joy to listen to. 

Cheryl (0:15:48): Exactly. Exactly. And other people are very friendly, very nice. I think we're in a very nice community of people and people are very forgiving. I've made absolute tons of mistakes over the years, as you said, like that time when my slides just, I didn't have slides, you know, and I had to get up there and wing it with, you know, what I had in my head. And that was it. I've had times when, you know, the mic has completely not worked at all and I've just had to shout. Gosh, I don't even know. Loads and loads and loads of things come up and people will know, you know, this is not a reflection on you as a person. This is stuff that just happens. So, yeah, I really strongly recommend that people get up there and, you know, it's very good for you, your career, your confidence. And it just takes time, but it's a skill. Anyone can learn it. 

Marc (0:16:34): Absolutely. Not exactly within your trends, but one of the things that I was thinking of when your talk was about the future of infrastructure and we're talking about portability between data centers and developers and whenever I think of Arm, I also think of, you know, kind of convergence devices, things like this. But do you have any viewpoints on how organizations are changing due to, you know, AI's influence on infrastructure and GitOps influence on infrastructure? And do you have any, you know, thoughts there or how do you see this is going? 

Cheryl (0:17:08): Let's start with the second one that you mentioned, GitOps. GitOps has been a trend for a couple of years now. GitOps is this idea that you manage your infrastructure entirely within Git. So every change that you make to the infrastructure is through a file that is checked into Git and it's version is changed and everything. And I think this is not a controversial idea now. Would you agree with that, Marc? 

Marc (0:17:37): Well, it's not controversial until you have an initiative within an organization to roll out GitOps that didn't come directly from the infrastructure team. 

Cheryl (0:17:46): OK, interesting. I feel like I've always seen it from the infrastructure team. So you've seen it from other? 

Marc (0:17:52): If I think about some enterprise level transformations whereby the CTO will come in and interview people and get a good understanding of what's going on in the organization, where the strengths and weaknesses are, and then realize, for example, maybe they have a really well running ticket op system for infrastructure and it has everything that GitOps has except for the API. And maybe they even have, you know, their rate limits and everything is kind of well orchestrated. But there's still humans pushing the buttons, getting a Jira ticket and pushing the buttons. And then it's like, OK, guys, we'd like to help you understand, you know, here's how we use cross-plane and here's how we can, you know, automate what has been you manually running Terraform scripts and things like this. So sometimes there can be a little bit of controversy. 

Cheryl (0:18:44): Very interesting. I must say, I've really only seen positive examples. Perhaps this is my bias from the kind of companies that are organizations that I that I speak to. But I feel like most companies that have adopted GitOps are on the whole very happy with how it's worked out. And then the other one you mentioned, AI, of course, back to AI. I think the challenge of AI from an infrastructure point of view is scale. A lot of the things that we've worked on are not really ready for the complexity, the storage and the computational complexity that AI brings. So perhaps in some bigger organizations that already exist, but that's going to filter down to the medium and to the smaller organizations as there's more and more demand to solve various problems using AI. So, I mean, that's my overall thoughts on how that's changing. 

Darren (0:19:34): Yeah, and it's going to come down also to the availability of hardware and such when it comes to, I mean, practically no one's going to be buying the cards themselves to run these things locally. It's going to have a huge shift towards the cloud. Anyone who was hanging onto physical infrastructure, I assume is going to be pushed more and more to cloud as AI adoption increases. 

Cheryl (0:19:58): I think that's a fair point. And there's probably a handful of organizations, very large banks, which could and still want to run on-prem data centers. But yes, I think that is fair to say most companies will look to utilize the cloud for their AI. 

Marc (0:20:16): Yeah, within finance, we're still finding those that absolutely insist upon the service in the basement. Still. We have a fresh customer that's looking to do some some quite interesting things. But still, it's the on-prem service in the basement. 

Darren (0:20:33): Well, there's so much regulation, it just becomes so difficult to move anywhere but the basement. One topic we haven't covered just yet, I think, would be the one that you brought up in your talk, which is the trend in open-source licensing. We've seen a lot of the big names in open-source lately shift their licensing model from open-source to more business-oriented licenses. Do you have something to open up on that? Some more information you might want to share? 

Cheryl (0:21:03): Well, first of all, let's point out the actual trend that we see. So I see two directions in licensing. One is, as you say, going towards a more business-friendly license, typically saying you can share, you can read, you can make modifications to this code, but you cannot sell it. You cannot commercialize it and sell it without paying some kind of royalties. The other direction that I've seen is to move towards something like GPL, where if you use this, then it now kind of infects everything else that you touch, that that code touches, right? But that makes it also less attractive for organizations to use compared to something like an Apache or MIT license, which doesn't affect the libraries that touch it. I have mixed feelings about this. I understand the companies that are building this open-source often are doing it because they need to survive as companies and they need to be able to commercialize it. They need to make money off it in order to pay maintainers and contribute towards the open-source ecosystem. But I'm also something of, I don't know, optimist or maybe optimist is not the right word. What's the opposite of a pragmatist? Idealist, maybe? An idealist at heart. You know, I like working in open-source for the sake of open-source. I like doing it as a general public good. And I come from a development background myself, and therefore my perspective is I like being able to use open-source for what it is without those economic realities behind it. So, yeah. 

Darren (0:22:36): It's curious because, for example, HashiCorp, when they switched their model, they were worth, I believe, like five billion. So there's some truth to what you say, I think, that lots of them switch the models to survive. But then there are these examples that are just not that case, going from like the MBL license to the business source license in 2023, when being worth a considerable amount of money is. You know, I think it was a surprising change for everyone. Do you think that's going to be something we see more of? 

Cheryl (0:23:11): I do. I do think we will see more of it. I think that is the reality of small companies who have to support, who want to use the open-source model behind for most of their company's products. But I'm also very willing to have my mind changed. So I'm curious, what do you think, Darren? 

Darren (0:23:32): I would kind of hope it doesn't shift in that direction. I think we're just going to see a lot of shifting ecosystem if things are going to shift in such directions. I would like us to stop using tools that switch to those licenses, but that's because I'm a nerd and I would kind of like us to remain true to the principles behind open source. But I imagine we'll use what's, you know, as a community, we'll use what's good and what's easiest. 

Cheryl (0:24:05): Yeah, my fear is companies will decide to slow down or reverse the trend of using open source because of fears that this might happen in the future. And that would be a shame. 

Marc (0:24:18): The thing that scares me the most is, in the regards of open source, is how the label itself may not mean what it used to. And instead of thinking about, OK, is this really available? Is the software really available for anyone to use for any purpose, which is the definition of open source, right? Or does open source now mean, well, yes, it has a restrictive license. It can't be used for certain purposes. And, you know, all of these types of things. So changing what the label actually means kind of scares me a little bit more. And that's kind of where I see it's going. And, you know, like Amanda Brock was speaking with us recently and sort of talking about this and it took me quite a while to understand why many open source personalities or humans, maintainers of projects were so serious about the definition. And then once I kind of came around to understand that it's really one of the most non-discriminatory things, you know, in tech, anyone can use this for any purpose without any restriction. 

Cheryl (0:25:26): I do agree. I think perhaps there's more caution, you know, open source is not just open source. But as I said in my talk as well, it now is on the responsibility of people who are using open source to manage that risk, to look into what open source really means or what you use. 

Marc (0:25:46): Absolutely. Just kind of curious in in your work with infrastructure, are there any specific tools that are trending in your area other than, you know, AI in general or something? But are there specific tools that are coming up more often lately? 

Cheryl (0:26:04): Not from the perspective of how I work with ecosystem. My goal in ecosystem is to look at broad as possible and then sort of top 20, top 50, top 1,000 packages. That's how I think of it. And of course, things always change a little bit in terms of, I don't know, I don't know. Ruby seemed to be a little bit more popular a few years ago and now it seems to have gone a little quieter, that sort of thing. But I wouldn't say there were massive shifts in the specific tools. 

Marc (0:26:36): All right. I think we have a few interesting things that I recorded as we went along, Cheryl. I think one of the most important ones is, is what should people do now if they have not yet to find out more about cloud native AI? 

Cheryl (0:26:51): Go to the CNCF website and read the white paper. It will take you 15 minutes and it is really worth it. 

Marc (0:27:00): Cool. I have followed Arm for a long time and seen that Arm is usually about the mobile devices, but is Arm only about the edge? 

Cheryl (0:27:09): Definitely not. Arm is becoming more and more prominent in data sensors and in infrastructure. And that's the biggest portion, fastest growing portion of our business as well. It's very exciting time to be in Arm. 

Marc (0:27:26): Cool. And is Arm supporting software more than it used to? 

Cheryl (0:27:32): Absolutely. Absolutely. Compared to a few years ago, pretty much every one of the top open source software packages is going to support Arm. That's not to say, it might not be perfect. You might need to do a little bit of testing still, but it's there. Support is getting wider and wider. 

Marc (0:27:47): Cool. Is WebAssembly the future, Wasm? 

Cheryl (0:27:52): I'm going to withhold my judgments on this one just yet. I think it could be the future, but I'm not going to commit to that yet. 

Marc (0:28:04): Cool. All right. Is open source going away? 

Cheryl (0:28:08): Oh, no. Open source is definitely not going away. Open source is getting more refined and maybe we need to look into it more closely, but it's not going away. 

Marc (0:28:17): All right. And would you tell our listeners, and I'll put it into my terms. Should our listeners follow your inspiration, go to meetups, maybe submit to a call for papers and find their way on the stages to tell their story? 

Cheryl (0:28:33): Absolutely. I think it's great for everybody to feel that they can share what they know, to understand that other people will also be interested in their journey. And yeah, any of the meetup organizers that I know, tend to be very friendly people. But if you are in London sometime, come and talk to me. Cloud Native London once on the first Wednesday of every month. You can find us on meetup.com and come and hang out and chat with me. And I'd be very happy to give you some more pointers. 

Marc (0:29:03): Awesome. Thank you so much, Cheryl, for coming on the podcast today and just having a lovely conversation with us. And it's always a privilege to get to speak with you or to listen to you speak. 

Cheryl (0:29:15): It's been an absolute pleasure. Thank you, Marc and Darren, for inviting me. 

Marc (0:29:19): All right. And thank you once again, Darren. 

Darren (0:29:20): Thanks, Marc. 

Marc (0:29:21): OK, we will see you back in the sauna next time. Thank you and goodbye. We'll now give our guest an opportunity to introduce himself and tell you a little bit about who we are. 

Cheryl (0:29:36): Hi, everyone. I'm Cheryl Hung and I'm the senior director of infrastructure ecosystem at Arm and also the founder of Cloud Native London. You can find me online at oicheryl.com. I love to chat about all things software, cloud native, open source. And yeah, come and find me and hang out. 

Marc (0:29:58): Hi, I'm Marc Dillon, lead consultant at Eficode in the advisory and coaching team, and I specialize in enterprise transformations.

Darren (0:30:06): Hey, I'm Darren Richardson, security architect at Eficode, and I work to ensure the security of our managed services offerings.

Marc (0:30:13): If you like what you hear, please like, rate and subscribe on your favorite podcast platform. It means the world to us.