GenAI best practices and use cases | Justin Reock

[Intro music] Thanks, everybody! Very happy to be here. We have a lot of content to get through in a relatively short amount of time. Unfortunately, no time for questions, but I will be hanging out here. I'll also be at the Stockholm show. So, hopefully, we get a chance to hang out and talk a little bit more about GenAI. So, we're going to talk briefly about the current impact of GenAI. We're going to look at some challenges that a lot of organisations are facing right now when it comes to adopting this technology. We'll look at some adoption strategies, some things that, especially as leaders, we can take away and hopefully implement with our teams. We'll look at a few interesting SDLC agent integration use cases. A lot of what you're going to learn in the meat of this session is applicable to both coding assistants like Cursor and Copilot and things, and also agents, when you build prompts for agents. So, keep that in mind when looking at some of these use cases. We're going to go over some prompting best practices and some high-impact use cases. This was part of a study. This was a guide that we actually built based on interviews and surveying developers to try to figure out what some of these really high-value use cases were. So, it's all kind of based on data. And I'll explain briefly that study methodology, and then we'll go into some next steps. Okay, so Gen AI is impacting development, right? We don't know necessarily if it's good or bad yet, but we look at some of these industry reports, for instance the DORA report, that came out in April of this year, which saw modest but positively-leaning results when we looked at industry averages. So, we saw that 25% increase in overall AI adoption was associated with a 7.5% increase in documentation quality. Okay, good. A 3.4% increase in code quality. Modest, but at least not trending in the opposite direction, right? A 3.1% increase in code review speed, 1.3% increase in overall approval speed, and a 1.8% decrease in code complexity. Okay, seems fairly innocuous. Until we break it down by company. [Chuckles] These are per-company metrics looking at the DORA metric for change/failure rate. What you're seeing here on the top are companies who actually increase their change/failure rate by 2%, which doesn't sound like a lot until you realise that the industry benchmark for CFR is 4%. That means 50% more defects than the industry benchmark. But then you also have these companies on the bottom who have decreased significantly the amount of defects they're shipping. So, 50% less decreases. We can't trust the industry-average data that we're seeing right now. But what we can do is work really hard to be the type of culture and company that's on the top of this graph, and do our best to be there. It's just not evenly distributed right now. Some organisations are seeing very positive impacts to KPIs. Others are struggling with overall adoption and even seeing these negative impacts that we saw. And we really wanted to set out and discover the differences. What is the difference and what can we do to help companies improve? At DX, we measure developer experience and productivity. So, we have a nice wide view of how a lot of companies are doing. Obviously, we're looking at correlating this AI usage to these foundational productivity and experience metrics, but we also want to be able to provide materials and education that can help people move those metrics in the right direction. One of the biggest indicators that we found, though, in talking to companies that are struggling is that there's just a lack of enablement and overall education on best practices and use cases. Not only do we have to provide training for our engineers, we have to give them time to learn and experiment. This is new tech. You can't just turn it on and expect everything to just work great. There's actually a lot of nuance to the way that this technology works. [Audience applauding] [Outro music playing]