Vibe coding meets UX: Can AI build great user experiences?

[Ray] (0:03 - 0:12) If the current LLMs are relying on the previous data, then they're going to be making the same mistakes that we’ve done previously. [Pinja] (0:14 - 1:34) Welcome to the DevOps Sauna, the podcast where we deep dive into the world of DevOps, platform engineering, security, and more as we explore the future of development. Join us as we dive into the heart of DevOps, one story at a time. Whether you're a seasoned practitioner or only starting your DevOps journey, we're happy to welcome you into the DevOps Sauna. Hello and welcome back to this DevOps Sauna. AI is one of the big topics we've been discussing lately. Today is not an exception to that. We've been talking a lot about the technicalities, we've been talking about any applications, we've been talking about the tools that are now supporting this. Lately also the human side of it, but user experience is one that we have not touched yet. And today we want to do that and we want to talk about how Vibe Coding and UX mix together. I'm not here to talk about this by myself, but I'm joined by my colleague Ray Pohjanheimo, who is a senior UX designer at Eficode. Welcome, Ray. It's nice to have you here joining us. We've been talking about Vibe Coding mainly lately because the use of AI assistance in software development has increased quite a bit in the past couple of years because now everyone can code. How do we understand Vibe Coding? What does it mean to a UX designer? [Ray] (1:34 - 2:13) Right. Yeah, that is a good question. Well, how I see it, Vibe Coding and UX design, how they go well, it will be faster and faster to solve issues that users might have because you can essentially produce code very quickly at the moment. So you could be able to basically skip the sort of pixel pushing process out of it. You wouldn't necessarily need to create the designs in Figma. You could go directly to code. [Pinja] (2:13 - 2:46) Yeah. We are now using LLMs to create actual functional applications. And somebody like me, for example, I'm not afraid to admit that I'm not a developer. I've tried to write Go a couple of years ago, but that doesn't get me a functional application. But with Vibe Coding, with AI tools, even I could actually be able to create one. Lovable is one tool that has been discussed in the UX circles quite a bit. Is that a commonly used tool? What is the experience on how Lovable is being used by our customers and industry? [Ray] (2:46 - 3:44) Yeah, I think Lovable is quite common possibly because the user experience of it is quite nice and you can fairly quickly and easily get something out and see essentially a demo of your application already in the interface. What I've been hearing is that more and more customers are directly building something with Lovable and then they might ask, for example, Eficode to check if everything is working okay and what could be improved. How could it be basically made into a full application? Because I suppose they also recognize that what they've built with Lovable could be like a minimal viable product, but not necessarily like the full product they want to release for their end users. [Pinja] (3:44 - 4:12) Because I guess that's one of the differences between a company actually releasing working software, a working application, and somebody, maybe a developer, doing in a small scale as a hobby project or a startup, for example, where the group of users might be very much more limited compared to bigger companies when the user experience needs to be taken even more into consideration. That's at least my assumption. Is that what we see? [Ray] (4:12 - 5:02) Well, I think user experience is most important in fields where there is more competition. So in contrast to that, if basically you're building an application or a product that is the only one on the market or it's an internal application, then the user experience is nice to have. But if the application is not easy to use, the end users might not have other choices, sadly. But in a more competitive market, and especially if you're a startup, then user experience is really important because you need to challenge the existing players and basically offer something that is better. And sometimes it's just the user experience that can be the winning recipe for that. [Pinja] (5:02 - 5:24) Yeah, that's a really good point. Well, traditionally, in the so-called traditional coding process, user experience has been very strongly linked to the process. But I also heard, Ray, that you have your own example app that you have Vibe-Coded. What was your process, especially now from a UX designer perspective? What was the app about and what was the process around it? [Ray] (5:24 - 7:06) Yeah, so I created this kind of a minimalist productivity app. I made it essentially for my own use because I personally didn't really enjoy writing to-do lists, but I did like the sense of achievement when you can cross off the items of the to-do list. So I just started with the idea, what if there was an app where you only write down goals that you have achieved and you can categorize them and so forth and so forth. But how I built this, I used Lovable for it, perhaps approximately five prompts in total I used for this. And I could already describe the visual style I was looking for, and made some small tweaks. I recall Lovable has this visual edits function as well, which kind of resembles tools that designers are more familiar with, where you can visually select something and then edit the components in question. So after I was happy with how the app felt and looked, I connected it to GitHub, then cloned locally, exported a web application out of that and then uploaded it to my server. I should say I know a little bit about coding, so it's not completely new to me, but I would imagine that this would be something that anyone could at least learn how to do quite quickly. Yeah, in this whole thing, I did not write any code. Perhaps the most I did was write commands on a terminal, and that was the most complicated part in this. [Pinja] (7:06 - 7:33) Most technical thing in the process. Now that we are moving towards LLM helping or creating the code for us, and as you say, many of our customers are coming to us with something that they've created with Lovable, they might have a prototype per se, and the flow is really fast. The feedback loops are even faster than before, we're flowing the code faster. What are the UX process steps you would recommend not to skip when creating code with LLMs? [Ray] (7:33 - 8:39) Right. Well, one quick thing that you could do is do a quick accessibility audit on what you've created. There are some tools and plugins to, for example, check the contrast, which doesn't require that much time and effort to do, and that would basically highlight things that you could very easily fix right from the start. But then what I would also really recommend is to test it with real users, if you have the time and capability of doing that, especially if you're intending to release something for a larger audience. Yeah, definitely test it with real users before doing that. So I think, yeah, in a way, user research is perhaps becoming a more important part in this whole process, and not so much the details of UX and UI design in that sense. [Pinja] (8:40 - 9:04) One of the questions I've heard being asked is, where does the user understanding live now that more and more code is being created so fast? And yes, the designers, I guess, should still be responsible for user understanding. How do you feel about maybe the relationship between a team creating the code and the user experience designer now with this kind of flow? Has it changed already? [Ray] (9:06 - 10:14) That's a good question, because on one hand, I'm thinking about the projects I'm involved with right now, there isn't that much AI use still, at least on that level. But I think going forward, yes, I do think designers should still be responsible for the user understanding. Designers in projects are usually the person to advocate for the user and the user experience. And I think that should be continued, even though LLMs can sort of simulate user understanding in a way, it still relies on, well, basically on the data that they've learned from. And if the data quality is good, and I think it has been, then LLMs might be able to solve user issues on the go. But I think a little bit more depth would be required. And I think in the end, there should be a human responsible for that. Yeah, that's something I still think we should hold on to. [Pinja] (10:15 - 10:43) Yes, I'm glad that everybody now that I've talked about AI, regardless of the angle, has said that we still need the human in the loop to at least review and give the direction. There is one thing that has been very characteristic to AI-generated solutions, that they look very polished. Is that an illusion, that this is something where a good user experience has been taken into consideration? Is it easy to create something aesthetically pleasing? Is it always user-friendly? [Ray] (10:43 - 11:46) Well, not exactly. It can be user-friendly, but just if something is aesthetically pleasing doesn't guarantee its user-friendliness. And there, in fact, have been studies, for example, posted by an NN group about these aesthetic usability effects, where users might think that something is easy to use just because it looks beautiful. But when they actually use the application, then they run into issues, they make errors, and they get confused or things are difficult to do, even though they still hold on to the impression that it would be easy to use. And I think it also must tie into the psychological concept of halo effects, just because something looks good. Or if something looks good, you sort of associate other good qualities to that, while it might not necessarily be true. [Pinja] (11:46 - 12:12) Yeah, I've heard of that too. I guess when we are looking into the user needs as well, there is one thing that, of course, user experience is taken into consideration. It's not just the look and feel, but also the actual user needs behind it. Do we have the risk of AI only reflecting the generic patterns instead of the real user needs in the final product, if we only rely on AI creating the prototype and the solution? [Ray] (12:12 - 13:14) Yeah, most definitely. I think AI does often rely on generic patterns. And oftentimes you hear that people can tell if something has been purely created with AI. It has a certain style, even certain rounding of buttons and cards that the AI seems to really enjoy doing. So I do think it also goes a bit deeper. So I think AI does tend to use certain patterns. And maybe some of these patterns are well tested and tried previously, and perhaps they were created once upon a time by UX designers who are really investigating the issues and thinking very carefully how to solve them. And the AI might be reusing them, but it doesn't mean that it applies to every situation. If you just use a broad brush to paint over, I don't think it solves all of the detailed issues. [Pinja] (13:15 - 13:38) Yeah. And you already mentioned that you yourself understand a little bit of coding as well. Now that the role is a little bit shifting and maybe towards more prompt engineering, what are the necessary skills from a UX expert in the future and not so distant in the future right now? Does it migrate towards more understanding of the code? What are the trends looking like at the moment? [Ray] (13:39 - 14:18) Yeah, I think UX design might be shifting more towards a UX engineer position perhaps, where understanding codes a little better than currently would be necessary, especially if these new UX engineers would be responsible for, for example, vibe coding an application and then perhaps making some detailed fixes in the code itself. Then I think understanding the code a bit better would be necessary. Yeah, that would be what I would say about that. [Pinja] (14:18 - 14:46) Yeah. Now we need to consider some of the characteristics, of course, between the UX process. There are certain requirements, there are characteristics of vibe coding. Is there a certain clash if we think of the vibe coding as more about the fast shipping and creating low friction for experimentation? Do the principles and the requirements of good UX, are they in clash with these characteristics, what is needed from the current process? [Ray] (14:46 - 15:24) In a way, yes. I think with fast shipping, you definitely are tempted to not really think too much about what is the actual user experience. You might just want to get something out quickly. Well, then you're essentially testing the user experience in production, which might not be the ideal case. Yeah, I still would advocate testing in development instead of in production. So there's definitely a risk of that happening. [Pinja] (15:25 - 15:46) Yeah. There is, of course, the user understanding that we've discussed, there's the usability and accessibility heuristics. One thing, which is in my understanding, a key element whenever I talk about user experience and then a product that delivers the value to its users, how well do the current AI solutions take into consideration accessibility and the heuristics around it? [Ray] (15:47 - 17:02) Accessibility, I would say, takes into account quite terribly. I've basically never been able to vibe code something that would pass an accessibility audit. It makes a lot of mistakes here. Part of the reason could be because accessibility is oftentimes a thing that even current products and services might not consider and would not pass an accessibility audit. And if the current LLMs are relying on the previous data, then they're going to be making the same mistakes that were done previously. With usability, I think it fares a little better. And I think usability has been considered for a longer time and more often than accessibility, perhaps. So it can follow the good patterns that have been established. There's still surely going to be some mistakes it makes. And therefore, I do think keeping a human in the loop is still important. [Pinja] (17:03 - 17:53) Yeah, there's many considerations that we do need to really think about. Last year, we were talking about accessibility, how that actually even impacts and benefits the people who might not actually need the accessibility features so much or are not the target group for them. So it's not just when we talk about accessibility, it's not only a marginalized group of people that we're thinking about here. When we think about the vibe coding and using LLMs to create code more, what are maybe the process steps of a software development lifecycle where we could use vibe coding? Actually, the good use cases of vibe coding would come into play when we take the UX principles into consideration. Somebody has been talking about pretotyping, for example, quite a lot recently. Is that the time and the place for that? [Ray] (17:54 - 18:41) Yeah, I think with vibe coding, because you can test some ideas, you can do low-risk experimentation quite easily and early prototyping. And in the end, with design, you're essentially trying to solve a problem that a user is facing. If you're, for example, creating a new product for that problem, then I think early prototyping is definitely very useful, even if just to get some ideas out there and see how it would look like. I think it definitely is useful and is going to be even more useful going forward. So in the ideation phase, it's definitely an important tool. [Pinja] (18:41 - 19:04) Is there a risk that not everybody who are in part of the process, whether that be somebody internally at a software development company or a customer who might not realize what stage we're in when they see a prototype that has been created with the use of AI, because it might look really nice. It might look very polished from the get-go. Is there a risk of that? [Ray] (19:04 - 20:03) Yeah, certainly. There's certainly a risk, which even previously, it's been sometimes observed, at least anecdotally, that if a designer creates a very nice looking prototype in Figma and gives it to a client, they might be confused at some point that isn't what we're actually releasing. But it is not the case. So there's still a gap between even a well-functioning early prototype and the final product. Perhaps it has to do with the scaling as well. If you've white-coded something that is functional and could be considered a minimum viable product, once you start scaling that up, it might easily break. So there's still steps in between an early prototype and a full product, even with white coding. [Pinja] (20:03 - 20:11) So I guess everybody should have an understanding on what is the process that we're in, and what is the reason why we're using this version right now. [Ray] (20:12 - 20:13) Yeah, exactly. [Pinja] (20:13 - 20:27) A couple of words about UX in general these days. What are the areas where you've seen that the UX principles are taken well into consideration? So where does the UX work well in software development and application development? [Ray] (20:28 - 23:07) Right. I think it works well when whatever you're building and a team and organization around it is very receptive towards creating good experiences for the user. And they find that important that their end users are happy with the product, and they want to use it, and they keep using it as well. Sometimes there might be some pitfalls in that sense. I think it's especially at risk when you're building an application that is basically one of a kind. This could be like an internal application, for example, where the end users don't have any other alternatives. So then, well, the product team or the organization might think that, well, the user experience isn't that important, because in the end, they will have to use this application, and they will have to learn how to use it. And there's, yeah, well, no other options they could do. But then again, I think especially in customer-facing applications, UX is much more critical to consider. So that would be a situation where UX works well, perhaps because it needs to work well as well. So it gets more emphasis. But yeah, talking about the pitfalls, what you sometimes might see is this design-by-committee type of behavior, where instead of the designer owning the design process, the design process isn't very clearly owned by anyone. And designs might be decided in meetings with several stakeholders, all voicing their opinion. And the end results can be a little messy. It's sort of like too many chefs creating one soup. And that's, well, I mean, sometimes if you're lucky, you might create something good with that. And sometimes it's also necessary to hear from different stakeholders, especially in complex projects where any one person doesn't have all of the knowledge about the product they're building. So you do need input from multiple sources. But it's still, I would look into who really owns the design process and who makes the call, because the diffusion of responsibility also, I think it has a negative effect on it. [Pinja] (23:07 - 23:14) All right. Those are the tips to watch out for. Ray, thank you so much for joining me. This was very insightful. [Ray] (23:14 - 23:18) Thank you. Thank you for having me. It was a pleasure talking about these topics. [Pinja] (23:18 - 23:29) And thank you everybody for joining and tuning in today. We'll see you in the sauna next time. We'll now tell you a little bit about who we are. [Ray] (23:29 - 23:41) Hi, my name is Ray Jonathan Pohjanheimo. I'm a Senior UX Designer at Eficode, really into AI and the possibilities that AI has to offer for future design. [Pinja] (23:42 - 23:57) I'm Pinja Kujala. I specialize in agile and portfolio management topics at Eficode. Thanks for tuning in.