Building AI on AWS: Overcoming the 5 mistakes that kill AI initiatives

Even though AI is relatively new and a bit of a Wild West, we can already see the mistakes of some companies building AI solutions.

Regardless of how far you’ve come in your AI development, you can save a lot of time and resources by learning from other companies’ mistakes. 

You need a solid foundation—and laying that foundation on AWS is a great start—so make sure your foundation doesn’t have the same massive, structural errors many other companies build on. It’s too easy to cut corners and rush ahead to cash in on this new technological business opportunity (AI) that suddenly became available to everyone.

The novelty of a new tool wears off fast, and once it does, you’d better have a solution of value—something well-crafted that solves real problems. 

It may sound obvious, but I see companies trade speed for quality all the time. And I don’t want you to be one of them. Read on and get it right from the start. Learn what the five big mistakes are and how to avoid them. 

But before we dive into the fun stuff, let’s take a step back and acknowledge something important:

AI is not your end goal 

It’s simply another tool in your already-stuffed toolbox. And since it’s a tool that can be configured in a gazillion ways, you need to be clear about what you are building. 

Broadly speaking, today, there are four main types of AI implementations:

• Customer experience: Chatbots, assistants, and personalization
• Productivity boosters: Interactive search, summarization, and code copilots
• Content creation: Creating text, images, and videos, such as animations
• Business intelligence: Mining and processing data and creating business insights

If you’re not clear about the nature of what you’re building, you risk choosing the wrong set of services. 

So, what are you trying to accomplish? Something that makes coding faster and better? A new product? A way to extract insight from existing data? Test automation? Features for an existing ecosystem?

If you don’t know the answer, you risk wasting a lot of money and effort. You will draw the wrong conclusions, create bad customer experiences, and be distracted from your core business.

In other words, there’s a lot at stake. This is the great innovation race of our time. And as races go, by selecting AWS as your starting point, you’re in a great position from the outset. 

A few words about building AI on AWS

For your AI innovation to function, multiple AWS services need to work together. So, let’s have a quick look at AWS’s three main AI services: 

Sagemaker: If you are, for example, a data scientist, you use Sagemaker to prepare data or train and finetune an ML model. You can also use it for code-based development with Jupyter Notebooks.

Bedrock: If you are a developer, this is an API for multiple different base models. You use it to build new or enhance existing services or apps.

Amazon Q: This is a generative AI assistant and expert guide in multiple areas of the AWS ecosystem. Within the AWS console, you can ask technical questions. It is your coding companion in CodeWhisperer and BI assistant in QuickSight. 

With Q, you can easily create a secure chatbot for your organization and enhance the chatbot with Retrieval Augmented Generation (RAG). This means you can supercharge it with your own data, whether it’s in documentation, website, or database form. 

I also want to add a couple of final notes on AI in AWS: 

1. Once you have a clear vision of the goal you want to achieve, you need to pick the right vehicle that will take you there. If you need to reach a mountain village, you can use a helicopter, cable car, or bicycle. The right option depends on what you are looking to achieve—speed, great views, or exercise.
2. As you already know, if you are working with AWS, you are dealing with a vast ecosystem of interoperable tools. Never lose sight of that, as there are plenty of synergies and efficiencies to be had with the right vision and overview. 

And now, without further ado, let’s see how you can avoid critical mistakes. 

Mistake 1: “We need AI because… ahem… why not?”

As I already touched on briefly, not having a clear idea of what a new feature should do or solve is the most fundamental mistake you can make. 

Don’t throw money and resources at some gimmick simply because “everybody else is doing it.” You will just end up with a failed, stranded pilot, accumulating costs without anybody using it. 

How to prevent shooting in the dark 

To prevent this from happening, you need to plan some real-life usage scenarios. Brainstorm and simulate potential real-world applications of the AI tool. 

Every decision should be a business decision. Once that requirement is met, picking a suitable technology stack is much easier.

What must it be able to do? What are its limitations? Only once you have answered these questions will you know whether it meets real user needs and solves problems.

Bonus tip

Recycle! Rummage through your backlog of ideas. Maybe you have previously discarded good ideas because the timing was wrong. Sometimes, great ideas are just waiting for the right technological advances to come along. AI is the technology that will enable many of those ideas to finally be developed.

Mistake 2: Missing a comprehensive data strategy

With a weak or missing strategy, your data handling will be inefficient. Data will be: 

• Disorganized
• Not cleaned and labeled 
• Inconsistently formatted 
• Duplicated  

If you train your AI model on that sort of data, the results and the tool will be unreliable. Your costs will also be unnecessarily high since you waste resources by using more storage, computer power, and services than necessary.

As for the AI service itself, this can be catastrophic. The whole model will be biased and produce skewed or unfair outcomes. This is particularly concerning in areas such as hiring, finance, or life sciences, where biased decisions impact people’s lives. 

And if that wasn’t enough, with poor governance, your data becomes more vulnerable. You’re inviting breaches and misuse.

How to succeed with your strategy

Define a clear data strategy before diving into AI development. In this strategy, include: 

• The services you will use 
• Integrations
• Data labeling (at least the separation of PI data) 
• Data handling and storage
• Backup protocols 

Since you’ll be building your AI tool on AWS, your strategy will be better informed if you familiarize yourself with a subset of AWS services (Glue, Macie, etc.) beforehand. It is also good practice to add Well-Architected Reviews (WAR) as part of your operations. 

I also recommend Well-Architected Framework workshops for business and sales operations. 

Bonus tip

Involve a broad spectrum of people when creating the data strategy. This includes cloud architects, data engineers, and business stakeholders. Only then can you be sure the strategy aligns with both technical requirements and business objectives. Remember to involve business and salespeople in the AWS Well-Architected Framework workshops, too!

Mistake 3: Not using your data to its full potential in your AI implementation

Many companies simply fail to properly use data to train, validate, and improve their AI models.

To build something sizeable, you need to lay a solid foundation with your data. And to do that, you absolutely need to optimize:

• File storage for structured and unstructured data 
• Storage of operational data (SQL, NoSQL, DocumentDB, VectorDB) 
• Analytics structures (batch jobs, search, streaming) 
• Integrations (capture, modify, stream) 
• Governance (privacy, data access, quality)

Your best bet when establishing a reliable foundation is to adhere to the AWS Well-Architected Framework. 

This is a fantastic way to manage data governance, infrastructure, and data quality. The framework has six pillars: Operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. 

In other words, it covers a lot of ground and takes you a long way toward building a solid foundation on AWS. 

If you want to learn more about the Well-Architected Framework, a colleague of mine wrote a great introduction to it here.

Mistake 4: Not managing your costs for serverless

Picking the right tools and services for your exact use case is difficult. 

You need to know—or at least have some idea—what the resource usage will be for the whole stack. You also need a good monitoring and alert system in case something unexpected happens, for example, if a service goes viral. Usually, you end up picking the services you are most familiar with or that you have seen in a demo. 

There are great serverless options out there—free from maintenance and upkeep—but they vary greatly in terms of cost. Perhaps more than most people realize.

With some services, you pay even when you don’t use them. Let’s look at a couple of scenarios. 

Let’s compare using OpenSearch Serverless for your VectorDB vs. using Aurora Serverless in a scenario where you have irregular demand, for example, in a proof-of-concept or roll-out phase. Both services deliver good performance, but the cost of using Aurora is tiny (€70) compared to OpenSearch Serverless (€1000).

In a different scenario, where you have a steady flow of DB usage, if you were to compare RDS with Aurora Serverless, then RDS can be a whopping 50% cheaper because you can reserve cheap capacity in RDS.

How to be smart about serverless

Make sure you keep your ear to the ground and know what’s on offer. Keep track of AWS news releases, participate in AWS workshops, follow suitable social media channels, and use price calculators.

Mistake 5: Not obsessing about the most important metrics

Even if your AI solution is technically superb, if it doesn’t meet the user’s needs, it’s a wasted effort.

You already know how important metrics are and that not all metrics are created the same. So, for your AI initiative to work, you need to know what data matters. Therefore, when you monitor and measure, be absolutely certain that you do so against clear objectives and targets. 

There are plenty of notorious examples of AI failures: AI trading, faulty medical diagnoses, offensive chatbots, and so on. But that’s only the tip of the iceberg. 

There are countless low-key examples, such as AI helpdesk assistants that are slow or give you “almost” good answers. In these cases, you may inadvertently “force” the operator to take the traditional path and simply copy-paste answers from previous, similar tickets. 

How to be smart about metrics 

Gather statistics and information on how things operate currently. For example, if you have a helpdesk AI, try to solve the most pressing support issues. One measure could be to come up with rock-solid answers to the three most common support ticket issues.  

Keep in mind that even the most cutting-edge technical prowess is a failure if it doesn’t solve the problem at hand. 

The three pillars of AI success are:

1. Fulfilling business or customer needs 
2. Making the life of the everyday operator easier 
3. Using the right service or technology to achieve this 

Usually, that third part is the most trivial, but quite often, projects have that priority completely upside down. 

Bonus tip

Your customer or end-user is the expert in their own systems and ways of working. Try to really listen and pinpoint areas where the new AI solution could help them, whether it’s a chatbot to support personnel or an X-ray analytics assistant for Radiology. 

Final thoughts

“Innovation” doesn’t mean using ChatGPT or a similar tool, as you would a search engine. 

Real innovation happens in unchartered territories when you find completely new ways of working with tools. Using fancy new techniques because of the hype or because your competitor is talking about it will most likely cause your pilot to fail. 

Building your AI on AWS is a great choice, so make sure to do your research on the whole ecosystem and keep listening because there is a constant flow of new tools coming out. 

Remember that the tech is only one part of the equation. The user and your business have to need and want it, and the end result will only be as good as the combination of the freshness of the idea, supercharged with the data that goes into it.