Skip to main content Search
Contact us

Guide

Delivering continuous quality

How to truly build quality into your software development

Delivering -continuous -quality_yelllow_2024_Guide Hero_Eficode

Stay on this page to read the full guide or download it as a PDF for easy access anytime.

You are in the business of delivering solutions to the masses

IT business success stories are more and more often about creating innovative products and services fast and making them widely available for a global audience. But at the same time these products and services need to be of high quality and attractiveness. By automating your quality assurance activities, you meet both these conflicting needs. At Eficode, we like to call this continuous quality assurance. Instead of quality being the concern of just a few experts, automation and tool choices make everybody a participant and responsible. The quality assurance experts can concentrate on thinking about how to integrate more quality to the way of working, while business, development, and operations naturally “build the quality in”, as the old adage goes.

Receive this guide as a PDF by e-mail

Four habits of highly successful IT businesses

Build sleek production pipelines

It will help to be the one that delivers first.

React fast to new requirements
Keep an eye on what happens in the marketplace and act swiftly. Be the first to start iterating with new solutions to gain market share.
Let machines do the boring things

Computers are fast, reliable, and work 24/7. To improve efficiency, let computers handle all the repetitive and error-prone tasks, and leave only creative work to people.

Keep tight quality gates

 Use automation to build the ultimate quality system that is very strict, but at the same time extremely fast.

What we will cover in this guide

Every product and service is unique, therefore the automation activities differ greatly. In this guide, you will learn the five key areas in which it affects any software development that your teams should focus on to deliver continuous quality

Test automation
What test automation really means

From requirements to test cases
How test design helps different stakeholders understand the link between requirements and test cases

Following the process
What you need for following up the development – metrics and reports

Deal less with test environments
How to provide production-like environments to enable large-scale testing

Managing the masses of data
How you get all the production-quality data needed in testing

1. Let’s begin with the four keys to continuous quality assurance

Build quality into the product from the start

Code quality does not improve on its own when you add testing. When you focus on testing from the beginning, you ensure that the code quality never drops.

Shift left, expand right

Testing is not a single phase in product development. Activities need to be started as soon as you are planning the first piece of code. Early testing enables a fast feedback loop. Testing needs to continue in the production as well; there are more bugs caused by configuration errors than coding mistakes.

Automate the deployments

Using time for manual deployment procedures is often complex and error-prone. Enable experts to concentrate purely on making more software instead.

Lead with real-time data

With data produced by automation, better decision-making can be done in the future.

2. Now let’s explore the key parts of quality assurance

As mentioned earlier, quality assurance touches all parts of software development. In this guide, we will go through the five key areas you should know about. So... Let’s take a tour!

  • Automation
  • Design
  • Metrics
  • Environments
  • Data
Tour guide

3. Test automation – the practice

Automation may appear scary, as it fundamentally changes the way product development is done. Old roles transform into new ones, collaboration is more essential than before, and fast feedback dramatically improves the performance of the development team.

Good automation is based on well-prepared requirements. Ideally, each functional requirement can be described as an acceptance test case combining tests and requirements into a single item. This enables the usage of a test-driven approach (with methodologies such as BDD, TDD, ATDD,

or SBE), where each new requirement comes with a test case, and each feature is approved with the passing test. This doesn’t mean that the requirements need to be complex and cryptic – modern testing tools work with test cases that are written in business language.

Testing also applies on the lowest level, where individual pieces of code are tested by the developers. These tests also need automation, and every new piece of code must come with a new set of tests which are automatically executed after every change.

In addition to functional testing, it is essential to also test and monitor the non-functional characteristics of the product. Continuous and automated testing is required to ensure that the product meets performance and security expectations.

You may also need other behavioral testing, such as accessibility and usability. Unlike functional testing, these are not needed early in the development, but cannot be ignored once first releases are done.

  • smart tool choices
  • good organizational culture
  • the courage to change how things are done
Test_Automation_stages_2024_Illustration_Eficode
Testing individual pieces of the product

Use available programming language-specific testing tools for the chosen technology stack. New test cases should be part of every software change from the very beginning.

Testing against business requirements

High-level tests have to be written in business language, and the test cases must link to high-level requirements.

Evaluating the performance

You need to be confident that what is being developed can also survive in the hands of your customers.

 

Checking the security

Security testing is a key factor in risk mitigation in a business.

4. What automation offers management

Cost savings

Automated tests are cheap to run and available 24/7. Releases can be made earlier than before.

Collaboration

Helps demonstrate the quality of deliveries coming from thirdparty suppliers and facilitates discussion with them about external deliveries.

Happiness

It improves job satisfaction when people can concentrate on creative work and leave repetitive, boring tasks to machines.

 

Status in real-time

Provides metrics in real-time for progress, planning, and capabilities of the product. This helps to avoid nasty surprises at the end of the project.

Shared status

All data can be collected in a single report which has links to more detailed information, providing transparency for all stakeholders.

5. What automation offers development

Efficiency 

Parallel test execution is limited only by hardware capacity. Good automation points the debugging efforts during development in the correct direction.

Teamwork

Improves collaboration by providing fast feedback. Shared quality gates bring uniformity to the code base, lessening context switching.

Reliability

Automation helps testing long and complicated procedures. The confidence to release increases when these corners are covered.

 

Tracing

Bugs are easier to reproduce, and the fixing of them is easy to track – any feature can be dug up years later to see exactly the business context at the time.

Regression

Adding new features is simple, as existing functionality can be fully tested. Even refactoring the complete application is straightforward due to the safety net that automation provides.

6. Test design – bring on the test cases

Software product development begins with defining requirements. A requirement may need multiple features, but all features should link to one requirement. Each feature should also have at least one test case, ensuring that the feature actually works as intended. Defining your test cases comes down to test design.

Since testing links directly with requirements, you should involve QA experts in defining your requirements. The earlier you start testing, the earlier you find your defects – and the cheaper it is to fix them, since you’re not building on top of broken code.

One solution for better test design is to use test-driven development (TDD) or acceptance test-driven development (ATDD/BDD/SBE). These approaches call for you to define your tests even before you implement any new functionality. This means all your new features will have a proper test case and are well-tested before they reach production.

As your system grows, so does the need for testing.

With an ever-growing regression set from each new feature, the need for test automation grows so sufficient exploratory testing can still be done. Proper requirement management and test design helps you transform your manual regression tests into fully functional automated tests.


  • Reliability: Ensures your system is sufficiently tested and features that worked previously still do so.

  • Readability: Everyone understands how your system is tested, and development is done with testability in mind.

  • Maintainability: Changing requirements also changes tests. Well-defined, coherent features and tests help minimize maintenance costs.

  • Traceability: Links features and test cases, so you know where a specific feature is tested and how.

  • Coverage: Ensures each of your features has a well-defined test case before they are deployed to production.

7. Test metrics – following the progress

Good usage of test metrics allows you to make decisions based on the available data, and potentially discover upcoming issues before they even become a problem – whether they are related to code quality, tests, or ways of working.

Regardless of what metrics you use, they should also be presented somehow. Real-time public dashboards allow everyone to understand what areas need improvement and why. With good dashboards and reliable data, your teams can be more self-organized and tackle issues without prompting.

Often, test metrics are considered to be only the test results. The results are ultimately the one metric everyone deems most important, but there are lots of other testing-related metrics, such as test coverage, execution time, and lead time. You may want to look at these too.

When everyone has access to the same data, they understand the reasoning behind certain decisions. Public data creates transparency between stakeholders, even though they wouldn’t be interested in the same data by default.

Facilities team_JSM (1)

A message to deliver to your CTO

Quality metrics should be combined with other software development metrics — preferably in real-time.

 

8. Test environments – where you should test

Software becomes ever more complex. Even comparatively small applications are built on top of many abstraction layers:

Different hosting solutions (like cloud), operating systems (including mobile), browsers, and programming libraries.

Also, the number of teams involved in the development adds organizational complexity. Expanding test automation is the first place you will face the challenges of needing well-defined operational environments for your application, so you can be assured that things work in your production as well.

Therefore you need modern virtualization technologies, like containerization with Docker, to be able to test and deliver software.

These technologies also allow you to have production- like environments that serve different purposes: small lightweight environments for rapid development, or bigger heavyweight ones for thorough testing.

With virtualized environments, you enable modern ways of working, such as automatic configuration management tools and Infrastructure as Code. These minimize software

errors related to environment configurations and help with compliance. They are the key enablers of more advanced deployment strategies – like blue-green deployments or canary releases – and leveraging the full potential of cloud services.

Cloud_environment_2024_Eficode

9. Test data – the beast that keeps growing

 

Long gone are the days when an average application stored only a few gigabytes of information during its lifetime. Nowadays it’s 10x, 100x, or even orders of magnitude more.

In addition to more dta, we also need to use that data responsibly and securely. Handling test data is one of the most painful points of modern software development.

Primarily, you should aim for synthetic data generation in testing. In other words, analyze your production data and create representative data yourself. The other option is to use masked and anonymized production data, but there are some drawbacks and challenges associated with that:

  • With synthetic data you can have only the optimal amount of data.
  • Production data is usually very repetitive, while you are interested in testing different options and paths in your product.
  • With production data, you need to take care of compliance (e.g. GDPR). Anonymization is tricky to make legally sound.

Even with synthetic data you might end up with hundreds of gigabytes – or even terabytes – of content. The optimal place for your data would be in a cloud service. All major cloud service providers have solutions for massive amounts of data.

If you need to keep the data in- house, the best solution is to build a data lake-like platform yourself and build in-house tooling for that with DataOps. There are also commercial  tools for test data management which may come “to the rescue” here.

10. The future of continuous quality assurance

Continuous quality assurance means keeping up with an ever-changing world. Organizations on top of their game have a keen understanding of how they are performing in different facets of software development, and are constantly evaluating and incorporating new technologies and methodologies. Being deeply immersed in the quality assurance world, we estimate that you should keep an eye on the following three areas:

Artificial intelligence 

Many tool vendors tout the possibilities of AI to ease the common grievances of test automation. Our viewpoint is that these tools mostly concentrate on lesser problems that do not have as big of an impact as the tool vendors would like you to think. However, it is evident that soon we will see the emergence of AI-powered assistants that enable QA experts to pinpoint and solve murky quality problems faster and better. We at Eficode are excited about the possibilities – let’s just hope “the next five years” don’t last for a long time.

Chaos engineering

Tests are no longer a separate part, and are instead executed constantly during development. Bots with specific logic can run commands constantly and trigger unexpected events in testing and production environments. Embedded test validations can help finding those extremely rare edge cases that developers and testers miss.

Hardware virtualization and simulation

Automating test cases for websites has been an ongoing trend for a while now, and the tools for good web test automation are getting better. At the same time, virtualization tools and simulators are getting more advanced, so test automation for embedded systems becomes easier to implement.

 

11. Before you go, here are a few tips...

 

Today’s winners are optimizing for time-to-market – they keep an eye on what happens in their market and are able to act swiftly. Being the first to start iterating with new solutions is of the highest priority.

Software development needs sleek pipelines using automation built by the development team: an ultimate quality system that is strict, yet extremely rapid. Computers are fast, reliable, and work 24/7, so machines should do the boring things and repetitive, error-prone work. This leaves the creative work to humans.

Think big, start small. What would you like to achieve with test automation? Define your ultimate goals. Create an initial scenario: where and how to get started. Implement a proof-of-concept project to see if you are on the right track. Scale up when it feels right.

Automation needs a factory. Tests as such do not automate anything – you need a place to run them. And you need to build a delivery pipeline and find the right time and place for all the tests. To achieve higher capability, check out Eficode Test-Automation- as-a-Service

Visibility is everything. You need to have a clear view of everything that happens in test automation. Information radiators to share the status, and reports on need-basis, all in real-time. And then follow-up meetings with all stakeholders. Need help?

Get the right buy-in. Demand for high quality ultimately needs to come from your company’s C-level. We have a guide for that too, which you can get here.