Globally, software defects cost the economy hundreds of billions of dollars each year. At a more micro level, it costs organisations anywhere between US$ 5 million to US$ 30 million annually in lost sales and customer goodwill depending on the scale of its operations. Even a single high profile outage could irreparably damage a business. Increasing the overall defect detection efficiency in an application during the software development lifecycle could shrink application maintenance cost considerably and have a direct impact on an organisation's top as well as bottom line. This makes rigorous and comprehensive testing of applications prior to go-live a critical business imperative.
While organisations have been conducting in-house testing and even outsourcing certain segments of the
testing process to external service providers, market dynamics demand faster test cycles, testing earlier in
the life cycle, and improved software quality at lower costs. Achieving all three key objectives simultaneously
has generally proved an ambitious task for most organisations. For instance, according to various analyst reports, testing can account for 20 to 40 percent of the overall application development time and if delivered onsite can account for a significant chunk of the total cost. This results in either quality being sacrificed at the altar of cost and speed-to-market or vice versa. Competitive organisations, therefore, seek ways to improve the performance and quality of applications, reduce testing costs, and boost productivity.
Outsourcing testing requirements to a managed testing service provider is known to reduce costs by up to 50 percent compared to managing the same in-house. Continuous improvement in quality, however, demands a delivery model that engages the service provider in a more collaborative relationship that will ensure long term benefits for the organisation. This is where the managed testing services model can make a difference. Managed Testing Service (MTS) providers offer a longterm commitment to improve software quality by optimising the application testing process, driving higher resource utilisation, building and maintaining a knowledge base equivalent to the actual user or in-house domain experts, and introducing enhanced levels of quality control.
Managed Testing Services
Outsourcing the software testing function to a MTS provider will enable organisations to focus on business development activities while concurrently ensuring significant improvements in product quality. Advantages include testing centers of excellence that add value across the software development lifecycle, centralised testing with standard tools & processes, flexible on-demand staffing, better governance, and greater transparency through management dashboards.
A Testing Centre of Excellence (TCoE) comprises testing specialists with a deep knowledge of testing methodologies, skilled resources trained in multiple domains, proven testing automation techniques, and standard, reusable tools to deliver comprehensive testing services. This ensures uniform output quality from the testing function and can help organisations reduce their testing costs by as much as 50 percent. Also, a TCoE is involved in the application development initiative from the design stage and performs testing across various stages of a development project thus ensuring higher product quality.
Centralised testing improves consistency of testing efforts, enables more objective assessment of software
quality, and provides more flexibility in deployment of testing resources. Centralised testing capability helps utilise scarce test resources much more efficiently and effectively. It reduces the need for organisations to send skilled operators to remote locations to set up test equipment and run test routines. Centralisation provides economies of scale that permit focus and specialisation on such areas as processes and tools. The testing methodologies and tools are designed to improve software quality and help organisations achieve higher application stability. Centralisation helps in defining standards and conventions across all projects ensuring cost, quality, and cycle time benefits through reusability, process efficiency, and elimination of wastage.
MTS also addresses issues like scarcity of skilled software testers and limited testing resources through a centralised team of highly experienced testers cross-trained in multiple technologies. Flexible staffing enables optimal resource utilisation and delivers greater cost benefits through economies of scale. The testers have exposure to a wider range of applications, tools, and techniques enabling on-demand allocation of resources based on project requirements.
Program and project level dashboards offer the management greater visibility and transparency into
the testing process. At any point, an enterprise can measure the performance of the testing team and
ascertain the quality of the software being tested.
The MTS model offers a business-driven structured testing methodology, well-defined governance procedure, comprehensive documentation on test processes, test approach, and test strategy, standardised processes, and clear service level agreements. This helps mitigate expensive post delivery errors by identifying and correcting the same earlier in the software development lifecycle. Services provided by MTS vendors cover the full lifecycle of testing services including requirement analysis; planning; design and development; execution and reporting, development testing, system testing, acceptance testing, system integration and functional testing, non-functional testing, performance testing, security testing, packaged application testing, and also consulting and advisory services on how to best introduce and conduct testing within an organisation. The three core benefits that organisations should expect from a MTS engagement, quality control, cost reduction, and reduced time lines, are discussed in greater detail below.
Control Quality
MTS providers have a clearly defined defect management and resolution process in place to assess the readiness of the application throughout its lifecycle. The goal is to remove maximum defects at the development stage, and in some cases the requirements stage itself, and prevent it from creeping into the production environment. This gains greater significance as defect removal in the production environment can otherwise prove an expensive and time-consuming experience.
Automating test scripts reduces cycle time and enables providers to run more tests during a test cycle. The additional testing ensures significant improvement in quality, as it enables detection of errors that might have escaped the initial rounds of testing and thus decreases the number of post-go-live defects. Additionally, testing automation provides tangible benefits like 24x7x365 availability of test scripts, and issue discovery early in the SDLC. It provides a repeatable process for verifying functionality and scalability.
The aim of any managed testing service engagement should be to continually improve process and methodologies across the application development lifecycle to provide a superior end product. To ensure adherence to this objective, organisations should ensure that clearly defined service levels are finalised prior to commencing the engagement. It's understandable that setting SLAs prior to the engagement could be difficult as the service provider would not have a view of the as-is quality of the applications and hence commit to it. The focus of the SLA should be defect density reduction within a single test cycle and enhanced Defect Removal Efficiency (DRE). To ensure this, certain penalties and rewards could be defined for a drop in quality or exceeding expectations respectively.
The MTS provider should offer metrics and dashboards, which provides the organisation's management a clear understanding of the quality gains at any stage of the engagement. This will help estimate and guide process improvements and thereby drive software quality. Metrics-based tracking and resources skilled in the latest tools and technologies ensure superior defect removal during each phase of testing.
Reduce Cost
As mentioned at the beginning of this paper, the reduction in overall defect density during the course of the software development lifecycle invariably leads to reduced maintenance costs and improved bottom lines. While IT budgets generally account for 4% to 5% of an organisation's turnover maintenance can account for as much as 70% of that budget. Rigorous testing during the development cycle can reduce this to less than 40% of the maintenance budget thus directly impacting margins positively. Additionally, outsourcing to a managed testing service provider offers significant cost savings while providing in-house staff the leeway to focus on strategic business imperatives. Opting for the MTS model provides organisations the option of maintaining spending at roughly the same levels while leveraging the lower cost of resources to increase the level of testing and thereby the reliability of the products and services.
Unlike in-house IT teams MTS providers have access to greater levels of technology skills, which the organisation can access on a needs basis. This does away with the need to hire expensive testing resources leading to considerable reduction in resource costs. The model also resolves the challenge of resource utilisation, as it can be ramped up or pared down to meet project requirements. Another intangible benefit is the reduction in process ramp up time. Since a common set of resources manage multiple testing projects they are internally cross-trained in multiple technologies thus ensuring quick ramp-up on the knowledge curve. This in turn reduces cost and increases productivity. Interestingly, a similar logic applies when standardised processes are followed.
Test automation too plays the dual role of improving quality while reducing costs, as fewer resources
need to be allocated due to increased testing coverage and rigorous regression testing. Another key cost area is investments into tools and technologies. An in-house test team or a vanilla testing outsourcing model would require investment into expensive application testing and load testing tools. An MTS provider, however, takes on the onus of ensuring the organisation access to the latest tools and technologies.
Shrink Timelines
Process standardisation and the proven frameworks and methodologies offered by MTS providers ensure a significant reduction in testing time lines. Effective use of automated testing tools and continuous service improvement also goes a long way in ensuring quicker time-to -market with a superior product. Automation can reduce test cycle by as much as 50%-75% compared to running the same manually.
The follow-the-sun global delivery model offered by MTS providers can significantly accelerate test cycles. Companies can reduce test cycle by as much as 50 percent, and the improve quality by adding more test scenarios.
In case the organisation faces pressing time-to-market deadlines, it can leverage the quick resource ramp up option provided by service providers to buy more testing resources and thereby meet that deadline.
What to Watch Out for
The MTS provider should have demonstrable experience across diverse industries, especially regulated sectors such as finance and pharmaceuticals. It should ideally have core verification and validation experience and a clear track record of successfully delivering multiple testing engagements backed by a robust and mature testing process framework.
Additionally, it should offer the flexibility of providing services both onsite and offshore across the SDLC. In case the organisation's application development work is managed by a third party service provider or the organisation has outsourced its testing requirements to multiple players, it should ensure that the service provider has the capabilities and skill set required to work with other providers.
SLAs should clearly define the minimum level of service that would be delivered and should be objective and measurable. While SLAs will vary based on scope and scale of the project, factors like on time delivery, volume of work delivered, and error containment should be clearly laid out. The organisation should collaborate with the service provider to establish clear rewards and penalties to provide the right motivation.
Testing engagements should have inbuilt flexibility and scalability to ensure fluctuations in scope and timescales can be met. Also, the service provider should have access to the infrastructure and resources to meet unforeseen demands that may arise during the course of the engagement.
Conclusion
Though the core focus of aligning with a MTS provides is cost reduction, organisations should not get carried away by the large cost benefits that may be offered. Instead, the focus should be on seeking testing services that help organisations reduce total cost of ownership and time-to-market for products and services with high levels of quality, stability, and reliability. Delivering without quality and performance places the business at risk and increases operation costs. Also, as the testers become more familiar with the applications being tested, they can add immense value to the engagement by conducting root cause analysis thereby shifting the focus from defect detection to defect prevention. However, testing should be synchronised with the SDLC right from the design stage. This will mitigate the probability of expensive defect repairs later on in the cycle and ensure adherence to delivery deadlines.
This necessitates the need to select an experienced testing partner with proven industry expertise and structured processes that will ensure best practices are consistently applied. The testing provider should have deep technical expertise and experience across multiple industry verticals in addition to the capability to work with multiple vendors across geographies. Organisations that are new to the concept of outsourcing can also look at Managed Testing services as a risk free way to mature outsourcing skills while simultaneously benefiting from improved testing efficiency, reduced time to deployment, and lower costs.
Comments
Post a Comment