Executive Summary

In today’s digital world, new electronic devices and gadgets are getting launched every other day in the market. The end users can access any website or application from a numerous possibility of browsers and devices. When a developer is working on an application or developing a website by keeping in mind the real world user who is going to use the system in long run, it has to be ensured that the application is compatible with all types of mobile phones, tablets, iPads which come in several screen sizes and platforms.

This whitepaper highlights and speaks about the importance of cross-platform testing, various testing approaches, the several challenges faced in testing, the best practices to adopt and selecting the right tools.

Introduction

It is evident that there has been a visible increase in type of devices and platforms that are more varied now than ever before. Users continue to demand the latest devices, features and functionality, as well as increased mobility and greater accessibility.

With the arrival of portable device platforms and the Internet of Things (IoT), the workload of developers, and especially testers, has increased to a great extent. As traditional testing practices prove unable to keep up with the demand, all the businesses are experiencing significant product delivery delays and, in some cases, there are product defects which prove to be expensive. Naturally, there is a growing demand for more efficient and cost-effective testing across all platforms. Crossplatform testing is a form of specialized testing in which we verify the suitability of your solution to work across various different platforms. Platform can include anything including OS, browser or a device that is necessary to run your solution.

Why is Cross-platform Testing Essential?

Cross-platform testing plays a crucial role in the phase of quality assurance. When a product is developed for multiple platforms, there is abundance in the choice of platform. Cross-platform testing is performed to determine the behavior of your application and website in different environments.

Cross-platform testing helps in identifying issues that may vary with platforms or configurations such as:

For example, an Android application can run correctly on Oreo but does not support Marshmallow. These types of issue can be identified only when cross-platform testing is done during the testing phase. Similarly, user interface of a website may appear alright in the Chrome browser but the layout might seem to be different in the Safari browser. Cross-platform testing which is carried out during the testing phase can address such issues.

Testing Approach

Testing applications for cross -platform capability can give rise to uncertainties if you don’t plan it beforehand Also, If the requirements for crossplatform functioning is not communicated to the developer and testers clearly, it may lead to product failure as the product might be good for only one or a specific platform or configuration.

Before carrying out cross-platform testing, you need to keep some essential things in mind.

Know your audience

You should have good knowledge of who will be the dedicated users of your application and what type of devices and platforms are commonly used. This will help you in deciding what platforms or devices should be focused on for cross-platform testing.

Decide the platform to support

Once you know your user-base, the next step is to decide which platform to support or how the solution will be implemented, with respect to your user-base.

Identify how much you can test

Usually it is not feasible to test all possible scenarios. Though exhaustive testing is impossible, it would not be an efficient approach. Instead of exhaustive testing, we use risks and priorities to focus on testing efforts. You need to identify the platform that you are going to test which covers maximum risk and targets maximum audience. Creating a multi-dimensional matrix with a heat map can be useful in determining things that are high priority and those which can are less important.

Test automation

Testing your solution across multiple platforms and devices is a repetitive and time consuming task. You might not see any results and hence, it is possible that you lose interest in testing the same thing on different platforms or environments. In such situations, automation is desirable. There are many automation tools that allow you to run application on multiple platforms. It saves a lot of time. While these automation tools are life-saving, manual intervention is still required because these tools cannot test the usability and accessibility of the applications.

Setting up a Virtual Box or Emulator

Procuring multiple devices and gadgets can be expensive. The best choice is to implement crossplatform testing on different environments and use emulators and virtual box. You can create own virtual boxes and emulators with various platforms and configuration combinations to test applications.

Cross-platform Testing Issues

The planning stage for cross-platform testing is a very important phase of software testing life cycle. When a product is developed for multiple platforms you are likely to face many challenges during testing.

Some common problems/issues faced by testers while performing cross- platform testing are:

Interface consistency

For an application that is supposed to run on multiple platforms, it is important that the interface of design is consistent. The design should be developed in such a way so that it does not cause any alignment issues or unnecessary text wrapping on UI. Such issues leave an unfavorable impression on the users and they lose interest quickly. Testers should capture all the incidents and report them in any incident management tool so that the developers can fix them prior to the release.

Input combinations

It is very important to know how end users are going to interact with the system and type of input they would prefer to use. For example, A GUI with too many buttons is supposed to be complex and not very user friendly to the end user.

Fulfilling user expectations

A good design fulfills the user’s expectations. Every user will set some expectations regarding your product (app, website, new feature, etc.), before they start using it. A good design not only fulfills the expectations but also, adds to the user’s contentment. Thus, fulfilling the customers’ expectations is of utmost importance.

Best Practices for Cross-platform Testing

Define a boundary

It is important to define a clear boundary for Platform testing. This will avoid unnecessary confusion.

Create a test matrix

Prepare a test matrix and use it to execute your test cases and for communication with your end users. The matrix will always be helpful when it comes to designing test scenarios.

Heat maps

Identify elements in the matrix that are of high risk of failure. Highlight the elements that are of high risk in test matrix. A heat map can be helpful to ideally identify areas of uncertainty or risk. This helps in proper prioritization when time and resource are limited.

Use of tools

Tools are always helpful in any type of testing. When the testing task is turning out to be repetitive then you can unitize the tools. This will not only save time and effort but also, prove to be useful in resource and task management.

Deciding and Selecting the Right Tools

There are plenty of tools which are available in the market. It is very difficult to decide and choose any one option. With time, as the features are improving more and more, new tools are getting added with new and advanced options. The first and foremost thing is to come up with a list of features and options according to your needs and requirements. The next step would be preparing a comparison chart with 2 or 3 tools that will help you in analyzing the features and usability of the tools. Some commonly used tools for cross-platform testing are:

• Oracle VM VirtualBox
• Browserstack
• Testize
• Test Plant cross-browser testing

Case Study

About the Client

The client is a provider of e-mail messenger app and secure email application that upgrades your Gmail or Outlook inbox. They help you to reorganize your mail around the people, tasks and teams that matter to the user. This enables the user to work smartly and drive productivity.

Requirement

The client’s requirement was to carry out the end to end testing in mobile / web / desktop applications.

Solution Highlights

• Created a test strategy for iOS / desktop / web application projects.
• Written test cases according to the new functional requirements.
• Executed functional / non-functional / usability test cases for all platforms (Windows, Mac, iOS
  mobile devices).
• Executed the application – syncing behavior throughout the platforms (for example- If you create a folder in the mail box, it needs to be in sync with all the platforms.).
• Compared the e-mail functionalities with other competitors and suggested the best approach.

Key Benefits

• Ensured that the performance of the application has improved to a great extent.
• All out – of – the – box testing deliverables (HTML threading, sync performance, cross – device/ platform syncing, app comparison feature, memory usage etc.) were well documented.
• Quality of all pre-release and internal release builds is ensured.

Conclusion

Cross-platform testing services are focused on ensuring the quality and integrity of application features and functionality. In today’s digital world, there is an abundance of platforms which the end users use. So, it is always recommended that the tester verifies the usability and accessibility of the applications in all the platforms prior to the release. Planning the test approach and memorizing the best practices will make the process easier and efficient.