Travis Wilson Travis Wilson - 3 years ago 87
Java Question

When (NOT how) should I test UI components on Android?

I found lots of material on how to conduct UI tests using the Espresso framework and others. However, I am struggling to find any information that tells me to what extent should I test the UI: should I write tests that test every possible GridItem in every gridView? Just the items that are clickable? How should I go about determining what, and in how in depth I should test any given Android UI?

For this question, you should assume that you are the developer, the owner, and the designer.

It would be nice to get something that gets at what to prioritize in the general case.

Answer Source

Any answer to this question will be opinion based and it also depends on many factors. Let me give you some factors I would consider myself.

Unit testing:

The first question to ask is whether UI tests will be the only automated tests.

If you already unit test your code and have a reliable test suit, then I wouldn't invest too much time in UI tests. Write some tests that check the basic integration between UI elements and the rest of the application, but don't try to test the internal logic with UI tests - this is unit tests purpose.

If you don't have unit tests and the only automated testing will be done through UI, then you want to go deeper. Map all the flows that your application should perform, sort them according to testing complexity, and start writing UI tests for each flow starting with simplest. You would also want to measure the coverage of your UI tests so that you could have some visibility into the parts of the application that are not being tested.

Manual testing (QA):

The next factor is the amount of manual testing that you can afford.

Since you mentioned that this is a one-man show, you probably can't allocate days for manual testing before each release. In this case, it makes sense to have a broader suit of UI tests. Especially for flows that involve data validation - you really don't want to ensure (each release) that register flow doesn't accept malformed emails/usernames/passwords.

If, on the other hand, you have dedicated testers, then you can drastically reduce the investment into UI testing. I would say that, in some cases, you can remove UI testing completely if you have dedicated testers and a good suite of unit tests.

Requirements change rate:

UI testing is a huge PITA if requirements change often. Especially if these changes affect UI/UX.

If you have a stable product with existing user base then you can add more UI tests because the rate of change will probably be relatively low, and you want to guarantee that the existing functionality will not break.

If, on the other hand, you are just bootstrapping the app or exploring ideas and UI/UX is very unstable, then maintenance of UI tests will require much effort.

Cost of errors:

Think about what will be the cost of errors and adjust your testing strategy appropriately.

If the cost of a bug is just bad user experience and minor inconvenience, then you don't need to invest much into UI testing. If the cost of a bug is personal data being compromised, I would invest some more into testing. If your app can cause people to get injured, please make sure to test thoroughly and have a separate team that will carry this out.

General remarks:

For quality assurance, IMHO, three techniques are the most effective on Android: code reviews, unit testing and exploratory manual testing.

UI tests are less effective than the above three. Therefore, if you have not mastered one of the above yet, then you'd better invest the time into them instead of UI testing.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download