Mark Mark - 1 year ago 124
Android Question

Flaky Android Espresso Test - Snackbar

1) All devices/emulators being tested have animations disabled.

2) I have an @BeforeClass which builds my Credentials object.

3) I have an IntenServiceIdlingResource and an EventBusIdlingResource, registered in @Before.

4) When the sign in button is clicked, the IntentService fires off. In this case, the server (a mocked server) is returning a 500 error. That information is posted back to the UI from the IntentService, via greenrobot's EventBus, and a Snackbar is shown with the error message.

Here's the code for the test:

@Test
public void a_userNamePasswordTest() throws Exception {
// email input
ViewInteraction userNameView = onView(withId(R.id.email));

// verify it's on screen and enabled
userNameView.check(matches(isDisplayed())).check(matches(isEnabled()));

// set the username
userNameView.perform(scrollTo(), replaceText(credentials.username), closeSoftKeyboard());

// password input
ViewInteraction passwordView = onView(withId(R.id.password));

// verify it's on screen and enabled
passwordView.check(matches(isDisplayed())).check(matches(isEnabled()));

// set the password.
passwordView.perform(scrollTo(), replaceText(credentials.password), closeSoftKeyboard());

// sign in button
ViewInteraction signInButton = onView(withId(R.id.email_sign_in_button));

// verify the button
signInButton.check(matches(allOf(
isDisplayed(), isEnabled(), withText("Sign In"), withContentDescription("Sign In")
)));

// clickity click the button
signInButton.perform(scrollTo(), click());

// verify the snackbar text
onView(withText(startsWith("Server Error: 500"))).check(matches(isDisplayed()));

}


This is the exception I usually get:


SignInExceptionTest > a_userNamePasswordTest[Nexus_6P_API_23(AVD) -
6.0] FAILED
android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with text: a string starting with
"Server Error: 500"


According to my logging, my idling resources are working. But looking at the timestamps for the logs, the exception is occurring about 5 seconds after the idling resources have gone idle.

It seems like there's a delay between when the resources go idle, and when it attempts to look for the view.

Other possibly pertinent details:


  • minSdk: 20

  • compile & targetSdk: 25

  • buildTools: 25.0.2

  • support-library: 25.1.1

  • espresso-core: 2.2.2

  • gradle plugin 2.3.0-beta3



How can I fix this test so it's not flaky, besides inflating how long my snackbars are displayed?

Answer Source

Espresso enforces a 5 second wait after an IdlingResource becomes active before it is checked again for being idle.

This has 2 negative impacts in my case.

First, it increases how long it takes the test to run. An extra 5 seconds (or multiple of 5) on every test can really add up.

Second, since the snackbar is displayed immediately, and only shows for about 3.5 seconds, pretty much any time you're waiting for an IdlingResource, the snackbar will be gone before Espresso realizes that the resource is idle, making it difficult to test.

ConditionWatcher, described here: https://medium.com/azimolabs/wait-for-it-idlingresource-and-conditionwatcher-602055f32356#.9rms52osh

And dode here: https://github.com/AzimoLabs/ConditionWatcher

Provides a solution to both these issues. Rather than 5 seconds, it defaults to 250ms, and this value can be adjusted (setWatchInterval).

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download