Michael Osofsky Michael Osofsky - 22 days ago 6
Java Question

Does setDefaultHighRepJobPolicyUnappliedJobPercentage(100) really work?

According to https://cloud.google.com/appengine/docs/java/tools/localunittesting#Writing_HRD_Datastore_Tests, "If your app uses the High Replication Datastore (HRD), you may want to write tests that verify your application's behavior in the face of eventual consistency. LocalDatastoreServiceTestConfig exposes options that make this easy." You're supposed to set

setDefaultHighRepJobPolicyUnappliedJobPercentage(100)
and then, "By setting the unapplied job percentage to 100, we are instructing the local datastore to operate with the maximum amount of eventual consistency. Maximum eventual consistency means writes will commit but always fail to apply, so global (non-ancestor) queries will consistently fail to see changes."

However, I don't think
setDefaultHighRepJobPolicyUnappliedJobPercentage(100)
works.

If it did, then my test case below,
testEventualConsistency()
should pass but it it fails on the second assertion. On the first assertion, I read back an object I've saved using an Objectify ancestor() query. It works as documented because the object is retrieved. However, the second assertion fails. In that assertion I've also read back the object I've saved but I haven't used an Objectify ancestor() query so it shouldn't retrieve anything because I've specified that no jobs should complete (i.e. the
setDefaultHighRepJobPolicyUnappliedJobPercentage(100)
setting).

EventualConsistencyTest Test Case

import static com.googlecode.objectify.ObjectifyService.begin;
import static com.googlecode.objectify.ObjectifyService.ofy;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.equalTo;

import java.util.List;

import org.junit.Test;

import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import com.googlecode.objectify.ObjectifyService;
import com.googlecode.objectify.Ref;
import com.googlecode.objectify.util.Closeable;
import com.netbase.followerdownloader.model.DownloadTask;
import com.netbase.followerdownloader.model.User;

public class EventualConsistencyTest {
private final LocalServiceTestHelper helper =
new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
.setDefaultHighRepJobPolicyUnappliedJobPercentage(100));

@Test
public void testEventualConsistency() {
helper.setUp();
ObjectifyRegistrar.registerDataModel();

User user = new User();
user.id = 1L;
Closeable closeable1 = begin();
ofy().save().entity(user);
closeable1.close();

Closeable closeable2 = begin();
DownloadTask downloadTask = new DownloadTask();
downloadTask.owner = Ref.create(user);
ofy().save().entity(downloadTask);
closeable2.close();

Closeable closeable3 = ObjectifyService.begin();
List<DownloadTask> downloadTasks1 = ofy().load().type(DownloadTask.class).ancestor(user).list();
assertThat(downloadTasks1.size(), equalTo(1));
closeable3.close();

Closeable closeable4 = ObjectifyService.begin();
List<DownloadTask> downloadTasks2 = ofy().load().type(DownloadTask.class).list();
assertThat(downloadTasks2.size(), equalTo(0)); // THIS SHOULD PASS IF setDefaultHighRepJobPolicyUnappliedJobPercentage(100) WORKED
closeable4.close();

helper.tearDown();
}

}


User Definition

import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;

@Entity
public class User {
@Id public Long id;

public User () {

}
}


DownloadTask Definition

import com.googlecode.objectify.Ref;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Parent;

@Entity
public class DownloadTask {
@Id public Long id;

@Parent public Ref<User> owner;

public DownloadTask() {

}
}


Environment:


  • appengine-api-1.0-sdk-1.9.17.jar

  • appengine-testing-1.9.17.jar

  • appengine-api-stubs-1.9.17.jar

  • junit-4.11.jar

  • objectify-5.1.3.jar



In case I missed anything else important, here is a more exhaustive list:

Environment

My questions are:


  1. Is
    setDefaultHighRepJobPolicyUnappliedJobPercentage(100)
    broken?

  2. Does
    setDefaultHighRepJobPolicyUnappliedJobPercentage(100)
    not really work as documented? Does it in fact apply the job even though the documentation says it's not supposed to?

  3. Is the value passed to
    setDefaultHighRepJobPolicyUnappliedJobPercentage()
    really supposed to be
    100
    and not maybe let's say,
    1.0f
    ?

  4. Do Objectify ancestor queries not really work as documented?


Answer

The problem is explained by an observation at https://cloud.google.com/appengine/docs/java/tools/localunittesting#Java_Writing_High_Replication_Datastore_tests : "In the local environment, performing a get() of an Entity that belongs to an entity group with an unapplied write will always make the results of the unapplied write visible to subsequent global queries."

In this contect, this means the ancestor-query:

List<DownloadTask> downloadTasks1 = ofy().load().type(DownloadTask.class).ancestor(user).list();

which internally "performs a get() of an Entity that belongs to an entity group with an unapplied write" influences the behavior of the immediately-following global query:

List<DownloadTask> downloadTasks2 = ofy().load().type(DownloadTask.class).list();

To avoid your tests influencing each other, and in particular, interfering w/each other in this way, it's best to use a separate method per operation under test (each with all the needed setup and teardown parts), rather than having successive operations-under-test within a single test method.