Niels Niels - 5 months ago 148
Android Question

Firebase Android offline performance

When storing approximately 5000 sub nodes under a single node, initialising firebase becomes very slow when making use of the offline capabilities. It takes ~30 seconds before the first query is executed. Once initialised, executing subsequent queries (e.g. listing the first 25 sub nodes) takes less than a second.

I'm making use of the following properties to enable the offline capabilities:
Firebase.getDefaultConfig().setPersistenceEnabled(true);
firebase.keepSynced(true);

My structure looks like this:

<root>
|-my-app-name
|-<uid>
|-node
|-sub node 1
|-...
|-sub node 5000


Keep synced is set on the
<uid>
node. The sub nodes are presented in a Recycler View. Preferably, I would like to list all (instead of 25 per page) but I understand that this is not possible since there is no Cursor like mechanism (as Android provides for SQLite) available for working with Firebase.

Is this by design and do I revise my data structure? Or can I reduce initialisation time in another way?

I provided some logging below. As you can see, a lot of garbage collection is going on. Does Firebase evaluate the whole database when initializing?

Thanks!
Niels

04-01 15:59:12.029 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 43005(1717KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 31MB/32MB, paused 5.674ms total 57.402ms
04-01 15:59:13.415 2222-2240/abcdef W/art: Suspending all threads took: 6.600ms
04-01 15:59:13.424 2222-2245/abcdef W/art: Suspending all threads took: 9.339ms
04-01 15:59:13.433 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 7097(281KB) AllocSpace objects, 0(0B) LOS objects, 0% free, 32MB/32MB, paused 11.175ms total 27.105ms
04-01 15:59:13.821 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 101674(5MB) AllocSpace objects, 18(530KB) LOS objects, 35% free, 28MB/44MB, paused 3.400ms total 152.664ms
04-01 15:59:15.107 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 394024(15MB) AllocSpace objects, 0(0B) LOS objects, 20% free, 30MB/38MB, paused 1.865ms total 152.182ms
04-01 15:59:15.817 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 218328(8MB) AllocSpace objects, 0(0B) LOS objects, 19% free, 31MB/38MB, paused 1.711ms total 112.325ms
04-01 15:59:16.451 2222-2240/abcdef W/art: Suspending all threads took: 27.786ms
04-01 15:59:16.465 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 190591(7MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 31MB/38MB, paused 1.832ms total 107.416ms
04-01 15:59:16.472 2222-2245/abcdef W/art: Suspending all threads took: 6.823ms
04-01 15:59:17.084 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 178714(6MB) AllocSpace objects, 0(0B) LOS objects, 15% free, 32MB/38MB, paused 1.717ms total 105.529ms
04-01 15:59:17.629 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 163584(6MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 33MB/38MB, paused 1.743ms total 110.764ms
04-01 15:59:18.941 2222-2240/abcdef W/art: Suspending all threads took: 5.078ms
04-01 15:59:19.691 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 95627(3MB) AllocSpace objects, 0(0B) LOS objects, 8% free, 35MB/38MB, paused 7.190ms total 86.171ms
04-01 15:59:19.961 2222-2240/abcdef W/art: Suspending all threads took: 18.208ms
04-01 15:59:20.965 2222-2245/abcdef W/art: Suspending all threads took: 5.254ms
04-01 15:59:20.990 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 55899(2MB) AllocSpace objects, 0(0B) LOS objects, 5% free, 36MB/38MB, paused 6.799ms total 66.923ms
04-01 15:59:22.495 2222-2240/abcdef W/art: Suspending all threads took: 45.180ms
04-01 15:59:22.509 2222-2245/abcdef W/art: Suspending all threads took: 14.254ms
04-01 15:59:22.562 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 198174(6MB) AllocSpace objects, 3(487KB) LOS objects, 32% free, 33MB/49MB, paused 16.949ms total 215.369ms
04-01 15:59:23.811 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 392437(15MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.936ms total 168.222ms
04-01 15:59:24.480 2222-2240/abcdef W/art: Suspending all threads took: 22.464ms
04-01 15:59:24.497 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 227043(8MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.723ms total 117.855ms
04-01 15:59:25.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 203910(7MB) AllocSpace objects, 0(0B) LOS objects, 16% free, 36MB/43MB, paused 1.694ms total 112.618ms
04-01 15:59:25.181 2222-2245/abcdef W/art: Suspending all threads took: 7.301ms
04-01 15:59:25.784 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 185627(7MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 37MB/43MB, paused 1.719ms total 115.362ms
04-01 15:59:26.345 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 167066(6MB) AllocSpace objects, 0(0B) LOS objects, 13% free, 37MB/43MB, paused 1.651ms total 106.055ms
04-01 15:59:26.865 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 154535(6MB) AllocSpace objects, 0(0B) LOS objects, 11% free, 38MB/43MB, paused 1.644ms total 104.888ms
04-01 15:59:28.357 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 151375(5MB) AllocSpace objects, 33(671KB) LOS objects, 9% free, 39MB/43MB, paused 2.740ms total 104.176ms
04-01 15:59:29.006 2222-2240/abcdef W/art: Suspending all threads took: 19.232ms
04-01 15:59:29.060 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 133554(5MB) AllocSpace objects, 29(580KB) LOS objects, 10% free, 39MB/43MB, paused 1.563ms total 100.220ms
04-01 15:59:30.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 131062(4MB) AllocSpace objects, 31(637KB) LOS objects, 9% free, 39MB/43MB, paused 1.653ms total 102.705ms
04-01 15:59:31.245 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 122085(4MB) AllocSpace objects, 26(522KB) LOS objects, 8% free, 39MB/43MB, paused 2.380ms total 100.776ms
04-01 15:59:32.024 2222-2240/abcdef W/art: Suspending all threads took: 20.662ms


PS: This is a cross post: https://groups.google.com/forum/#!topic/firebase-talk/migEAwv26ns

Answer

So sharding the data so one root node contains at most 200 sub nodes seems to be the answer for now. I'm setting .keepSynced(true) on the shards and this results in much better performance.

In order to present the sharded list in a single recycler view, I created a class FirebaseArrays which is a collection of FirebaseArray that aggregates multiple arrays into a single observable collection.
https://gist.github.com/ndefeijter/2191f8a43ce903c5d9ea69f02c7ee7e9

I also adapted the FirebaseRecyclerAdapter to use a FirebaseArrays as underlying data structure instead of a single FirebaseArray. The interface is extended using some methods to add additional Firebase paths (i.e. shards).
https://gist.github.com/ndefeijter/d98eb5f643b5faf5476b8a611de912c1

These paths are added upon a 'load more' event (e.g. in case endless scrolling).

private void loadMore() {

    final View view = getView();
    if (null != view) {
        final RecyclerView recyclerView = (RecyclerView) view.findViewById(R.id.recycler_view);
        final FirebaseRecyclerAdapter2<Visit, VisitViewHolder> adapter = (FirebaseRecyclerAdapter2<Visit, VisitViewHolder>) recyclerView.getAdapter();
        adapter.addQuery(nextQuery());
    }
}
Comments