VLeonovs VLeonovs - 1 month ago 14
Android Question

Firebase Database JSON loading time

I'm using Firebase Database to store JSON with a size of 5MB. I would like to download this JSON and store it locally, but the speed of response is not good:

10-03 22:44:10.514 D: I'm started
10-03 22:47:39.909 D: I'm finished


And the code:

Log.d(TAG, "I'm started");
final FirebaseDatabase database = FirebaseDatabase.getInstance();
DatabaseReference myRef = database.getReference();
myRef.addValueEventListener(new ValueEventListener() {
@Override
public void onDataChange(DataSnapshot dataSnapshot) {
Log.d(TAG, "I'm finished");
}
}


Is it a good time response? How to reduce this? I'm not sure that my WiFi is fast, but maybe the size of the JSON is very huge?

Edit

After I have setted
database.setLogLevel(Logger.Level.DEBUG)
the log starts with:

10-05 23:01:51.853 W: ClassLoader referenced unknown path: /system/priv-app/PrebuiltGmsCore/lib/x86
10-05 23:01:51.874 I: Considering local module com.google.android.gms.firebase_database:3 and remote module com.google.android.gms.firebase_database:4
10-05 23:01:51.874 I: Selected remote version of com.google.android.gms.firebase_database, version >= 4
10-05 23:01:51.881 W: ClassLoader referenced unknown path: /data/user/0/com.google.android.gms/app_chimera/m/00000005/n/x86
10-05 23:01:51.891 D: pc_0 - Scheduling connection attempt
10-05 23:01:51.891 D: Scheduling retry in 0ms
10-05 23:01:51.903 D: pc_0 - Trying to fetch auth token
10-05 23:01:51.916 D: pc_0 - Listening on / (params: {})
10-05 23:01:51.916 D: pc_0 - Adding listen query: / (params: {})
10-05 23:01:51.916 D: pc_0 - Successfully fetched token, opening connection
10-05 23:01:51.923 D: conn_0 - Opening a connection
10-05 23:01:52.683 D: ws_0 - websocket opened
10-05 23:01:52.683 D: ws_0 - Reset keepAlive
10-05 23:01:52.707 D: ws_0 - ws message: {"t":"c","d":{"t":"r","d":"s-usc1c-nss-105.firebaseio.com"}}
10-05 23:01:52.708 D: ws_0 - Reset keepAlive. Remaining: 44975
10-05 23:01:52.709 D: ws_0 - HandleNewFrameCount: 1
10-05 23:01:52.713 D: ws_0 - handleIncomingFrame complete frame: {d={d=s-usc1c-nss-105.firebaseio.com, t=r}, t=c}
10-05 23:01:52.713 D: conn_0 - Got control message: {d=s-usc1c-nss-105.firebaseio.com, t=r}
10-05 23:01:52.713 D: conn_0 - Got a reset; killing connection to daugavpils-1efea.firebaseio.com; Updating internalHost to s-usc1c-nss-105.firebaseio.com
10-05 23:01:52.714 D: conn_0 - closing realtime connection
10-05 23:01:52.714 D: ws_0 - websocket is being closed
10-05 23:01:52.714 D: pc_0 - Got on disconnect due to SERVER_RESET
10-05 23:01:52.714 D: pc_0 - Scheduling connection attempt
10-05 23:01:52.715 D: Scheduling retry in 0ms
10-05 23:01:52.716 D: pc_0 - Trying to fetch auth token
10-05 23:01:52.717 D: pc_0 - Successfully fetched token, opening connection
10-05 23:01:52.722 D: conn_1 - Opening a connection
10-05 23:01:52.723 D: ws_0 - WebSocket reached EOF.
10-05 23:01:52.725 D: ws_0 - closed
10-05 23:01:53.462 D: ws_1 - websocket opened
10-05 23:01:53.462 D: ws_1 - Reset keepAlive
10-05 23:01:53.463 D: ws_1 - ws message: {"t":"c","d":{"t":"h","d":{"ts":1475697713467,"v":"5","h":"s-usc1c-nss-105.firebaseio.com","s":"aiVwAqOfKkxgXWZdYejceVrlivZ8MPTA"}}}
10-05 23:01:53.465 D: ws_1 - Reset keepAlive. Remaining: 44996
10-05 23:01:53.465 D: ws_1 - HandleNewFrameCount: 1
10-05 23:01:53.467 D: ws_1 - handleIncomingFrame complete frame: {d={d={v=5, h=s-usc1c-nss-105.firebaseio.com, ts=1475697713467, s=aiVwAqOfKkxgXWZdYejceVrlivZ8MPTA}, t=h}, t=c}
10-05 23:01:53.468 D: conn_1 - Got control message: {d={v=5, h=s-usc1c-nss-105.firebaseio.com, ts=1475697713467, s=aiVwAqOfKkxgXWZdYejceVrlivZ8MPTA}, t=h}
10-05 23:01:53.468 D: conn_1 - realtime connection established
10-05 23:01:53.468 D: pc_0 - onReady
10-05 23:01:53.468 D: pc_0 - handling timestamp
10-05 23:01:53.475 D: pc_0 - Sending first connection stats
10-05 23:01:53.476 D: conn_1 - Sending data: {d={a=s, r=0, b={c={sdk.android.3-0-0=1}}}, t=d}
10-05 23:01:53.476 D: ws_1 - Reset keepAlive. Remaining: 44989
10-05 23:01:53.481 D: pc_0 - calling restore state
10-05 23:01:53.482 D: pc_0 - Not restoring auth because token is null.
10-05 23:01:53.482 D: pc_0 - Restoring outstanding listens
10-05 23:01:53.482 D: pc_0 - Restoring listen / (params: {})
10-05 23:01:53.484 D: conn_1 - Sending data: {d={a=q, r=1, b={h=, p=/}}, t=d}
10-05 23:01:53.486 D: ws_1 - Reset keepAlive. Remaining: 44991
10-05 23:01:53.491 D: pc_0 - Restoring writes.
10-05 23:01:53.648 D: ws_1 - ws message: {"t":"d","d":{"r":0,"b":{"s":"ok","d":""}}}
10-05 23:01:53.649 D: ws_1 - Reset keepAlive. Remaining: 44838
10-05 23:01:53.649 D: ws_1 - HandleNewFrameCount: 1
10-05 23:01:53.649 D: ws_1 - handleIncomingFrame complete frame: {d={r=0, b={d=, s=ok}}, t=d}
10-05 23:01:53.649 D: conn_1 - received data message: {r=0, b={d=, s=ok}}
10-05 23:01:54.114 D: ws_1 - ws message: 368
10-05 23:01:54.114 D: ws_1 - Reset keepAlive. Remaining: 44534
10-05 23:01:54.114 D: ws_1 - HandleNewFrameCount: 368
10-05 23:01:54.415 D: ws_1 - ws message: {"t":"d","d":{"b":{"p":"","d":{"route":{"autobuss-nr-1-autoosta-regionala-slimnica-celinieku-ciemats":{"routes":{"0":{"lat":"55.871109","lng":"26.523550"},"1":{"lat":"55.869579","lng":"26.519173"},"2":{"lat":"55.868752","lng":"26.519945"},"3":{"lat":"55.866943","lng":"26.514666"},"4":{"lat":"55.865353","lng":"26.520460"},"5":{"lat":"55.866463","lng":"26.525246"},"6":{"lat":"55.867714","lng":"26.530544"},"7":{"lat":"55.868931","lng":"26.535587"},"8":{"lat":"55.869690","lng":"26.538033"},"9":{"lat":"55.870590","lng":"26.539707"},"10":{"lat":"55.872761","lng":"26.543268"},"11":{"lat":"55.875336","lng":"26.547388"},"12":{"lat":"55.877731","lng":"26.551596"},"13":{"lat":"55.880199","lng":"26.555521"},"14":{"lat":"55.880825","lng":"26.556894"},"15":{"lat":"55.881340","lng":"26.558439"},"16":{"lat":"55.882389","lng":"26.561680"},"17":{"lat":"55.884327","lng":"26.567753"},"18":{"lat":"55.888271","lng":"26.580027"},"19":{"lat":"55.890450","lng":"26.580734"},"20":{"lat":"55.893158","lng":"26.581539"},"21":{"lat":"55.898533","lng":"26.583214"},"22":{"lat":"55.898766","lng":"26.588503"},"23":{"lat":"55.895168","lng":"26.590389"},"24":{"lat":"55.891716","lng":"26.592793"},"25":{"lat":"55.893604","lng":"26.600262"},"26":{"lat":"55.894588","lng":"26.604059"},"27":{"lat":"55.895119","lng":"26.606291"},"28":{"lat":"55.895420","lng":"26.607019"},"29":{"lat":"55.895721","lng":"26.607342"},"30":{"lat":"55.898380","lng":"26.609659"},"31":{"lat":"55.901489","lng":"26.612568"},"32":{"lat":"55.904564","lng":"26.615345"},"33":{"lat":"55.907810","lng":"26.618135"}},"stations":{"0":{"decription":"","geo":{"lat":"55.870842","lng":"26.525305"},"htlist":{"0":"06:25","1":"07:25","2":"08:30","3":"11:05","4":"12:20","5":"13:45","6":"15:10","7":"16:25","8":"17:40"},"name":"Autoosta","number":"1","other":{"0":{"link":"autobuss-nr-1-autoosta-regionala-slimnica-celinieku-ciemats","name":"Autoosta – reģionālā slimnīca – Ceļinieku ciemats","number":"1","ohtlist":{"0":"06:25","1":"07:25","2":"08:30","3":"11:05","4":"12:20","5":"13:45","6":"15:10","7":"16:25","8":"17:40"},"owtlist":{"0":"06:25","1":"07:25","2":"08:30","3":"11:05","4":"12:20","5":"13:45","6":"15:10","7":"16:25","8":"17:40"},"sid":"8768"},"1":{"link":"autobuss-nr-1a-autoosta-celinieku-ciemats","name":"Autoosta - Ceļinieku ciemats","number":"1A","owtlist":{"0":"06:50"},"sid":"8768"},"2":{"link":"autobuss-nr-3-autoosta-kooperativs","name":"Autoosta – Jaunie Stropi (Kooperatīvs)","number":"3","ohtlist":{"0":"07:50","1":"09:00","2":"10:30","3":"11:40","4":"12:50","5":"14:00","6":"15:00","7":"16:10","8":"17:20","9":"18:35","10":"19:50"},"owtlist":{"0":"05:48","1":"06:35","2":"07:50","3":"09:00","4":"10:30","5":"11:40","6":"12:50","7":"14:00","8":"15:00","9":"16:10","10":"17:20","11":"18:35","12":"19:50"},"sid":"8768"},"3":{"link":"autobuss-nr-3b-autoosta-ziglier-masinbuve","name":"Autoosta – Spaļu iela – Zieglera Mašīnbūve","number":"3B","ohtlist":{"0":"07:05","1":"08:05","2":"09:05","3":"10:35","4":"11:25","5":"12:20","6":"13:15","7":"14:45","8":"15:35","9":"16:30","10":"17:25","11":"18:20","12":"19:50","13":"20:40","14":"21:30"},"owtlist":{"0":"05:35","1":"06:17","2":"07:05","3":"08:05","4":"09:05","5":"10:35","6":"11:25","7":"12:20","8":"13:15","9":"14:45","10":"15:35","11":"16:30","12":"17:25","13":"18:20","14":"19:50","15":"20:40","16":"21:30"},"sid":"8768"},"4":{"link":"autobuss-nr-3c-autoosta-ziglier-masinbuve","name":"Autoosta – Maiznieks – Spaļu iela","number":"3C","ohtlist":{"0":"08:00","1":"08:50","2":"09:40","3":"11:15","4":"12:05","5":"13:00","6":"13:55","7":"15:25","8":"16:20","9":"17:15","10":"18:10","11":"19:08"},"owtlist":{"0":"06:05","1":"06:58","2":"08:00","3":"08:50","4":"09:40","5":"11:15","6":"12:05","7":"13:00","8":"13:55","9":"15:25","10":"16:20","11":"17:15","12":"18:10","13":"19:08"},"sid":"8768"},"5":{"link":"autobuss-nr-4-autoosta-cietoksnis-mezciems-autoosta","name":"Autoosta – Cietoksnis – Mežciems – Autoosta","number":"4","ohtlist":{"0":"05:15","1":"06:20","2":"07:20","3":"08:20","4":"09:
10-05 23:01:54.417 D: ws_1 - Reset keepAlive. Remaining: 44697
10-05 23:01:54.594 D: ws_1 - ws message: s-nr-21-daugavpils-ao-cietoksnis","name":"Daugavpils AO - Cietoksnis-Mežciems-Viduspoguļanka-Jaunforštadte-Daugavpils AO","number":"21","ohtlist":{"0":"03:40","1":"04:40"},"owtlist":{"0":"03:40","1":"04:40"},"sid":"8768"},"27":{"link":"autobuss-nr-22-autobusu-parks-krizi","name":"Autobusu parks – Daugavpils AO – Nīderkuni – Judovka – Ruģeļi-Jaunbūve – Ķimiķu c-ts-Jaunstropi – Vecstropi – Križi","number":"22","ohtlist":{"0":"01:10"},"owtlist":{"0":"01:10"},"sid":"8768"},"28":{"link":"autobuss-nr-22-krizi-autobusu-parks","name":"Križi – Vecstropi – Jaunstropi-Ķimiķu c-ts – Jaunbūve-Ruģeļi – Nīderkuni – Daugavpils AO – Autobusu parks","number":"22","ohtlist":{"0":"04:00","1":"05:15"},"owtlist":{"0":"04:00","1":"05:15"},"sid":"8768"},"29":{"link":"autobuss-nr-23-rugeli-regionala-slimnica","name":"Ruģeļi - Reģionālā slimnīca","number":"23","owtlist":{"0":"06:57","1":"-","2":"15:35"},"sid":"8768"},"30":{"link":"autobuss-nr-14-autoosta-cietoksnis-kimija-autoosta","name":"Autoosta - Cietoksnis – Ķīmiķu ciemats (ar iebraukšanu Plaušu centrā)","number":"14","owtlist":{"0":"06:25","1":"07:04","2":"07:58","3":"15:30","4":"16:45"},"sid":"8768"}},"sid":"8768","wtlist":{"0":"06:25","1":"07:25","2":"08:30","3":"11:05","4":"12:20","5":"13:45","6":"15:10","7":"16:25","8":"17:40"}},"1":{"decription":"","geo":{"lat":"55.867573","lng":"26.516882"},"htlist":{"0":"06:27","1":"07:27","2":"08:32","3":"11:07","4":"12:22","5":"13:47","6":"15:12","7":"16:27","8":"17:42"},"name":"Daugavas iela","number":"1","other":{"0":{"link":"autobuss-nr-1-autoosta-regionala-slimnica-celinieku-ciemats","name":"Autoosta – reģionālā slimnīca – Ceļinieku ciemats","number":"1","ohtlist":{"0":"06:27","1":"07:27","2":"08:32","3":"11:07","4":"12:22","5":"13:47","6":"15:12","7":"16:27","8":"17:42"},"owtlist":{"0":"06:27","1":"07:27","2":"08:32","3":"11:07","4":"12:22","5":"13:47","6":"15:12","7":"16:27","8":"17:42"},"sid":"8769"},"1":{"link":"autobuss-nr-1a-autoosta-celinieku-ciemats","name":"Autoosta - Ceļinieku ciemats","number":"1A","owtlist":{"0":"06:52"},"sid":"8769"},"2":{"link":"autobuss-nr-3-autoosta-kooperativs","name":"Autoosta – Jaunie Stropi (Kooperatīvs)","number":"3","ohtlist":{"0":"07:52","1":"09:02","2":"10:32","3":"11:42","4":"12:52","5":"14:02","6":"15:02","7":"16:12","8":"17:22","9":"18:37","10":"19:52"},"owtlist":{"0":"05:50","1":"06:37","2":"07:52","3":"09:02","4":"10:32","5":"11:42","6":"12:52","7":"14:02","8":"15:02","9":"16:12","10":"17:22","11":"18:37","12":"19:52"},"sid":"8769"},"3":{"link":"autobuss-nr-3b-autoosta-ziglier-masinbuve","name":"Autoosta – Spaļu iela – Zieglera Mašīnbūve","number":"3B","ohtlist":{"0":"07:07","1":"08:07","2":"09:07","3":"10:37","4":"11:27","5":"12:22","6":"13:17","7":"14:47","8":"15:37","9":"16:32","10":"17:27","11":"18:22","12":"19:52","13":"20:42","14":"21:32"},"owtlist":{"0":"05:37","1":"06:19","2":"07:07","3":"08:07","4":"09:07","5":"10:37","6":"11:27","7":"12:22","8":"13:17","9":"14:47","10":"15:37","11":"16:32","12":"17:27","13":"18:22","14":"19:52","15":"20:42","16":"21:32"},"sid":"8769"},"4":{"link":"autobuss-nr-3c-autoosta-ziglier-masinbuve","name":"Autoosta – Maiznieks – Spaļu iela","number":"3C","ohtlist":{"0":"08:02","1":"08:52","2":"09:42","3":"11:17","4":"12:07","5":"13:02","6":"13:57","7":"15:27","8":"16:22","9":"17:17","10":"18:12","11":"19:10"},"owtlist":{"0":"06:07","1":"07:00","2":"08:02","3":"08:52","4":"09:42","5":"11:17","6":"12:07","7":"13:02","8":"13:57","9":"15:27","10":"16:22","11":"17:17","12":"18:12","13":"19:10"},"sid":"8769"},"5":{"link":"autobuss-nr-6-autoosta-niderkuni","name":"Autoosta - Nīderkūni","number":"6","ohtlist":{"0":"07:52","1":"08:32","2":"09:12","3":"10:52","4":"11:42","5":"12:22","6":"13:02","7":"13:42","8":"14:27","9":"15:42","10":"16:22","11":"17:02","12":"17:42","13":"18:22","14":"19:02","15":"19:42","16":"20:42"},"owtlist":{"0":"06:32","1":"07:12","2":"07:52","3":"08:32","4":"09:12","5":"10:52","6":"11:42","7":"12:22","8":"13:02","
10-05 23:01:54.594 D: ws_1 - Reset keepAlive. Remaining: 44822


After that a lot of output with:
enter image description here

And this all finish with:

enter image description here

enter image description here

I'm using default bandwidth.

Answer

What I found ?

The Firebase is bad working with the large structured data (I don't believe that I can make problems for GOOGLE with the size of json 5mb, but it happens).

My approach is: to reduce loading time I need to avoid requesting all data in one time.

Better to divide it to several requests.

To simplify my JSON:

{
  routes: {
    routeName: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName1: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName2: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName3: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName4: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName5: {
      gpsData: {

      },
      timeData: {

      }
    }
  },
  routesReverse: {
    routeName: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName1: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName2: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName3: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName4: {
      gpsData: {

      },
      timeData: {

      }
    },
    routeName5: {
      gpsData: {

      },
      timeData: {

      }
    }
  }
}

The problems were with this code:

Log.d(TAG, "I'm started");
final FirebaseDatabase database = FirebaseDatabase.getInstance();
        DatabaseReference myRef = database.getReference();
        myRef.addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                Log.d(TAG, "I'm finished");
            }
        }

The problem was gone:

Log.d(TAG, "I'm started");
final FirebaseDatabase database = FirebaseDatabase.getInstance();
        DatabaseReference myRef = database.getReference().child("route").child("routeName2");
        myRef.addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                Log.d(TAG, "I'm finished");
            }
        }

And the time reduced from 2-3 minutes to 2-3 seconds.