cohenadair cohenadair - 1 month ago 7
Swift Question

Swift - Firebase Database query range of items

Consider the following database tree:

root: {
followees: {
<uid-1> : {
// list of all users "uid-1" is following (each child node is a different FCM topic)
<uid-1-1>: true, // true = subscribed to FCM topic
<uid-1-2>: false,
// theoretically, "uid-1" could be following millions of users, but 100k is probably a more reasonable maximum
}
<uid-2> : {
// list of all users "uid-2" is following
}
}
}


What would be the most efficient and memory-saving method to iterate all children in
subtree
?

I've created a recursive solution using a limiting query that reads 1000 children at a time. This works, but due to the recursive aspect, all children are stored in memory until the base case it hit; it's essentially the same as loading all children at once.

I've thought of clearing out the collection returned by
observeSingleEvent(.value)
, but that doesn't work because the collection is immutable.

The best solution I can think of is to query the database for the first 1000 children, then the second 1000, and so on:

query(startIndex:0, endIndex:999)
query(startIndex:1000, endIndex:1999)
query(startIndex:2000, endIndex:2999)
...


How can this be done using Firebase? Can it be done at all? Should my database structure be redesigned so a subtree can't contain millions of entries?

Any advice is appreciated!

P.S. Here's my recursive solution if you're interested. Note it's not valid Swift code -- it just shows the concept.

func iterateChildren(startingAt: String, block: (child) -> Void) {
// The actual query would use queryStartingAt and queryLimited.
ref.query { children in
// Used in recursive call.
var nextStartingId: String? = nil

for (index, child) in children.enumerated() {
if (UInt(index) == limit - 1) {
// Firebase Database's queryStarting(atValue:) method is inclusive of
// atValue, so skip the last item as it will be included in the next
// iteration.
nextStartingId = child.key
break
}
block(child)
}

if (nextStartingId != nil) {
self.iterateChildren(startingAtId: nextStartingId, block: block)
}
}
}

Answer Source

Instead of a single big child, try to use small and concise child. Why you don't try to group your sub child with some logic?

A nice solution might be to group data by insertion date (but depends of your use case):

root : {
   child : {
      "20/09/2017" : {
          "uid" : true  
      },
      "21/09/2017" : {
          "uid" : false
          "uid" : true
      }
   }
}

With this solution you could iterate your child node with observe(.childAdded) where every completionHandler give you a small block of data.

Anyway, what I'm sure is that using a .value observer is a bad solution.

Firebase suggest to avoid god child, so try to follow this simple rule: https://firebase.google.com/docs/database/ios/structure-data