costa costa - 1 month ago 9
Scala Question

Is it advisable to use for comprehensions with yield to return a huge number of items?

I am working on a little program that generates combinations and I am using

for
comprehensions. Something like this:

def posibilities2(n: Int): Seq[List[Int]] = {
val maxValues = (1 to 3).map(i => n / i).toList
for {
n1 <- 0 to maxValues(0)
n2 <- 0 to maxValues(1)
n3 <- 0 to maxValues(2)
if n1 * 1 + n2 * 2 + n3 * 3 == n
}
yield List(n1, n2, n3)
}

posibilities2(1000).foreach(doSomething)


For bigger values of n it can lead to lots of items.

My question is this: Is this the way to do it, given that, for each item generated, I have to do some additional processing? I am not concerned about the program taking a long time to run, I am concerned about running out of memory.

Thank you

Answer

As amount of values for-comprehension produces is very high its better to go for iterator or stream implementation.

The below function will generate values on demand basis thus not risking out of memory errors.

  def posibilities2(n: Int): Iterator[(Int, Int, Int)] = {

    val maxValues = (1 to 3).map(i => n / i).toList
    for {
      n1 <- (0 to maxValues(0)).toIterator
      n2 <- (0 to maxValues(1)).toIterator
      n3 <- (0 to maxValues(2)).toIterator
      if n1 * 1 + n2 * 2 + n3 * 3  == n
    } yield (n1, n2, n3)

  }