MIkCode MIkCode - 3 months ago 29
Scala Question

Scala/Akka WSResponse recursively call

Im trying to parse some data from an API

I have a recursion method that calling to this method

def getJsonValue( url: (String)): JsValue = {
val builder = new com.ning.http.client.AsyncHttpClientConfig.Builder()
val client = new play.api.libs.ws.ning.NingWSClient(builder.build())
val newUrl = url.replace("\"", "").replace("|", "%7C").trim
val response: Future[WSResponse] = client.url(newUrl).get()
Await.result(response, Duration.create(10, "seconds")).json
}


Everything is working well but after 128 method calls i'm getting this warning

WARNING: You are creating too many HashedWheelTimer instances. HashedWheelTimer is a shared resource that must be reused across the application, so that only a few instances are created.


After about 20 More calls im getting this exception

23:24:57.425 [main] ERROR com.ning.http.client.AsyncHttpClient - Unable to instantiate provider com.ning.http.client.providers.netty.NettyAsyncHttpProvider. Trying other providers.
23:24:57.438 [main] ERROR com.ning.http.client.AsyncHttpClient - org.jboss.netty.channel.ChannelException: Failed to create a selector.


Questions

1.Im assuming that the connections didnt closed ?? and therefore i can't create new connections.

2.What will be the correct and the safe way to create those HTTP calls

Answer

Had the same problem.

Found 2 interesting solutions:

  • make sure you are not creating tons of clients with closing them
  • the threadPool you are using may be causing this.

My piece of code (commenting that line of code solved, I'm now testing several configurations):

  private[this] def withClient(block: NingWSClient => WSResponse): Try[WSResponse] = {
    val config = new NingAsyncHttpClientConfigBuilder().build()
    val clientConfig = new AsyncHttpClientConfig.Builder(config)
     // .setExecutorService(new ThreadPoolExecutor(5, 15, 30L, TimeUnit.SECONDS, new SynchronousQueue[Runnable]))
      .build()
    val client = new NingWSClient(clientConfig)
    val result = Try(block(client))
    client.close()
    result
  }
Comments