Reza Reza - 2 months ago 59
HTTP Question

Vertx http server Thread has been blocked for xxxx ms, time limit is 2000

i have written a large scale http server using , but im getting this error when number of concurrent requests increases

WARNING: Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 8458 ms, time limit is 1000


io.vertx.core.VertxException: Thread blocked

here is my full code :

public class MyVertxServer {

public Vertx vertx = Vertx.vertx(new VertxOptions().setWorkerPoolSize(100));
private HttpServer server = vertx.createHttpServer();
private Router router = Router.router(vertx);

public void bind(int port){


server.requestHandler(router::accept).listen(port);

}

public void createContext(String path,MyHttpHandler handler){

if(!path.endsWith("/")){
path += "/";
}

path+="*";


router.route(path).handler(new Handler<RoutingContext>() {

@Override
public void handle(RoutingContext ctx) {

String[] handlerID = ctx.request().uri().split(ctx.currentRoute().getPath());

String suffix = handlerID.length > 1 ? handlerID[1] : null;
handler.Handle(ctx, new VertxUtils(), suffix);

}
});
}


}


and how i call it :

ver.createContext("/getRegisterManager",new ProfilesManager.RegisterHandler());
ver.createContext("/getLoginManager", new ProfilesManager.LoginHandler());
ver.createContext("/getMapcomCreator",new ItemsManager.MapcomCreator());
ver.createContext("/getImagesManager", new ItemsManager.ImagesHandler());

ver.bind(PORT);


how ever i dont find eventbus() useful for http servers that process send/receive files , because u need to send the RoutingContext in the message with is not possible.

could you please point me to the right direction? thanks

added a little bit of handler's code:

class ProfileGetter implements MyHttpHandler{

@Override
public void Handle(RoutingContext ctx, VertxUtils utils, String suffix) {

String username = utils.Decode(ctx.request().headers().get("username"));
String lang = utils.Decode(ctx.request().headers().get("lang"));

display("profile requested : "+username);

Profile profile = ProfileManager.FindProfile(username,lang);

if(profile == null){
ctx.request().response().putHeader("available","false");
utils.sendResponseAndEnd(ctx.response(),400);
return;
}else{
ctx.request().response().putHeader("available","true");
utils.writeStringAndEnd(ctx, new Gson().toJson(profile));
}

}



}


here ProfileManager.FindProfile(username,lang) does a long running database job on the same thread

...

basically all of my processes are happening on the main thread , because if i use executor i will get strange exceptions and nullpointers in Vertx , making me feel like the request proccessors in Vertx are parallel

Answer

Given the small amount of code in the question lets agree that the problem is on the line:

Profile profile = ProfileManager.FindProfile(username,lang);

Assuming that this is internally doing some blocking JDBC call which is a anti-pattern in Vert.x you can solve this in several ways.

Say that you can totally refactor the ProfileManager class which IMO is the best then you can update it to be reactive, so your code would be like:

ProfileManager.FindProfile(username,lang, res -> {
  if (res.failed()) {
    // handle error, sent 500 back, etc...
  } else {
    Profile profile = res.result();

    if(profile == null){
      ctx.request().response().putHeader("available","false");
      utils.sendResponseAndEnd(ctx.response(),400);
      return;
    }else{
      ctx.request().response().putHeader("available","true");
      utils.writeStringAndEnd(ctx, new Gson().toJson(profile));
    }
  }
});

Now what would be hapening behind the scenes is that your JDBC call would not block (which is tricky because JDBC is blocking by nature). So to fix this and you're lucky enough to use MySQL or Postgres then you could code your JDBC against the async-client if you're stuck with other RDBMS servers then you need to use the jdbc-client which in turn will use a thread pool to offload the work from the event loop thread.

Now say that you cannot change the ProfileManager code then you can still off load it to the thread pool by wrapping the code in a executeBlocking block:

vertx.executeBlocking(future -> {
  Profile profile = ProfileManager.FindProfile(username,lang);
  future.complete(profile);
}, res -> {
  if (res.failed()) {
    // handle error, sent 500 back, etc...
  } else {
    Profile profile = res.result();

    if(profile == null){
      ctx.request().response().putHeader("available","false");
      utils.sendResponseAndEnd(ctx.response(),400);
      return;
    }else{
      ctx.request().response().putHeader("available","true");
      utils.writeStringAndEnd(ctx, new Gson().toJson(profile));
    }
  }
});
Comments