cora cora - 1 year ago 104
Java Question

ThreadPoolTaskExecutor missing tasks

I am attempting to diagnose an issue where a list that I feed into a taskExecutor service is processed through EVERY OTHER TIME. I am loading these from a helper table and when I look back at the results, every other row shows the expected data, and every other is null. The below method accepts the list and once there is a thread available, sends task to the next method to be queried and sent on a queue to another application. Is there any obvious reason half my tasks are being discarded/rejected with no error? It was my understanding that CALLER_RUNS_POLICY shuffled missed tasks back to the back of the line, i.e. the task queue.

public void processList(MySQLDao mySQLDao, RabbitService rabbitService, List<Integer> productIds) {
Iterator<Integer> productIterator = productIds.iterator();
taskExecutor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
while (productIterator.hasNext()) {
while (0 == taskExecutor.getThreadPoolExecutor().getQueue().remainingCapacity()) {
try {
Thread.sleep(100);"threadPool full, sleeping 100 mil");
} catch (InterruptedException ie) {
try {
taskExecutor.execute(new ProcessTask(mySQLDao, rabbitService,;"executing task for id: " +;
} catch (TaskRejectedException e) {
logger.error(e.getMessage(), "Task rejected");

public void run() {

Product product = mySQLDao.selectOne("productSql", Product.class, id, "ProductTask");
if (product != null) {
rabbitService.sendItemToQueue("my_exchange", "product-mongo", "product-mongo", product);
} else {
logger.error(id + "returned null query result");
}"sending id " + id + " to rabbit queue");

Here is the bean setup in application-context.xml:

<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="20" />
<property name="maxPoolSize" value="25" />
<property name="queueCapacity" value="50" />
<property name="WaitForTasksToCompleteOnShutdown" value="true" />
<property name="awaitTerminationSeconds" value="5" />

Answer Source

You're advancing the iterator twice, thus throwing away half of your tasks:

        taskExecutor.execute(new ProcessTask(mySQLDao, rabbitService,;"executing task for id: " +;//this is advancing the iterator and discarding the task.

Here's how you should do it:

       Integer nextProductId =;
       taskExecutor.execute(new ProcessTask(mySQLDao, rabbitService, nextProductId));"executing task for id: " + nextProductId);
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download