I am trying to tinker with the YARN container allocation code. By container allocation, I mean the decision to place the container on a specific machine in the cluster.
I want to write my own container allocation code. To begin with, I am running Hadoop in pseudo-distributed mode with YARN. I am trying to locate the relevant points in the source code. So far, using print statements, I have been able to pinpoint the class
I have not been able to print anything
At first, I thought logging is application specific but all information related to resource manager is under log file named
log folder. Instead of print statement, I used
LOG.info for debugging.
Location of allocation mechanism in hadoop source code
I am using FIFO scheduler and allocation mechanism is under method
FifoScheduler#assignContainersOnNode which is called from
FifoScheduler#assignContainers which is called from
FifoScheduler#handle method (more information here), which keeps on tracking of different events.
NODE_UPDATE is among one of those which is triggered often and hence assignment of container on given node takes place.