amoe amoe - 2 years ago 103
Java Question

How to change log level of userlogs in Hadoop 2?

I've been trying to change the log level on userlogs i.e the files that appear under

on CDH 5.2.1. However, no matter what I try, only
level logs will appear. I want to enable
level logs for debugging.

Things I have tried so far:

  • Setting all loggers to TRACE level in

  • Setting

  • Setting
    in the job configuration before submitting it.

  • Including a
    in my job jar file that sets the root Log4j logger to TRACE.

  • Modifying
    to specify

None of these worked -- they didn't break anything, but they didn't have any effect on the log outputs under the
directory. Modifying did cause the ResourceManager and NodeManager logs to enter trace level. Unfortunately these are not useful for my purpose.

I get the following error appearing in
that may be relevant.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/hadoop-yarn/nm-local-dir/usercache/tomcat/appcache/application_1419961570089_0001/filecache/10/job.jar/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See for more info.

I don't understand why the log4j "no configuration" message would happen, given that there is a
file at the root of the job jar file that specifies a root logger:

log4j.rootLogger=TRACE, stdout

log4j.appender.stdout.layout.ConversionPattern=%5p [%t] %m%n

My code does not knowingly use SLF4J for logging, it purely uses Log4j.

Answer Source

The actual answer was to set to the level you need, but, crucially, it needs to be set in the Hadoop job configuration at submission time. It cannot be set on the cluster globally. The cluster global will always default to INFO, as it is hardcoded.

Using alone will not work as YARN will override the log level value on the command line. See the method addLog4jSystemProperties of org.apache.hadoop.mapreduce.v2.util.MRApps and cross reference with org.apache.hadoop.mapreduce.MRJobConfig. will indeed be honoured, but it can't override the level set by this property.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download