AntoineB AntoineB - 4 months ago 30
Java Question

Log4J - JsonLayout and RollingFileAppender generating invalid JSON

I'm trying to store my logs in JSON files using Log4J (2.6.2). I'm using a

JsonLayout
in a
RollingFileAppender
, and it's working properly as long as I'm not trying to append to a file that has already been written previously.

Here's my code for setting up the Layout and the Appender:

Layout<?> layout = JsonLayout.createLayout(config, false, false, false, true, false, true, "[", "]", Charset.defaultCharset());

String appenderFileName = "mylogfile-latest.log.json";
String appenderFilePattern = "mylogfile-%i.log.json";
String appenderName = "MyAppender";
Appender appender = RollingFileAppender.createAppender(appenderFileName, appenderFilePattern, "true", appenderName, "true", "256", "true",
SizeBasedTriggeringPolicy.createPolicy(this.configuration.getLogMaxSize().toString()),
null, layout, null, "false", "false", null, config);


As I said, it's working fine the first time I'm writing in my log file:

[
{
"timeMillis" : 1469620840442,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "ERROR",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 243,
"threadPriority" : 5
}
, {
"timeMillis" : 1469620840442,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "DEBUG",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 243,
"threadPriority" : 5
}
]


I then close my appender, etc... and relaunch my application, and when I'm writing new logs to this existing file, here's what I'm getting:

[
{
"timeMillis" : 1469620840442,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "ERROR",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 243,
"threadPriority" : 5
}
, {
"timeMillis" : 1469620840442,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "DEBUG",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 243,
"threadPriority" : 5
}

]
{
"timeMillis" : 1469620840490,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "ERROR",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 245,
"threadPriority" : 5
}
, {
"timeMillis" : 1469620840492,
"thread" : "SimpleAsyncTaskExecutor-43",
"level" : "DEBUG",
"loggerName" : "MyLogger",
"message" : "my log message",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 245,
"threadPriority" : 5
}

]


This issue makes it impossible to parse the logs as Gson will fail to create a
List
from this file.

Do you have any idea on how to fix this issue without doing a crappy hack removing the extra
]
and replacing it with a comma?

Thanks!

Answer

Since no other solution was satisfying, I ended up using a regex to replace the invalid characters...

It's pretty safe since it checks if it's not in between two double-quotes, but it's not the ideal solution obviously:

String content = new String(Files.readAllBytes(Paths.get(pathToLogFile)));
content = content.replaceAll("(\\}\\R{2}\\]\\R\\{)(?=(?:(?:\\.|[^\"\\\\])*\"(?:\\.|[^\"\\\\])*\")*(?:\\.|[^\"\\\\])*)", "}\n, {");
Files.write(Paths.get(pathToLogFile), content.getBytes());
Comments