Java log4j2: Sanitising / Editing Log Messages

Intro

It feels like quite a common requirement to want to sanitise your Java application logs to remove passwords, PII, or other sensitive data. This took far more Googling than I had expected – particularly if you are using JsonLayout or JsonTemplateLayout, as it seems log4j2 is a bit of a minefield of complexity.

The simplest way I have found to achieve this requirement is to write a Rewrite Policy for log4j’s Rewrite appender.

Some Context

Your log4j config probably looks a bit like this… I won’t claim to know very much at all about log4j configs but here’s a basic config to output your logs as JSON:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration shutdownHook="disable">
  <Appenders>
    <!-- Output JSON to console -->
    <Console name="Console" target="SYSTEM_OUT">
      <JsonLayout complete="false" compact="true" eventEol="true">
        <KeyValuePair key="traceId" value="$${ctx:correlationId:-}" />
      </JsonLayout>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="info">
      <!-- Call the Console Appender first -->
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

What you have here is a single “Console” Appender which uses the JsonLayout class to output JSON formatted logs to stdout, adding an extra field called traceId.

Appenders in log4j2 can be chained. So to solve this problem, we want to add a new Appender before our Console Appender. log4j2 has a Rewrite Appender and we can write Rewrite Policy for this. This is what we shall do.

The Java Bit

The Rewrite Appender uses Policies. A Policy is a class which extends RewritePolicy. Policies need a static factory method and to override the rewrite() method of the parent.

rewrite() method takes a LogEvent object and must also return a LogEvent. Because… reasons?… LogEvent is entirely immutable. As such, you must create a new LogEvent using Log4jLogEvent.Builder and copy all of the relevant stuff into it. The Builder has a setMessage() method. This is where you can add a modified message.

Alas, setMessage() doesn’t take a String. It takes an Object which implements the Message interface. There’s an awful lot of standard log4j2 classes which implement this interface. The most basic of which seems to be SimpleMessage. The Message interface implements a getFormattedMessage() method. This can be used on any implementation of Message to get the message as a string. As such, the most logical thing to do seems to be to get the current message, as String, using getFormattedMessage() and then create a new SimpleMessage with that String.

Enough chat. Here’s the code:

package uk.me.lavin.phil.logging;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.appender.rewrite.RewritePolicy;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.core.impl.Log4jLogEvent;
import org.apache.logging.log4j.message.SimpleMessage;

@Plugin(name = "MessageSanitiserRewritePolicy", category = "Core", elementType = "rewritePolicy", printObject = true)
public final class MessageSanitiserRewritePolicy implements RewritePolicy {

	@Override
	public LogEvent rewrite(final LogEvent event) {
		Log4jLogEvent.Builder builder = new Log4jLogEvent.Builder();

		builder.setContextStack(event.getContextStack());
		builder.setInstant(event.getInstant());
		builder.setLevel(event.getLevel());
		builder.setLoggerFqcn(event.getLoggerFqcn());
		builder.setLoggerName(event.getLoggerName());
		builder.setMarker(event.getMarker());
		builder.setThrown(event.getThrown());
		builder.setNanoTime(event.getNanoTime());
		builder.setSource(event.getSource());
		builder.setThreadId(event.getThreadId());
		builder.setThreadName(event.getThreadName());
		builder.setThreadPriority(event.getThreadPriority());
		builder.setThrown(event.getThrown());
		builder.setThrownProxy(event.getThrownProxy());
		builder.setTimeMillis(event.getTimeMillis());

		builder.setMessage(
			new SimpleMessage(
				this.sanitise( event.getMessage().getFormattedMessage() )
			)
		);

		return builder.build();
	}

	@PluginFactory
	public static MessageSanitiserRewritePolicy createPolicy() {
		return new MessageSanitiserRewritePolicy();
	}

	// Sanitises log messages
	protected String sanitise(String message) {
		return "FOOOO " + message;
	}

}

As you can see, we have created a sanitise() method which takes and returns a String. This is where you can modify the log message. In this simple case, we’re simply prepending “FOOOO” to it.

The Config Bit

Now you need to modify your log4j config to add a Rewrite Appender. This will be your first Appender and will subsequently call your existing Console appender.

You also need to add a packages attribute to your Configuration element. This tells log4j where to look for your Plugin. For the above code, this is uk.me.lavin.phil.logging.

Here’s the config:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration shutdownHook="disable" packages="uk.me.lavin.phil.logging">
  <Appenders>
    <!-- Output JSON to console -->
    <Console name="Console" target="SYSTEM_OUT">
      <JsonLayout complete="false" compact="true" eventEol="true">
        <KeyValuePair key="traceId" value="$${ctx:correlationId:-}" />
      </JsonLayout>
    </Console>
    <!-- Rewrite Appender -->
    <Rewrite name="Rewrite">
      <!-- Call MessageSanitiserRewritePolicy class -->
      <MessageSanitiserRewritePolicy />
      <!-- Pass to the Console appender -->
      <AppenderRef ref="Console" />
    </Rewrite>
  </Appenders>
  <Loggers>
    <Root level="info">
      <!-- Call Rewrite Appender first -->
      <AppenderRef ref="Rewrite"/>
    </Root>
  </Loggers>
</Configuration>

The Results

When you run your app, you’ll see logs that look like this. Message with have FOOOO prepended to it:

{"instant":{"epochSecond":1694157823,"nanoOfSecond":322000000},"thread":"main","level":"INFO","loggerName":"org.apache.coyote.http11.Http11NioProtocol","message":"FOOOO Initializing ProtocolHandler [\"http-nio-8080\"]","endOfBatch":false,"loggerFqcn":"java.util.logging.Logger","threadId":1,"threadPriority":5}
{"instant":{"epochSecond":1694157823,"nanoOfSecond":326000000},"thread":"main","level":"INFO","loggerName":"org.apache.catalina.core.StandardService","message":"FOOOO Starting service [Tomcat]","endOfBatch":false,"loggerFqcn":"java.util.logging.Logger","threadId":1,"threadPriority":5}
{"instant":{"epochSecond":1694157823,"nanoOfSecond":327000000},"thread":"main","level":"INFO","loggerName":"org.apache.catalina.core.StandardEngine","message":"FOOOO Starting Servlet engine: [Apache Tomcat/10.1.12]","endOfBatch":false,"loggerFqcn":"java.util.logging.Logger","threadId":1,"threadPriority":5}

Now you have it working, you can edit the sanitise() method to do as you require.