Java Web App Logging in Tomcat

When I get a call from operations or QA describing some strange problem that occurred out of the blue, usually the first thing I do is ask them to do is to capture the logs (or get them myself, if I have access).  Logging provides much needed insight into faceless server processes and is an essential diagnostic tool for both development and operations.  This article discusses the high level options and basic configuration for logging in a web application in the Tomcat web server.

There are two major Java logging frameworks:

• Apache Log4j 
• java.util.logging

These logging frameworks are very similar in design, but share no interfaces.  Both have Logger objects and LogManager objects.  java.util.logging has handler objects to physically put log entries somewhere such as a file whereas log4j has appenders for the same purpose.  java.util.logging has formatter objects for formatting log entries whereas log4j has layout objects. 

Quick Setup with java.util.logging

Tomcat is configured out of the box to use java.util.logging with a custom LogManager called JULI.  Basically it is similar to, but slightly more configurable then the stock java.util.logging LogManager.  It is so easy to get logging out of your web app with this setup, I’ll just go ahead and show the full details:

1. Edit %CATALINA_BASE%/conf/logging.properties to create a logger, for example:
   com.anexinet.sample = INFO
   com.anexinet.sample.handlers = 1catalina.org.apache.juli.FileHandler
2. Bounce Tomcat to pick up the changes to logging.properties.
3. Use the java.util.logging.Logger class in your code to get a java.util.logging.Logger object and log away:
        Logger log = Logger.getLogger("com.anexinet.sample");    
        log.info(“Hello, world”);

Now you should see your logging written to %CATALINA_BASE%/logs/catalina-YYYY-MM-dd.log.  It’s also easy to define your own handlers following the examples in Tomcat’s default logging.properties file, if for example you want your app’s logging to go in a separate file instead of the catalina log file.  But note that the name of the handler must start with a number as required by JULI (unlike the plain java.util.logging LogManager).

Getting to the Big Leagues with Log4j

You will probably want to consider using log4j (version 2).  java.util.logging is fine to get something simple working with ease, but log4j is the real deal.  Among other things, it provides a serious set of appenders that do things like compress old log files, log to a relational or noSQL database, log to a JMS topic, or send out log events via email.

To get a log4j setup going, follow these general steps:

1. Create a log4j2.xml file (or alternatively log4j2.json if you prefer to use JSON) somewhere on your classpath.  The possibilities here are quite limitless.  See here for some examples. 
2. Use the org.apache.logging.logj.LogManager to get a org.apache.logging.logj.Logger object and log away:
        Logger log = LogManager.getLogger(“com.anexinet.sample”);
        log.info(“Hello, world”)

You'll need to decide where you will keep the logging configuration.  The two fundamental options are:

1. In the WAR file itself
2. In the environment (for example %CATALINA_BASE%/lib or a location specified by a shared.loader property to Tomcat)

In either case, if the file is on the classpath, log4j will find it.

The right answer, of course, depends on the situation at hand. I prefer minimizing the amount of data that constitutes an environment so that setting up an environment is easy (and easy to automate).  On the other hand, there is a decent chance logging configuration will not be environment-independent especially with advanced configuration containing host names or database connection strings.  You certainly do not want environment-specific data in your WAR file.

With a simple configuration that involves only console and file system logging, the only environment-like information needed is the path to the directory where the log files will be written.  Fortunately, with Tomcat you can simply place a relative path to the logging location (such as logs/mylog.txt) and the log file will magically be written to %CATALINA_BASE%/logs.  So now your environment set up will not involve also setting up logging.  So for this type of configuration, you can put the logging configuration in the WAR file itself. 

But what do you do if you need to override logging configuration in a particular environment?  No problem.  In a pinch, you can override the log4j2.xml in your WAR file on your class path by placing a log4j2-test.xml on your classpath (either %CATALINA_BASE%/lib or some place you've defined with a Tomcat shared.loader property.  This works because log4j2 first looks for log4j2-test.xml on your classpath before looking for log4j2.xml.

Picking up this change requires reloading the web application, but the change will persist across all restarts until you take it out.  You can also configure this file for automatic reconfiguration and it will be periodically checked for updates (which, by the way, you cannot do with java.util.logging).

If you want to change a logging level without restarting your web application, you can do so by configuring JMX extensions in Tomcat as described here.  Once configured, the jconsole GUI let's you see the logging level of every defined log4j logger and change it accordingly.  You can also get a wealth of other diagnostic information from Tomcat in the jconsole GUI, so it is well worth considering.  The only disadvantage of configuring JMX in Tomcat is that you will have an additional port open on your web server and so will need to secure it with a password and SSL encryption keys.

Logging in Automated Tests

If you have a log4j2.xml file in your WAR file, you will likely want a different one to be used for automated unit and integration tests that are part of your build.  You will probably want to stick to a console appender but not necessarily limit it to INFO logging and higher.  Do this by creating a log4j2-test.xml in your test resources which should be on your classpath for tests, but not even packaged with the WAR file.  This file will be picked up before the production log4j2.xml file when running tests.

Where To Go From Here

Tomcat itself can be configured to use log4j, without much difficulty.  You would likely do that as part of an effort to centralize all your logging from various applications and servers for quick and easy access.

You may also want to take a look at Apache Chainsaw which along with an XML or socket appender lets you graphically view and navigate through log data.