When developing a web application, access logs may not seem like a priority in the early stages. However, they can provide valuable insights into how your application is used and offer opportunities to optimize performance and security. Some questions access logs can help answer include:

  • Who or what is making requests to your application?
  • Are resources that should be cached by browsers being repeatedly requested?
  • Which pages are loading slowly?
  • Are bots or scrapers consuming excessive server resources?
  • Which URLs are visited most frequently?
  • What are the common referrers?

What Is an Access Log?

An access log is a record of requests made to a system’s resources. Access logs are common in for applications running connected to network such as operating systems, proxies, FTP servers, load balancers, and web servers. In the case of a web application, an access log typically contains:

  • The IP address of the request source
  • Information about the client making the request (User-Agent)
  • Request protocol, method, and URI
  • Request processing time
  • Response status code and size
  • Headers, cookies, session data, and more

With this information, you can gain valuable insights into the behavior and performance of your application.

Configuring Access Logs

If you're using Spring Boot with an embedded Tomcat, you can easily configure access logs through the application.properties file.

Enabling Access Logs

To enable access logs, simply add the following line to your application.properties:

server.tomcat.accesslog.enabled=true

By default, this logs to the "temp" directory since we’re not running a standalone Tomcat installation. To store the logs in a more appropriate directory, you can define a base directory for Tomcat.

Defining Tomcat Base Directory

server.tomcat.basedir=tomcat

With this configuration, the logs will appear under tomcat/logs with filenames such as access_log.2024-10-01.log. The prefix (access_log) and suffix (log) are configurable, and the filename includes the date in a configurable format.

A sample log entry might look like this:

1.1.1.1 - - [01/Oct/2024:23:45:25 +0100] "GET /posts HTTP/1.1" 200 4602

Configuring the Log Pattern

By default, Tomcat uses the common log pattern, which can be set with the following property:

server.tomcat.accesslog.pattern=common

The common pattern corresponds to:

%h %l %u %t "%r" %s %b

Where:

  • %h: Remote host (or IP address)
  • %l: Remote logical username (always -)
  • %u: Remote user (if authenticated)
  • %t: Date and time (in Common Log Format)
  • %r: First line of the request (method and URI)
  • %s: HTTP status code of the response
  • %b: Bytes sent (excluding headers)

Another built-in pattern is combined, which adds the Referrer and User-Agent headers to the common pattern.

However, I prefer a more detailed log format that includes request/response processing times and thread name, which can provide valuable insights into performance and allow me to correlate access logs with application logs. Here's my custom pattern:

server.tomcat.accesslog.pattern="%t" %h "%{CF-Connecting-IP}i" "%r" %s %b %D %F %I "%{Referer}i" "%{User-Agent}i"

This pattern includes:

  • %D: Time taken to process the request (in microseconds)
  • %F: Time to first byte (in milliseconds)
  • %I: Thread name handling the request

Be mindful when adding details to your log pattern, as logging too much information can cause log files to grow rapidly, especially if bots or scrapers are generating a lot of requests.

Configuring Log File Name and Directory

You can further customize the log file name and the directory where logs are stored within the Tomcat base directory:

server.tomcat.accesslog.suffix=.txt
server.tomcat.accesslog.prefix=my_access_log
server.tomcat.accesslog.file-date-format=.dd-MM-yyyy
server.tomcat.accesslog.directory=mylogs

With this setup, log files will be stored in the tomcat/mylogs directory with filenames like my_access_log.01-10-2024.txt.

Log Retention

Another critical aspect of log management is retention. By default, Tomcat retains access logs for 90 days, but this can be adjusted using the following property:

server.tomcat.accesslog.max-days=10

This setting specifies how many days of logs to keep. If you don’t want Tomcat to manage log retention (e.g., you use a separate log rotation tool), you can set this value to -1.

Conclusion

Although access logging happens "behind the scenes" and doesn’t directly affect what users see, it’s a powerful tool for analyzing usage patterns and identifying potential performance issues. By customizing and maintaining access logs to fit your application’s specific needs, you can unlock valuable insights to optimize and protect your web application.