Moving log location for Tomcat 9 on Ubuntu 18.04
On a fresh instance in AWS of Ubuntu 18.04, I have installed tomcat9 through apt. I'm replacing a 14.04 Ubuntu install where i was running tomcat7. Due to the nature of my application, I have a very large disk mounted under /data/ebs1. On this disk I have set-up a folder to house a very large set of tomcat logs.
/data/ebs1/tomcat-logsGetting logging moved here is giving me a headache. It seems that tomcat9 is set up to log to syslog. So bear with me as I try to go down the rabbit hole of configurations here.
Here's my /etc/systemd/system/tomcat9.service
#
# Systemd unit file for Apache Tomcat
#
[Unit]
Description=Apache Tomcat 9 Web Application Server
Documentation=
After=network.target
[Service]
# Configuration
Environment="CATALINA_HOME=/usr/share/tomcat9"
Environment="CATALINA_BASE=/var/lib/tomcat9"
Environment="CATALINA_TMPDIR=/tmp"
Environment="JAVA_OPTS=-Djava.awt.headless=true"
EnvironmentFile=-/etc/default/tomcat9
# Lifecycle
Type=simple
ExecStartPre=+/usr/libexec/tomcat9/tomcat-update-policy.sh
ExecStart=/bin/sh /usr/libexec/tomcat9/tomcat-start.sh
SuccessExitStatus=143
Restart=on-abort
# Logging
SyslogIdentifier=tomcat9
# Security
User=tomcat
Group=tomcat
PrivateTmp=yes
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
LogsDirectory=tomcat9
LogsDirectoryMode=750
CacheDirectory=tomcat9
CacheDirectoryMode=750
ProtectSystem=strict
ReadWritePaths=/etc/tomcat9/Catalina/
ReadWritePaths=/var/lib/tomcat9/webapps/
ReadWritePaths=/data/ebs1/
[Install]
WantedBy=multi-user.targetThe only thing I changed here was i added data/ebs1/ to the list of ReadWritePaths.
This file outlines a SyslogIdentifier of tomcat9.
And there's a custom handler set up for tomcat9 in /etc/rsyslog.d/tomcat9.conf
# Send Tomcat messages to catalina.out when using systemd
$template TomcatFormat,"[%timegenerated:::date-year%-%timegenerated:::date-month%-%timegenerated:::date-day% %timegenerated:::date-hour%:%timegenerated:::date-minute%:%timegenerated:::date-second%] [%syslogseverity-text%]%msg%\n"
:programname, startswith, "tomcat9" { /data/ebs1/tomcat-logs/catalina.out;TomcatFormat stop
}I changed this from /var/log/tomcat9 to /data/ebs1/tomcat-logs.
So now when I go to look inside /data/ebs1/tomcat-logs, I see catalina datestamped log files like catalina.2020-03-25.log But no catalina.out. And the catalina datestamped log file doesn't contain everything. So some of the logging is going there, some of the logging is still going to syslog. I was hoping to get ALL logging to go to my catalina.out. I can't even figure out who is responsible for creating the datestamped catalina files. Syslog? Tomcat? My worries is that if my logs are still going to /var/log/syslog i'm going to fill up my disk quickly.
Thank you!
1 Answer
Ok so turns out the configs I posted were in fact, correct. But I had a few things going wrong for me.journalctl -u rsyslog revealed that rsyslog didn't have write permissions to the new catalina.out. adm was the group on the folder (and luckily syslog was in that group too). Adding group-write permissions solved that.
So I was still getting output in /var/log/syslog even though /etc/rsyslog.d/tomcat9.conf had a "stop" in it. What I didn't realize is that /etc/rsyslog.conf loads config files from /etc/rsyslog.d/ in name order. The contents of /etc/rsyslog.d/ were like this:
20-ufw.conf
21-cloudinit.conf
50-default.conf
tomcat9.confI just renamed the file to come before "default" and everything was happy. No more logs in /var/log/syslog!
20-ufw.conf
21-cloudinit.conf
30-tomcat9.conf
50-default.confAnyway. High-five-self.
1
