Configure Logrotate For Rails With Puppet

I saw a scout rails plugin failing on our production systems because the main rails log file got to big after a few weeks of running (>6GB). The recommended workaround for this is to rotate your logfiles. This means in a specified interval, the current logfile is copied somewhere else and a new empty one is generated. With this you can keep your logs small, save disk space and make all your little applications happy. With logrotate you can do many fancy things like compressing archived logfiles or rotate as soon as the logfile hits a certain size. Its a standard linux tool and you should really checkout the manpage

man logrotate

or read some decent article about. Since all our server configuration is managed through puppet, I had to find a module to do it. I used this one from puppet forge. It took me some time to come up with a working manifest, but here it is:

logrotate::rule { 'rails':
  path          => '/home/deployer/shared/log/production.log',
  rotate_every  => 'week',
  su            => true,
  su_owner      => 'deployer',
  su_group      => 'deployer',
  create        => true,
  create_mode   => '0755',
  create_owner  => 'deployer',
  create_group  => 'deployer',
  missingok     => true,
  rotate        => 10,
  compress      => true,
  delaycompress => true,
  copytruncate  => true,
}

The problem was that all the logfiles are stored as the deployer user. So we need to tell logrotate to act as this user to avoid this error:

error: "/home/deployer/shared/logs/production.log" has insecure permissions. It
must be owned and be writable by root only to avoid security problems. Set the
"su" directive in the config file to tell logrotate which user/group should be
used for rotation.

The current version of the module on puppetforge doesn't support this yet. Luckily the su, su_owner and su_group options are added on github already, so I used the module directly from there with adding it like this to the Puppetfile:

mod "rodjek/logrotate", :git => "git://github.com/rodjek/puppet-logrotate.git"

After applying your config, you can run logrotate manually to see if everything works with this command:

sudo logrotate -f /etc/logrotate.d/rails

Yay!

Protip #1:

To avoid having trouble if your cute little start-up idea goes crazy, you should add kind of every logfile which can get big really fast to logrotate.

Protip #2:

Use some external logfile storage. You can do it on your own with logstash or using an external service like logentries (like we do). With that you can delete you logs after they rotated but still have access to them. Backupping to S3 or similar is also cool.