Setup your central rsyslog server

BlogPower

Rsyslog can be configured to receive logs from the network and can then store them in a structured way,
that’s what I suggest you see in this article. Here I am using LINUX DEBIAN machines.

Central server configuration

rsyslog.conf :

– Here we use the rsyslog modules imudp and imtcp which allow the server to listen for rsyslog client connections.
– We will use a template « remote-logs » to create a log file name and its path dynamically.
– We will use « remote-logs » template only if host IP is not the local server ( 127.0.0.1 ).

###################################
# RSYSLOG SERVER CONFIGURATION 
###################################
# /etc/rsyslog.conf configuration file for rsyslog
#
# For more information install rsyslog-doc and see
# /usr/share/doc/rsyslog-doc/html/configuration/index.html

#################
#### MODULES ####
#################

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
#module(load="immark")  # provides --MARK-- message capability

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")

# Remote logs directory : (/var/log/remoteservers/[CLIENT]/)
$template remote-logs,"/var/log/remoteservers/%HOSTNAME%/%PROGRAMNAME%.log"

if not($fromhost-ip == '127.0.0.1') then {
*.* ?remote-logs
& ~
}

###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

#
# Set the default permissions for all log files.
#
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf


###############
#### RULES ####
###############

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*			/var/log/auth.log
*.*;auth,authpriv.none		-/var/log/syslog
#cron.*				/var/log/cron.log
daemon.*			-/var/log/daemon.log
kern.*				-/var/log/kern.log
lpr.*				-/var/log/lpr.log
mail.*				-/var/log/mail.log
user.*				-/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info			-/var/log/mail.info
mail.warn			-/var/log/mail.warn
mail.err			/var/log/mail.err

#
# Some "catch-all" log files.
#
*.=debug;\
	auth,authpriv.none;\
	mail.none		-/var/log/debug
*.=info;*.=notice;*.=warn;\
	auth,authpriv.none;\
	cron,daemon.none;\
	mail.none		-/var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg				:omusrmsg:*

Client server configuration

rsyslog.conf :

###################################
# RSYSLOG CLIENT CONFIGURATION 
###################################
# /etc/rsyslog.conf configuration file for rsyslog
#
# For more information install rsyslog-doc and see
# /usr/share/doc/rsyslog-doc/html/configuration/index.html

#################
#### MODULES ####
#################

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
#module(load="immark")  # provides --MARK-- message capability

###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

#
# Set the default permissions for all log files.
#
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

#
# Where to place spool and state files
#
#$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf

###############
#### RULES ####
###############

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*			/var/log/auth.log
*.*;auth,authpriv.none		-/var/log/syslog
#cron.*				/var/log/cron.log
daemon.*			-/var/log/daemon.log
kern.*				-/var/log/kern.log
lpr.*				-/var/log/lpr.log
mail.*				-/var/log/mail.log
user.*				-/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info			-/var/log/mail.info
mail.warn			-/var/log/mail.warn
mail.err			/var/log/mail.err

#
# Some "catch-all" log files.
#
*.=debug;\
	auth,authpriv.none;\
	mail.none		-/var/log/debug
*.=info;*.=notice;*.=warn;\
	auth,authpriv.none;\
	cron,daemon.none;\
	mail.none		-/var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg				:omusrmsg:*

#CLI configuration : 
#Enable sending system logs over UDP to rsyslog server
*.* @[rsyslog_server]:514
#Enable sending system logs over TCP to rsyslog server
*.* @@[rsyslog_server]:514

It is possible to create a dedicated configuration, for example to change the spool folder, which allows not to modify the main configuration file rsyslog.conf,
To do this we add this file:  /etc/rsyslog.d/01-client.conf

– The client sends its logs to the IP [rsyslog_server].
– The spool folder used (apache log for example) is: /var/log/rsyslogspools
– A quota of 1G is set on it.

#/etc/rsyslog.d/01-client.conf
$WorkDirectory /var/log/rsyslogspools
$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1

In the case of an APACHE service, you will have to tell rsyslog where to fetch the data and then send it to the remote rsyslog server : [rsyslog_server]

#/etc/rsyslog.d/02-apache-defaultlog.conf.j2

$ModLoad imfile

# Default Apache Error Log
$InputFileName /var/log/apache2/error.log
$InputFileTag apache-error-default:
$InputFileStateFile apache-error-default
$InputRunFileMonitor

# Default Apache Access Log
$InputFileName /var/log/apache2/access.log
$InputFileTag apache-access-default:
$InputFileStateFile apache-access-default
$InputRunFileMonitor
$InputFilePollInterval 30

if $programname == "apache-error-default" then @@[rsyslog_server]:514
if $programname == "apache-error-default" then ~

if $programname == "apache-access-default" then @@[rsyslog_server]:514
if $programname == "apache-access-default" then ~

And There you go !