How to Monitor Log Files with Graylog v3.1 on Debian 10

 💡🏆🌎👉 👈🌎🏆💡 Download Zip https://t.co/gg2ykmGTNG


On the command line you can provide a path to the configuration file with the -c switch. The default configuration path on Linux systems is /etc/graylog/sidecar/sidecar.yml and C:\\Program Files\\Graylog\\sidecar\\sidecar.yml on Windows.


Additionally with the list_log_files option a directory listing is displayed in the Graylog web interface. This way an administrator can see which files are available for collecting. The list is periodically updated and files with write access are highlighted for easy identification. After enabling send_status or send_status + list_log_files go to the collector overview and click on one of them, a status page with the configured information will be displayed.


Check the log files in /var/log/graylog-sidecar for any errors. Understand that not only the Sidecar but also all backends, like filebeat, will be started as sidecar user after these changes. So all log files that the backend should observe also need to be readable by the sidecar user. Depending on the Linux distribution there is usually an administrator group which has access to most log files. By adding the sidecar user to that group you can grant access fairly easy. For example on Debian/Ubuntu systems this group is called adm (see System Groups in Debian Wiki or Security/Privileges - Monitor system logs in Ubuntu wiki).



1. Describe your incident:

I came across graylog in my attempts to setup a syslog server for my unraid server. When I searched google, I found a github distro for instructions to setup not just a graylog server but also do it with the purpose of logging pfsense which I also use. I decided to follow these instructions because I would then get both.





Docker is designed to be modular so my recommendation is to use our example Graylog docker compose file for running graylog. Using docker volumes and bind mounts its possible to add any files needed into graylog so the custom build step is unnecessary.



UPDATE

Its been about 24 hours since my last major change. I had set my /etc/default/graylog-server at half my available memory per the site instructions. Having avoided the issues I had been experiencing recently on an ElasticSearch-backed Graylog instance, before, with much less memory defined in the /etc/default/graylog-server, I backed it down to -Xms16g -Xmx16g on my current build. So far so good.


To test your configuration file, change to the directory where theFilebeat binary is installed, and run Filebeat in the foreground withthe following options specified: ./filebeat test config -e. Make sure yourconfig files are in the path expected by Filebeat (see Directory layout),or use the -c flag to specify the path to the config file.


A brief description of my set-up: I have 11 Docker containers in a Docker Compose single-server configuration. Some of these containers produce logs, which I write to (mostly individual) on-host volumes. This results in eight log files, which are shared with a Rsyslog container, again via Docker volumes. Finally this container forwards the logs to Papertrail, the cloud-based log aggregator app.


Some very helpful engineers who patrol the Rsyslog issues list have come up with a good-enough fix. It seems that, by default, state files are only written per-log when Rsyslog exits, which is why they do not appear straight away.


I will use the strategy of using a low value (e.g. 20) on slow log files, and higher ones (e.g. 100) on fast-moving files. I think I am happier with fairly frequent writes in this fashion, since it will guard against power failure or a container crash.


1 other thing: if you do not want to delete the contents you can compress the files by tarring or gzipping them. That will have you end up with files probably 10% of what they are now. That is

How To Monitor Log Files With Graylog V3.1 On Debian 10

Jeff Markley