We will use Filebeat, Logstash, Elasticsearch and Kibana to visualize Nginx access logfiles.
As I have all running on one server I use as the SSL common name localhost.
mkdir -p /usr/local/etc/pki/tls/certs mkdir -p /usr/local/etc/pki/tls/private cd /usr/local/etc/pki/tls openssl req -subj '/CN=localhost/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/beat.key -out certs/beat-cacert.crt
The beat-cacert.crt will be copied to all computer you want to send logs from.
pkg install elasticsearch2
We only change one line in the config file to make sure only localhost can connect to elasticsearch:
Enable it with:
sysrc elasticsearch_enable="YES"
Start it with:
service elasticsearch start
pkg install filebeat
filebeat: prospectors: - paths: - /var/log/auth.log - /var/log/messages input_type: log document_type: syslog - document_type: web_access_nginx input_type: log paths: - /usr/home/http/poudriere/logs/access.log output: logstash: hosts: ["localhost:5044"] bulk_max_size: 1024 tls: certificate_authorities: ["/usr/local/etc/pki/tls/certs/beat-cacert.crt"] shipper: logging: rotateeverybytes: 10485760 # = 10MB
Verify the format of the file with:
filebeat -configtest
Enable Filebeat with:
sysrc filebeat_enable="YES"
And start it with:
service filebeat start
It should now directly start to deliver logfile information defined in section prospectors. You can test it with:
curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty'
If you see something like this everything is fine:
{ "took" : 1, "timed_out" : false, "_shards" : { "total" : 20, "successful" : 20, "failed" : 0 }, "hits" : { "total" : 18157, "max_score" : 1.0, "hits" : [ { "_index" : "filebeat-2016.08.03", "_type" : "syslog", "_id" : "AVZcJLZL5UZfyQchYySN", ...