Benutzer-Werkzeuge

Webseiten-Werkzeuge


freebsd:filebeats_logstash_elasticsearch_kibana

Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

Link zu dieser Vergleichsansicht

freebsd:filebeats_logstash_elasticsearch_kibana [2016/08/06 13:34] (aktuell)
Zeile 1: Zeile 1:
 +====== Filebeat, Logstash, Elasticsearch,​ Kibana, Nginx ======
 +We will use Filebeat, Logstash, Elasticsearch and Kibana to visualize Nginx access logfiles.
 +
 +===== Create the x509 Certificate =====
 +As I have all running on one server I use as the SSL common name localhost.
 +<note tip>If you would like to deliver logfiles to another IP address use here the correct FQDN.</​note>​
 +<code console>
 +mkdir -p /​usr/​local/​etc/​pki/​tls/​certs
 +mkdir -p /​usr/​local/​etc/​pki/​tls/​private
 +cd /​usr/​local/​etc/​pki/​tls
 +openssl req -subj '/​CN=localhost/'​ -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/​beat.key -out certs/​beat-cacert.crt
 +</​code>​
 +The beat-cacert.crt will be copied to all computer you want to send logs from.
 +
 +===== Install and configure Elasticsearch =====
 +<​code>​
 +pkg install elasticsearch2
 +</​code>​
 +We only change one line in the config file to make sure only localhost can connect to elasticsearch:​
 +<code yaml /​usr/​local/​etc/​elasticsearch/​elasticsearch.yml
 +network.host:​ localhost
 +</​code>​
 +Enable it with:
 +<code console>
 +sysrc elasticsearch_enable="​YES"​
 +</​code>​
 +Start it with:
 +<code console>
 +service elasticsearch start
 +</​code>​
 +
 +===== Install and configure Filebeat =====
 +<code console>
 +pkg install filebeat
 +</​code>​
 +<note important>​Use only spaces and no tabs in the configuration file!</​note>​
 +<code yaml /​usr/​local/​etc/​filebeat.yml>​
 +filebeat:
 +  prospectors:​
 +    -
 +      paths:
 +        - /​var/​log/​auth.log
 +        - /​var/​log/​messages
 +      input_type: log
 +      document_type:​ syslog
 +    -
 +      document_type:​ web_access_nginx
 +      input_type: log
 +      paths:
 +        - /​usr/​home/​http/​poudriere/​logs/​access.log
 +
 +output:
 +  logstash:
 +    hosts: ["​localhost:​5044"​]
 +    bulk_max_size:​ 1024
 +    tls:
 +      certificate_authorities:​ ["/​usr/​local/​etc/​pki/​tls/​certs/​beat-cacert.crt"​]
 +
 +shipper:
 +
 +logging:
 +    rotateeverybytes:​ 10485760 # = 10MB
 +</​code>​
 +Verify the format of the file with:
 +<code console>
 +filebeat -configtest
 +</​code>​
 +Enable Filebeat with:
 +<code console>
 +sysrc filebeat_enable="​YES"​
 +</​code>​
 +And start it with:
 +<code console>
 +service filebeat start
 +</​code>​
 +It should now directly start to deliver logfile information defined in section prospectors.
 +You can test it with:
 +<code console>
 +curl -XGET '​http://​localhost:​9200/​filebeat-*/​_search?​pretty'​
 +</​code>​
 +If you see something like this everything is fine:
 +<code json>
 +{
 +  "​took"​ : 1,
 +  "​timed_out"​ : false,
 +  "​_shards"​ : {
 +    "​total"​ : 20,
 +    "​successful"​ : 20,
 +    "​failed"​ : 0
 +  },
 +  "​hits"​ : {
 +    "​total"​ : 18157,
 +    "​max_score"​ : 1.0,
 +    "​hits"​ : [ {
 +      "​_index"​ : "​filebeat-2016.08.03",​
 +      "​_type"​ : "​syslog",​
 +      "​_id"​ : "​AVZcJLZL5UZfyQchYySN",​
 +...
 +</​code>​
 +
  
freebsd/filebeats_logstash_elasticsearch_kibana.txt · Zuletzt geändert: 2016/08/06 13:34 (Externe Bearbeitung)