Support #414
Updated by Daniel Curtis about 9 years ago
{{>toc}}
This is a guide on setting up an ElasticSearch, Logstash, Kibana stack with Nginx on FreeBSD 9.3-RELEASE.
h1. Prepare the system
* Update the system
<pre>
pkg update && pkg upgrade
portsnap fetch extract
</pre>
* Install portmaster:
<pre>
cd /usr/ports/ports-mgmt/portmaster
make install clean
pkg2ng
</pre>
---
h1. Installing ElasticSearch
* Install elasticsearch:
<pre>
portmaster textproc/elasticsearch
</pre>
* Start and enable ElasticSearch at boot
<pre>
echo 'elasticsearch_enable="YES"' >> /etc/rc.conf
service elasticsearch start
</pre>
h2. Securing Elasticsearch
* Up to version 1.2, Elasticsearch's dynamic scripting capability was enabled by default. Since this tutorial sets up the Kibana dashboard to be accessed from the public Internet, let's disable dynamic scripting by appending the following line at the end of the ElasticSearch configuration file:
<pre>
echo "script.disable_dynamic: true" >> /usr/local/etc/elasticsearch/elasticsearch.yml
</pre>
* Also enable cross origin access:
<pre>
echo 'http.cors.allow-origin: "/.*/"' >> /usr/local/etc/elasticsearch/elasticsearch.yml
echo 'http.cors.enabled: true' >> /usr/local/etc/elasticsearch/elasticsearch.yml
</pre>
* Restart Elasticsearch:
<pre>
service elasticsearch restart
</pre>
---
h1. Install Logstash
* Install logstash:
<pre>
portmaster sysutils/logstash
</pre>
* Create a basic configuration:
<pre>
vi /usr/local/etc/logstash/logstash.conf
</pre>
#* Then modify the following:
<pre>
output {
# Emit events to stdout for easy debugging of what is going through
# logstash.
#stdout { debug => "true" }
# This will use elasticsearch to store your logs.
# The 'embedded' option will cause logstash to run the elasticsearch
# server in the same process, so you don't have to worry about
# how to download, configure, or run elasticsearch!
elasticsearch {
embedded => false
#embedded_http_port => 9200
cluster => elasticsearch
host => localhost log.example.com
port => 9200
}
</pre>
* Start and enable logstash at boot:
<pre>
echo 'logstash_enable="YES"' >> /etc/rc.conf
service logstash start
</pre>
h1. Install Kibana
* Install kibana:
<pre>
portmaster textproc/kibana
</pre>
h2. Configuring Kibana
Since Kibana will use the default port 80 9200 to talk to Elasticsearch as opposed to the default port 9200, Elasticsearch, Kibana's config.js must will not need to be updated.
* Open Kibana configuration file and look for the following line:
<pre>
vi /usr/local/www/kibana/config.js
</pre>
#* And change the elasticsearch: "http://"+window.location.hostname+":9200", parameter to the following:
<pre>
elasticsearch: "http://"+window.location.hostname+":80",
</pre>
h1. Installing Nginx
We will use Nginx as a proxy server to allow access to the dashboard from the Public Internet (with basic authentication).
* Install Nginx:
<pre>
portmaster www/nginx security/py-htpasswd
</pre>
* Start and enable nginx at boot
<pre>
echo 'nginx_enable="YES"' >> /etc/rc.conf
service nginx start
</pre>
* Edit the nginx configuration file and change the primary server block as follows:
<pre>
vi /usr/local/etc/nginx/nginx.conf
</pre>
#* And add the following:
<pre>
# Nginx proxy for Elasticsearch + Kibana
#
server {
listen 80;
server_name localhost;
access_log /var/log/nginx-logstash.log;
location / {
root /usr/local/www/kibana;
index index.html index.htm;
}
location ~ ^/_aliases$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
}
location ~ ^/.*/_aliases$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
}
location ~ ^/_nodes$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
}
location ~ ^/.*/_search$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
}
location ~ ^/.*/_mapping {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
}
# Password protected end points
location ~ ^/kibana-int/dashboard/.*$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
limit_except GET {
proxy_pass http://127.0.0.1:9200;
auth_basic "Restricted";
auth_basic_user_file /usr/local/etc/nginx/log.example.com.htpasswd;
}
}
location ~ ^/kibana-int/temp.*$ {
proxy_pass http://127.0.0.1:9200;
proxy_read_timeout 90;
limit_except GET {
proxy_pass http://127.0.0.1:9200;
auth_basic "Restricted";
auth_basic_user_file /usr/local/etc/nginx/log.example.com.htpasswd;
}
}
</pre>
* And generate a htpasswd file:
<pre>
python2.7 /usr/local/bin/htpasswd.py -c -b /usr/local/etc/nginx/log.example.com.htpasswd username SuperSecretPassword
</pre>
* Finally, restart nginx as follows:
<pre>
service nginx restart
</pre>