Skip to content

Instantly share code, notes, and snippets.

@viglesiasce

viglesiasce/logstash.conf

Forked from anonymous/logstash.conf
Last active September 22, 2015 21:59
Show Gist options
  • Save viglesiasce/33d482ebd119b8c2dd19 to your computer and use it in GitHub Desktop.
Save viglesiasce/33d482ebd119b8c2dd19 to your computer and use it in GitHub Desktop.
Download ZIP
Eucalyptus Logstash Filter
Raw
logstash.conf
input {
tcp { port => 3333 }
}
filter {
grok {
match => { "message" => ["%{TIMESTAMP_ISO8601:euca_timestamp}%{SPACE}%{LOGLEVEL:euca_log_level}%{SPACE}\|%{GREEDYDATA:euca_message}",
"%{TIMESTAMP_ISO8601:euca_timestamp}%{SPACE}%{LOGLEVEL:euca_log_level}%{SPACE}%{BASE10NUM:euca_pid}%{SPACE}%{WORD:euca_function}%{SPACE}\|%{GREEDYDATA:euca_message}",
"%{TIMESTAMP_ISO8601:euca_timestamp}%{SPACE}%{LOGLEVEL:euca_log_level}%{SPACE}%{GREEDYDATA:euca_message}"] }
}
date {
match => ['euca_timestamp', "YYYY-MM-dd HH:mm:ss", "EEE MMM dd HH:mm:ss YYYY"]
}
}
output { elasticsearch_http { host => "localhost" port => "9200" } }
@viglesiasce
Copy link
Author

In order to use this, first ensure you have Java 1.7 installed and download ELK:

java -version
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.2.zip
unzip elasticsearch-1.7.2.zip
wget https://download.elastic.co/logstash/logstash/logstash-1.5.4.zip
unzip logstash-1.5.4.zip
wget https://download.elastic.co/kibana/kibana/kibana-4.1.2-darwin-x64.zip
unzip kibana-4.1.2-darwin-x64.zip

Start the ELK stack:

pushd elasticsearch-1.7.2; ./bin/elasticsearch & popd
pushd logstash-1.5.4; ./bin/logstash -f logstash.conf & popd # logstash.conf should contain whats in the gist
pushd kibana-4.1.2-darwin-x64; ./bin/kibana & popd

Load up your eucalyptus log files into Logstash. In this case I'm pulling them from my unzipped sosreport archives:

cat sosreport-*/var/log/eucalyptus/* | nc localhost 3333

Visit http://localhost:5601 to browse/search/splice your logs via Kibana.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment