- 在本机安装logstash
- 使用rvm安装jruby.
- 进入logstash安装目录(如
/usr/share/logstash
), 运行sudo -E bin/logstash-plugin install --no-verify --version 6.4.0 logstash-output-amazon_es
安装logstash-output-amazon_es插件。如果logstash-plugin安装很慢的话,请安装 haveged 后尝试。见此issue, 此文章 - 在aws iam中创建一个用户,赋予编程访问,和AmazonESFullAccess权限,将此用户设为elasticsearch域的主用户,获取access_key和secret(如果已经配置了elasticsearch域的主用户,则用该用户的access_key和secret)
- 配置logstash pipeline, 如
input {
file {
path => "/var/log/nginx/access.log"
start_position => "beginning"
}
}
filter {
grok {
patterns_dir => "/etc/logstash/patterns"
match => { "message" => "%{NGINXACCESS}" }
}
}
output {
stdout {}
amazon_es {
hosts => ["$ES_host"]
ssl => true
region => "$ES_region"
aws_access_key_id => '$access_key_id'
aws_secret_access_key => '$secret_access_key'
index => "nginx-access-logs-%{+YYYY.MM.dd}"
}
}