filebeats 收集Nginx日志
1. 介绍¶
通过stdin收集日志,通过console输出结果
2. 下载安装¶
cd /usr/local/src/elk/ sudo wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.6.16-linux-x86_64.tar.gz tar xf filebeat-5.6.16-linux-x86_64.tar.gz -C /opt/ ln -s /opt/filebeat-5.6.16-linux-x86_64 /opt/filebeat cd /opt/filebeat mkdir conf cd conf cat>>nginx.yml<<EOF filebeat.prospectors: - input_type: stdin output.console: pretty: true EOF head -n 2 /var/log/nginx.log |/opt/filebeat/filebeat -e -c /opt/filebeat/conf/nginx.yml
可以使用迅雷下载,导入到系统中安装。
3. 详细过程¶
root@master:/home/cmz# cd /usr/local/src/elk/ root@master:/usr/local/src/elk# sudo wget https://artifacts.elastic.co/downloads/beats/filebeat/fileb --2019-10-09 16:08:47-- https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.6.16-linux-x Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.110.222, 2a04:4e42:1a::734 Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.110.222|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8838693 (8.4M) [application/x-gzip] Saving to: ‘filebeat-5.6.16-linux-x86_64.tar.gz’ filebeat-5.6.16-linux-x86_64.tar.gz 100%[================================================= 2019-10-09 16:10:18 (97.1 KB/s) - ‘filebeat-5.6.16-linux-x86_64.tar.gz’ saved [8838693/8838693] root@master:/usr/local/src/elk# tar xf filebeat-5.6.16-linux-x86_64.tar.gz -C /opt/ root@master:/var/log# ln -s /opt/filebeat-5.6.16-linux-x86_64 /opt/filebeat root@master:/opt/filebeat# mkdir conf root@master:/opt/filebeat# cd conf/ root@master:/opt/filebeat/conf# egrep -v '#|^$' nginx.yml filebeat.prospectors: - input_type: stdin output.console: pretty: true root@master:/opt/filebeat/conf# head -n 2 /var/log/nginx.log |/opt/filebeat/filebeat -e -c /opt/filebe at/conf/nginx.yml 2019/10/09 08:27:41.580046 beat.go:297: INFO Home path: [/opt/filebeat] Config path: [/opt/filebeat] D ata path: [/opt/filebeat/data] Logs path: [/opt/filebeat/logs]2019/10/09 08:27:41.580065 beat.go:192: INFO Setup Beat: filebeat; Version: 5.6.16 2019/10/09 08:27:41.580095 outputs.go:108: INFO Activated console as output plugin. 2019/10/09 08:27:41.580136 metrics.go:23: INFO Metrics logging every 30s 2019/10/09 08:27:41.580149 publish.go:300: INFO Publisher name: master 2019/10/09 08:27:41.580260 async.go:63: INFO Flush Interval set to: 1s 2019/10/09 08:27:41.580268 async.go:64: INFO Max Bulk Size set to: 2048 2019/10/09 08:27:41.580413 beat.go:233: INFO filebeat start running. 2019/10/09 08:27:41.580435 registrar.go:68: INFO No registry file found under: /opt/filebeat/data/regi stry. Creating a new registry file.2019/10/09 08:27:41.627096 registrar.go:106: INFO Loading registrar data from /opt/filebeat/data/regis try2019/10/09 08:27:41.627131 registrar.go:123: INFO States Loaded from registrar: 0 2019/10/09 08:27:41.627163 crawler.go:38: INFO Loading Prospectors: 1 2019/10/09 08:27:41.627224 registrar.go:236: INFO Starting Registrar 2019/10/09 08:27:41.627251 sync.go:41: INFO Start sending events to output 2019/10/09 08:27:41.627326 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s 2019/10/09 08:27:41.627325 prospector.go:124: INFO Starting prospector of type: stdin; id: 14799454217 977376761 2019/10/09 08:27:41.627345 crawler.go:58: INFO Loading and starting Prospectors completed. Enabled pro spectors: 12019/10/09 08:27:41.627405 log.go:91: INFO Harvester started for file: - 2019/10/09 08:27:41.627440 log.go:114: INFO End of file reached: . Closing because close_eof is enable d.{ "@timestamp": "2019-10-09T08:27:41.627Z", "beat": { "hostname": "master", "name": "master", "version": "5.6.16" }, "input_type": "stdin", "message": "80.82.70.187 - - [09/Oct/2019:08:22:42 +0800] \"GET http://www.baidu.com/cache/global/im g/gs.gif HTTP/1.1\" 401 188 \"-\" \"Mozilla\"", "offset": 0, "source": "", "type": "log" } { "@timestamp": "2019-10-09T08:27:41.627Z", "beat": { "hostname": "master", "name": "master", "version": "5.6.16" }, "input_type": "stdin", "message": "80.82.70.187 - - [09/Oct/2019:08:22:42 +0800] \"GET http://www.baidu.com/cache/global/im g/gs.gif HTTP/1.1\" 401 188 \"-\" \"Mozilla\"", "offset": 0, "source": "", "type": "log" } 2019/10/09 08:28:11.580663 metrics.go:39: INFO Non-zero metrics in the last 30s: filebeat.harvester.cl osed=1 filebeat.harvester.open_files=-1 filebeat.harvester.started=1 libbeat.publisher.published_events=2 publish.events=3 registrar.states.cleanup=1 registrar.states.update=1 registrar.writes=2