資訊專欄INFORMATION COLUMN
摘要：新增了一個，它是一個輕量級的日志收集處理工具，占用資源少，適合于在各個服務器上搜集日志后傳輸給，官方也推薦此工具。具體的結構如下主機系統部署服務安裝準備是需要的，建議安裝。先下載通用安裝包。
ELK簡介
ELK是三個開源軟件的縮寫，分別表示：Elasticsearch , Logstash, Kibana , 它們都是開源軟件。新增了一個FileBeat，它是一個輕量級的日志收集處理工具(Agent)，Filebeat占用資源少，適合于在各個服務器上搜集日志后傳輸給Logstash，官方也推薦此工具。
Elasticsearch是個開源分布式搜索引擎，提供搜集、分析、存儲數據三大功能。它的特點有：分布式，零配置，自動發現，索引自動分片，索引副本機制，restful風格接口，多數據源，自動搜索負載等。
Logstash 主要是用來日志的搜集、分析、過濾日志的工具，支持大量的數據獲取方式。一般工作方式為c/s架構，client端安裝在需要收集日志的主機上，server端負責將收到的各節點日志進行過濾、修改等操作在一并發往elasticsearch上去。
Kibana 也是一個開源和免費的工具，Kibana可以為 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面，可以幫助匯總、分析和搜索重要數據日志。
Filebeat隸屬于Beats。目前Beats包含四種工具：
Packetbeat（搜集網絡流量數據）
Topbeat（搜集系統、進程和文件系統級別的 CPU 和內存使用情況等數據）
Filebeat（搜集文件數據）
Winlogbeat（搜集 Windows 事件日志數據）
一般而言，ELK主要用在海量零散數據的匯總和信息提取分析上。在分布式系統的日志統計，大數據的數據分析，業務數據的快速檢索，服務器集群上每臺服務器的運行情況查詢監控等方面有很強大的功能。
拿ELK在分布式系統上的日志收集舉例。隨著微服務的流行，分布式的使用，以往日志文件寫在具體的服務器上的某一位置做法變得不符合需求，首先是服務器越來越多而且后端服務集群橫跨多個服務器導致日志越來越散，不論是開發，測試還是線上的日志定位越來越難，準確的找到有用的信息需要運維/開發不段的排查，這時ELK就派上用場了，它將服務集群里面的日志收集匯總并建立索引，當出現問題是定位問題就像Google這類搜素引擎一樣高效簡單。
安裝一般單臺機器就可以安裝了，我這里為了貼近實際使用，分為3個機器來部署一個入門的ELK。
具體的結構如下
| 主機 | IP | 系統 | 部署服務 |
|---|---|---|---|
| thinkvmc01 | 192.168.50.207 | CentOS7 | ElasticSearch |
| thinkvmc02 | 192.168.50.19 | CentOS7 | Logstash |
| thinkvmc03 | 192.168.50.54 | CentOS7 | Kibana |
ELK是需要Java的，建議安裝Java8。這里我就啰嗦了
# 先檢查JDK環境 [thinktik@thinkvmc01 ~]$ java -version java version "1.8.0_201" Java(TM) SE Runtime Environment (build 1.8.0_201-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)開始安裝
安裝ELK不難，按照官方的文檔即可，官網鏈接下
開源搜索與分析 · Elasticsearch
我們先安裝 ElasticSearch。先下載 Linux 通用安裝包 elasticsearch-6.7.1.tar.gz
。當然為了簡單，你也可以下載具體Linux發行版的預編譯包，這樣安裝更加簡單，不過缺少靈活性。
# 下載
[thinktik@thinkvmc01 thinktik]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.tar.gz
--2019-04-08 22:51:05-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.230.222, 2a04:4e42:1a::734
Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.230.222|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 148542786 (142M) [application/x-gzip]
Saving to: ‘elasticsearch-6.7.1.tar.gz.1’
2% [> ] 3,699,945 42.4KB/s eta 25m 36s
....
# 下載完畢，解壓
[thinktik@thinkvmc01 thinktik]# ls
elasticsearch-6.7.1.tar.gz java8 jdk-8u201-linux-x64.tar.gz
[thinktik@thinkvmc01 thinktik]# tar -zxvf elasticsearch-6.7.1.tar.gz
elasticsearch-6.7.1/
elasticsearch-6.7.1/lib/
....
elasticsearch-6.7.1/logs/
elasticsearch-6.7.1/plugins/
# 進入安裝文件目錄
[thinktik@thinkvmc01 thinktik]# cd elasticsearch-6.7.1
[thinktik@thinkvmc01 elasticsearch-6.7.1]# ls
bin config lib LICENSE.txt logs modules NOTICE.txt plugins README.textile
[thinktikt@thinkvmc01 elasticsearch-6.7.1]# cd config/
[thinktik@thinkvmc01 config]# ls
elasticsearch.yml jvm.options log4j2.properties role_mapping.yml roles.yml users users_roles
# 修改配置，綁定我們的網卡。不修改默認為127.0.0.1，那樣其余的機器上的Logstash，Kibana就沒法訪問這臺機的ES了
[thinktik@thinkvmc01 config]# vim elasticsearch.yml
#修改如下
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# IP地址
network.host: 192.168.50.207
#
# Set a custom port for HTTP:
# 端口，默認9200
http.port: 9200
# 啟動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
warning: Falling back to java on path. This behavior is deprecated. Specify JAVA_HOME
[2019-04-08T23:11:44,120][INFO ][o.e.e.NodeEnvironment ] [ZVfIMzv] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [26.7gb], net total_space [28.9gb], types [rootfs]
[2019-04-08T23:11:44,126][INFO ][o.e.e.NodeEnvironment ] [ZVfIMzv] heap size [1015.6mb], compressed
....
# 這里報了錯，很明顯了
ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-04-08T23:12:06,558][INFO ][o.e.n.Node ] [ZVfIMzv] stopping ...
[2019-04-08T23:12:06,636][INFO ][o.e.n.Node ] [ZVfIMzv] stopped
[2019-04-08T23:12:06,637][INFO ][o.e.n.Node ] [ZVfIMzv] closing ...
[2019-04-08T23:12:06,673][INFO ][o.e.n.Node ] [ZVfIMzv] closed
# 我們按它的提示該系統配置
[thinktik@thinkvmc01 bin]$ vim /etc/security/limits.conf
[thinktik@thinkvmc01 bin]$ su
Password:
#添加如下配置
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
# 繼續啟動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
# 報錯，那么繼續修改
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-04-08T23:19:38,825][INFO ][o.e.n.Node ] [ZVfIMzv] stopping ...
[2019-04-08T23:19:38,844][INFO ][o.e.n.Node ] [ZVfIMzv] stopped
[2019-04-08T23:19:38,845][INFO ][o.e.n.Node ] [ZVfIMzv] closing ...
[2019-04-08T23:19:38,887][INFO ][o.e.n.Node ] [ZVfIMzv] closed
[2019-04-08T23:19:38,889][INFO ][o.e.x.m.p.NativeController] [ZVfIMzv] Native controller process has stopped - no new native processes can be started
# 繼續修改
[thinktik@thinkvmc01 bin]$ su
Password:
[root@thinkvmc01 bin]# sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
# 繼續啟動
[thinktik@thinkvmc01 bin]$ ./elasticsearch
[2019-04-08T23:22:37,612][INFO ][o.e.c.s.ClusterApplierService] [ZVfIMzv] new_master {ZVfIMzv}{ZVfIMzviR5ie4WVCaO9CZA}{B3vTE3wKSriPc-LwHC8J-A}{192.168.50.207}{192.168.50.207:9300}{ml.machine_memory=1927471104, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {ZVfIMzv}{ZVfIMzviR5ie4WVCaO9CZA}{B3vTE3wKSriPc-LwHC8J-A}{192.168.50.207}{192.168.50.207:9300}{ml.machine_memory=1927471104, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-04-08T23:22:37,792][INFO ][o.e.h.n.Netty4HttpServerTransport] [ZVfIMzv] publish_address {192.168.50.207:9200}, bound_addresses {192.168.50.207:9200}
[2019-04-08T23:22:37,792][INFO ][o.e.n.Node ] [ZVfIMzv] started
[2019-04-08T23:22:38,740][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [ZVfIMzv] Failed to clear cache for realms [[]]
[2019-04-08T23:22:38,850][INFO ][o.e.l.LicenseService ] [ZVfIMzv] license [41e1ad3d-893b-48c6-98b1-71e02ab1a367] mode [basic] - valid
[2019-04-08T23:22:38,873][INFO ][o.e.g.GatewayService ] [ZVfIMzv] recovered [0] indices into cluster_state
# 成功
驗證
[thinktik@thinkvmc01 ~]$ netstat -nlp
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN -
# 9200,9300 被ES監聽
tcp6 0 0 192.168.50.207:9200 :::* LISTEN 12829/java
tcp6 0 0 192.168.50.207:9300 :::* LISTEN 12829/java
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 ::1:25 :::* LISTEN -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
raw6 0 0 :::58 :::* 7 -
raw6 0 0 :::58 :::* 7 -
# 防火墻開放端口
[root@thinkvmc01 thinktik]# firewall-cmd --zone=public --add-port=9200/tcp --permanent
success
[root@thinkvmc01 thinktik]# firewall-cmd --zone=public --add-port=9300/tcp --permanent
success
[root@thinkvmc01 thinktik]# firewall-cmd --reload
success
# thinkvmc02 主機驗證 thinkvmc01 的ES效果。你用瀏覽器驗證下面的地址也可以
[thinktik@thinkvmc02 ~]$ curl -i http://192.168.50.207:9200/
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 493
{
"name" : "ZVfIMzv",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "mhuFY2EcRl6Bt9xqKiyY7Q",
"version" : {
"number" : "6.7.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "2f32220",
"build_date" : "2019-04-02T15:59:27.961366Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
你用瀏覽器驗證下面的地址也可以
到這里ES就安裝好了
thinkvmc02 先安裝Logstash# 驗證java
[thinktik@thinkvmc02 java8]$ java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
# 下載
[thinktik@thinkvmc02 java8]$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.7.1.tar.gz
--2019-04-08 23:32:55-- https://artifacts.elastic.co/downloads/logstash/logstash-6.7.1.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.110.222, 2a04:4e42:36::734
Connecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.110.222|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 175824421 (168M) [application/x-gzip]
Saving to: ‘logstash-6.7.1.tar.gz’
6% [==> ] 10,605,295 113KB/s eta 9m 57s
...
[thinktik@thinkvmc02 ~]$ ls
java8 jdk-8u201-linux-x64.tar.gz logstash-6.7.1.tar.gz
[thinktik@thinkvmc02 ~]$ tar -zxvf logstash-6.7.1.tar.gz
...
logstash-6.7.1/x-pack/src/test/java/org
logstash-6.7.1/x-pack/src/test/java/org/logstash
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test/RSpecIntegrationTests.java
logstash-6.7.1/x-pack/src/test/java/org/logstash/xpack/test/RSpecTests.java
logstash-6.7.1/LICENSE.txt
logstash-6.7.1/logstash-core/lib/logstash/build.rb
[thinktik@thinkvmc02 ~]$ cd logstash-6.7.1
[thinktik@thinkvmc02 logstash-6.7.1]$ ls
bin CONTRIBUTORS Gemfile lib logstash-core modules tools x-pack
config data Gemfile.lock LICENSE.txt logstash-core-plugin-api NOTICE.TXT vendor
[thinktik@thinkvmc02 logstash-6.7.1]$ cd config/
[thinktik@thinkvmc02 config]$ ls
jvm.options log4j2.properties logstash-sample.conf logstash.yml pipelines.yml startup.options
[thinktik@thinkvmc02 config]$ cp logstash-sample.conf logstash.conf
[thinktik@thinkvmc02 config]$ vim logstash.conf
# 這里將ES地址寫對就可以了
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["http://192.168.50.207:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#user => "elastic"
#password => "changeme"
}
}
[thinktik@thinkvmc02 config]$ vim logstash.yml
# 這里寫正確自己的IP
# ------------ Metrics Settings --------------
#
# Bind address for the metrics REST endpoint
#
http.host: "192.168.50.19"
# 啟動
[thinktik@thinkvmc02 bin]$ ./logstash -f ../config/logstash.conf
Sending Logstash logs to /home/thinktik/logstash-6.7.1/logs which is now configured via log4j2.properties
[2019-04-08T23:47:53,295][WARN ][logstash.config.source.multilocal] Ignoring the "pipelines.yml" file because modules or command line options are specified
[2019-04-08T23:47:53,324][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.7.1"}
[2019-04-08T23:48:08,245][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-04-08T23:48:09,323][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.50.207:9200/]}}
# 日志顯示ES地址對了
[2019-04-08T23:48:09,919][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://192.168.50.207:9200/"}
[2019-04-08T23:48:10,080][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-04-08T23:48:10,096][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won"t be used to determine the document _type {:es_version=>6}
[2019-04-08T23:48:10,174][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://192.168.50.207:9200"]}
[2019-04-08T23:48:10,250][INFO ][logstash.outputs.elasticsearch] Using default mapping template
[2019-04-08T23:48:10,318][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2019-04-08T23:48:11,308][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2019-04-08T23:48:11,360][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#"}
[2019-04-08T23:48:11,499][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
# 日志顯示5044，9600被監聽
[2019-04-08T23:48:11,589][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2019-04-08T23:48:12,194][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
# 檢查下端口監聽
[thinktik@thinkvmc02 ~]$ netstat -nlp
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN -
tcp6 0 0 :::5044 :::* LISTEN 27467/java
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 ::1:25 :::* LISTEN -
tcp6 0 0 192.168.50.19:9600 :::* LISTEN 27467/java
udp 0 0 0.0.0.0:68 0.0.0.0:* -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
raw6 0 0 :::58 :::* 7 -
raw6 0 0 :::58 :::* 7 -
# 防火墻打開
[root@thinkvmc02 thinktik]# firewall-cmd --zone=public --add-port=9600/tcp --permanent
success
[root@thinkvmc02 thinktik]# firewall-cmd --zone=public --add-port=5044/tcp --permanent
success
[root@thinkvmc02 thinktik]# firewall-cmd --reload
success
到這里logstash安裝完畢
thinkvmc03 先安裝Kibana[thinktik@thinkvmc03 ~]$ java -version java version "1.8.0_201" Java(TM) SE Runtime Environment (build 1.8.0_201-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode) # 修改配置 [thinktik@thinkvmc03 config]$ pwd /home/thinktik/kibana-6.7.1-linux-x86_64/config [thinktik@thinkvmc03 config]$ vim kibana.yml # 這里修改為自己的IP，端口默認5601 # To allow connections from remote users, set this parameter to a non-loopback address. server.host: "192.168.50.54" # 這里修改ES服務的地址 # The URLs of the Elasticsearch instances to use for all your queries. elasticsearch.hosts: ["http://192.168.50.54:9200"] # 啟動 [thinktik@thinkvmc03 bin]$ ./kibana log [16:04:24.455] [info][status][plugin:kibana@6.7.1] Status changed from uninitialized to green - Ready log [16:04:24.507] [info][status][plugin:elasticsearch@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [16:04:24.510] [info][status][plugin:xpack_main@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [16:04:24.523] [info][status][plugin:graph@6.7.1] Status changed from uninitialized to yellow - Waiting for Elasticsearch # 檢查 [thinktik@thinkvmc03 config]$ netstat -nlp (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - # 監聽正常 tcp 0 0 192.168.50.54:5601 0.0.0.0:* LISTEN 27474/./../node/bin tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 ::1:25 :::* LISTEN - udp 0 0 0.0.0.0:68 0.0.0.0:* - udp 0 0 0.0.0.0:68 0.0.0.0:* - raw6 0 0 :::58 :::* 7 - raw6 0 0 :::58 :::* 7 - # 防火墻開放端口 [root@thinkvmc03 config]# firewall-cmd --zone=public --add-port=5601/tcp --permanent success [root@thinkvmc03 config]# firewall-cmd --reload success
kibana效果
到這里就完成了ELK基礎搭建
thinkvmc02 先安裝Filebeat接下來我們安裝 Filebeat，使用ELKF架構來實現log4j的日志收集。
為了方便Filebeat安裝在thinkvmc03上與thinkvmc02的Logstash形成分布式結構來模擬日志數據的收集與傳輸
官方的安裝教程也很簡單，屬于基本操作了
[thinktik@thinkvmc03 ~]$ tar -zxvf filebeat-6.7.1-linux-x86_64.tar.gz
filebeat-6.7.1-linux-x86_64/.build_hash.txt
filebeat-6.7.1-linux-x86_64/fields.yml
filebeat-6.7.1-linux-x86_64/LICENSE.txt
filebeat-6.7.1-linux-x86_64/NOTICE.txt
filebeat-6.7.1-linux-x86_64/kibana/
filebeat-6.7.1-linux-x86_64/kibana/5/
filebeat-6.7.1-linux-x86_64/
...
filebeat-6.7.1-linux-x86_64/module/traefik/access/machine_learning/visitor_rate.json
filebeat-6.7.1-linux-x86_64/module/traefik/access/manifest.yml
filebeat-6.7.1-linux-x86_64/module/traefik/module.yml
filebeat-6.7.1-linux-x86_64/filebeat.reference.yml
filebeat-6.7.1-linux-x86_64/filebeat
# 修改配置使filebeat對準我們的輸出
[thinktik@thinkvmc03 filebeat-6.7.1-linux-x86_64]$ vim filebeat.yml
#=========================== Filebeat inputs =============================
# 設置Filebeat讀取/home/thinktik/ELKF_TEST.log日志
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
# 這里設為True開啟日志讀入
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /home/thinktik/ELKF_TEST.log
#- /var/log/*.log
#- c:programdataelasticsearchlogs*
#-------------------------- Elasticsearch output ------------------------------
# 直接輸出到Elasticsearch 這里我們不建議直接輸出
#output.elasticsearch:
# Array of hosts to connect to.
# hosts: ["192.168.50.207:9200"]
# Enabled ilm (beta) to use index lifecycle management instead daily indices.
#ilm.enabled: false
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
# 這里才是Logstash，直接輸出到logstash 這里我們建議直接輸出，地址配對就可以
output.logstash:
# The Logstash hosts
hosts: ["192.168.50.19:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
# 保存后啟動
[thinktik@thinkvmc03 filebeat-6.7.1-linux-x86_64]$ ./filebeat
# 后續的你直接修改/home/thinktik/ELKF_TEST.log，寫入一些數據到這個文件里面等待Kibana顯示出來效果
# 我們先理下思路，流程是：filebeat -> Logstash -> ES -> Kibana
# 如果沒有問題，那么我們再Kibana檢查效果
檢查ELKF效果
這里看到日志被正確讀取
我們再細節設置下
我們搜索下看看
先搜索主機 host.name=thinkvmc03 的日志
再搜索主機 host.name=thinkvmc03 的日志 而且日志源文件是 source =/home/thinktik/ELKF_TEST.log的
看到匹配正確
我們繼續收集下其他類型的日志，那log4j來試一下
本文原創鏈接
Linux ELK 安裝（服務器架設篇）
參考鏈接
https://juejin.im/entry/59e6b...
文章版權歸作者所有，未經允許請勿轉載,若此文章存在違規行為，您可以聯系管理員刪除。
轉載請注明本文地址：http://specialneedsforspecialkids.com/yun/77528.html
閱讀 1897·2021-11-24 09:39
閱讀 2134·2021-09-22 15:50
閱讀 1991·2021-09-22 14:57
閱讀 699·2021-07-28 00:13
閱讀 1064·2019-08-30 15:54
閱讀 2356·2019-08-30 15:52
閱讀 2686·2019-08-30 13:07
閱讀 3787·2019-08-30 11:27
