Ubuntu18.04 – EFK 구축

ElasticSearch + Fluentd + Kibana

• 자바 11 설치(최신 elasticsearch는 자바11로 적용)

sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt install openjdk-11-jdk
java -version

• elasticsearch 설치
  • 패키지로 설치

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.2-amd64.deb
dpkg -i elasticsearch-7.6.2-amd64.deb
// curl 로 정상적인지 확인
curl -XGET localhost:9200
// 아래와 같이 나옴
{
   "name" : "ip-172-31-19-12",
   "cluster_name" : "elasticsearch",
   "cluster_uuid" : "hf2TTutAS1mfEiyXPNs3sQ",
   "version" : {
     "number" : "7.6.2",
     "build_flavor" : "default",
     "build_type" : "deb",
     "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
     "build_date" : "2020-03-26T06:34:37.794943Z",
     "build_snapshot" : false,
     "lucene_version" : "8.4.0",
     "minimum_wire_compatibility_version" : "6.8.0",
     "minimum_index_compatibility_version" : "6.0.0-beta1"
   },
   "tagline" : "You Know, for Search"
 }

• /etc/elasticsearch/elasticsearch.yml 수정

# curl -XGET localhost:9200이 connection refues일때재시작
systemctl restart elasticsearch.service

• kibana 설치

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.6.2-amd64.deb
shasum -a 512 kibana-7.6.2-amd64.deb
sudo dpkg -i kibana-7.6.2-amd64.deb

• kibana 설정

vi /etc/kibana/kibana.yml

server.host: "localhost" => server.host: 0.0.0.0 (외부 접속)
elasticsearch.hosts: ["http://localhost:9200"] (elasticsearch 주석해제, ip는 설정에 맞게)

• fluentd 설치

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent3.sh | sh

• fluentd 플러그인 설치 (elasticsearch 플러그인)

sudo apt-get install make libcurl4-gnutls-dev --yes
sudo /opt/td-agent/embedded/bin/fluent-gem install fluent-plugin-elasticsearch
sudo /opt/td-agent/embedded/bin/fluent-gem install fluent-plugin-record-reformer

• td-agent.conf 수정
  • pos_file이 없으면 재시작시 문제가 생길수있다.
  • pos_file은 만들어주면 끝(권한 이슈로 에러날수있으니 파일에권한부여)

vi /etc/td-agent/td-agent.conf

<source>
  @type tail
   path /home/shotan/app.log
   pos_file /home/shotan/fluentd/log.pos
   tag gamelog
   format json
</source>

<match>
  @type elasticsearch
   host localhost
   port 9200
   logstash_format true
</match>

• 재시작 명령어

systemctl restart elasticsearch.service 
systemctl restart td-agent
systemctl restart kibana

• kibana 페이지 설정

• 

• 

• 

• 

• 

• 

• EFK 각 사이트 주소 (다운로드 주소)

 https://docs.fluentd.org/installation/install-by-deb 
 https://www.elastic.co/kr/downloads/elasticsearch 
 https://www.elastic.co/guide/en/kibana/current/deb.html