Ubuntu18.04 – EFK 구축

ElasticSearch + Fluentd + Kibana

  • 자바 11 설치(최신 elasticsearch는 자바11로 적용)
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt install openjdk-11-jdk
java -version
  • elasticsearch 설치
    • 패키지로 설치
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.2-amd64.deb
dpkg -i elasticsearch-7.6.2-amd64.deb
// curl 로 정상적인지 확인
curl -XGET localhost:9200
// 아래와 같이 나옴
{
   "name" : "ip-172-31-19-12",
   "cluster_name" : "elasticsearch",
   "cluster_uuid" : "hf2TTutAS1mfEiyXPNs3sQ",
   "version" : {
     "number" : "7.6.2",
     "build_flavor" : "default",
     "build_type" : "deb",
     "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
     "build_date" : "2020-03-26T06:34:37.794943Z",
     "build_snapshot" : false,
     "lucene_version" : "8.4.0",
     "minimum_wire_compatibility_version" : "6.8.0",
     "minimum_index_compatibility_version" : "6.0.0-beta1"
   },
   "tagline" : "You Know, for Search"
 }
  • kibana 설치
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.6.2-amd64.deb
shasum -a 512 kibana-7.6.2-amd64.deb
sudo dpkg -i kibana-7.6.2-amd64.deb
  • kibana 설정
vi /etc/kibana/kibana.yml
server.host: "localhost" => server.host: 0.0.0.0 (외부 접속)
elasticsearch.hosts: ["http://localhost:9200"] (elasticsearch 주석해제, ip는 설정에 맞게)
  • fluentd 설치
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent3.sh | sh
  • fluentd 플러그인 설치 (elasticsearch 플러그인)
sudo apt-get install make libcurl4-gnutls-dev --yes
sudo /opt/td-agent/embedded/bin/fluent-gem install fluent-plugin-elasticsearch
sudo /opt/td-agent/embedded/bin/fluent-gem install fluent-plugin-record-reformer
  • td-agent.conf 수정
    • pos_file이 없으면 재시작시 문제가 생길수있다.
    • pos_file은 만들어주면 끝(권한 이슈로 에러날수있으니 파일에권한부여)
vi /etc/td-agent/td-agent.conf
<source>
  @type tail
   path /home/shotan/app.log
   pos_file /home/shotan/fluentd/log.pos
   tag gamelog
   format json
</source>

<match>
  @type elasticsearch
   host localhost
   port 9200
   logstash_format true
</match>
  • 재시작 명령어
systemctl restart elasticsearch.service 
systemctl restart td-agent
systemctl restart kibana
  • kibana 페이지 설정
  • EFK 각 사이트 주소 (다운로드 주소)
 https://docs.fluentd.org/installation/install-by-deb 
 https://www.elastic.co/kr/downloads/elasticsearch 
 https://www.elastic.co/guide/en/kibana/current/deb.html 

Leave a Reply

Your email address will not be published. Required fields are marked *