Logstash 설치

https://biggongjam.notion.site/Logstash-AWS-EC2-Logstash-7-17-2-b79a2613f6cb4148b8ba0c820b685731

Logstash - AWS EC2에서 Logstash 7.17.2 설치 및 실행

 

Siem 로그 시스템 한번 구현해볼 생각이다.

결국은 상용 시스템 도입할 예정이지만 어떻게 돌아가는지 확인해볼 필요는 있다.

 

sudo apt-get -y update sudo apt-get -y upgrade sudo apt-get -y dist-upgrade sudo apt-get install -y vim wget unzip ssh openssh-* net-tools sudo hostnamectl set-hostname logstash01 sudo apt-get install -y openjdk-8-jdk java -version sudo find / -name java-8-openjdk-amd64 2>/dev/null sudo mkdir -p /install_dir && cd /install_dir pwd wget https://artifacts.elastic.co/downloads/logstash/logstash-8.2.2-linux-x86_64.tar.gz ls sudo tar -zxvf logstash-8.2.2-linux-x86_64.tar.gz -C /usr/local sudo mv /usr/local/logstash-8.2.2/ /usr/local/logstash sudo chown -R $USER:$USER /usr/local/logstash && cd /usr/local/logstash logstash.conf YML CONF 확장자 확인 필수  ls clear ls -al vi /usr/local/logstash/config/logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls vim /usr/local/logstash/config/logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls vi config ls cd config/ ls vi logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls clear ls vi logstash.conf input { } output { clear /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf pwd ls ls -al ps -ef | grep logstash kill 57810 /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ps -ef | grep logstash kill -9 57810

 

AWS 오픈서치나 엘렉스틱 서치 클라우드 서비스를 받는다고 해도 스토리지 요금이 꽤나 많이 나온다고 한다.

그래서 로컬에서 NAS로 구축이 가능하다면 로컬로 구축해서 쓰는것도 나쁘지 않다고 한다.

 

input {   http_poller {     urls => {             naver => {             method => get             url => "https://api.coingecko.com/api/v3/exchanges/upbit"             }     }     request_timeout => 10     schedule => { cron => "* * * * * UTC " }     codec => "json"     metadata_target => "http_poller_metadata"   } } output {   stdout {   } }

 

input {   http_poller {     urls => {       test1 => "https://api.coingecko.com/api/v3/exchanges/upbit"       test2 => {         # Supports all options supported by ruby's Manticore HTTP client         method => get         url => "https://api.coingecko.com/api/v3/exchanges/upbit"         headers => {           Accept => "application/json"         }      }     }     request_timeout => 10     # Supports "cron", "every", "at" and "in" schedules by rufus scheduler     schedule => { cron => "* * * * * UTC"}     codec => "json"     # A hash of request metadata info (timing, response headers, etc.) will be sent here     metadata_target => "http_poller_metadata"   } }  output {    elasticsearch {      hosts => "localhost:9200"          index => "covid_test"          document_type => "search"          #document_id => "%{[@metadata][_id]}"    }    stdout { codec => rubydebug }  }

 

 

 

 

 

파싱이 되어온다 indexer를 어떤 방식으로 쓴는지 공부해보자