https://biggongjam.notion.site/Logstash-AWS-EC2-Logstash-7-17-2-b79a2613f6cb4148b8ba0c820b685731
Logstash - AWS EC2에서 Logstash 7.17.2 설치 및 실행
작성장 : 김재민
biggongjam.notion.site
Siem 로그 시스템 한번 구현해볼 생각이다.
결국은 상용 시스템 도입할 예정이지만 어떻게 돌아가는지 확인해볼 필요는 있다.
| sudo apt-get -y update sudo apt-get -y upgrade sudo apt-get -y dist-upgrade sudo apt-get install -y vim wget unzip ssh openssh-* net-tools sudo hostnamectl set-hostname logstash01 sudo apt-get install -y openjdk-8-jdk java -version sudo find / -name java-8-openjdk-amd64 2>/dev/null sudo mkdir -p /install_dir && cd /install_dir pwd wget https://artifacts.elastic.co/downloads/logstash/logstash-8.2.2-linux-x86_64.tar.gz ls sudo tar -zxvf logstash-8.2.2-linux-x86_64.tar.gz -C /usr/local sudo mv /usr/local/logstash-8.2.2/ /usr/local/logstash sudo chown -R $USER:$USER /usr/local/logstash && cd /usr/local/logstash logstash.conf YML CONF 확장자 확인 필수 ls clear ls -al vi /usr/local/logstash/config/logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls vim /usr/local/logstash/config/logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls vi config ls cd config/ ls vi logstash.conf /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ls clear ls vi logstash.conf input { } output { clear /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf pwd ls ls -al ps -ef | grep logstash kill 57810 /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/logstash.conf ps -ef | grep logstash kill -9 57810 |
AWS 오픈서치나 엘렉스틱 서치 클라우드 서비스를 받는다고 해도 스토리지 요금이 꽤나 많이 나온다고 한다.
그래서 로컬에서 NAS로 구축이 가능하다면 로컬로 구축해서 쓰는것도 나쁘지 않다고 한다.
| input { http_poller { urls => { naver => { method => get url => "https://api.coingecko.com/api/v3/exchanges/upbit" } } request_timeout => 10 schedule => { cron => "* * * * * UTC " } codec => "json" metadata_target => "http_poller_metadata" } } output { stdout { } } |
| input { http_poller { urls => { test1 => "https://api.coingecko.com/api/v3/exchanges/upbit" test2 => { # Supports all options supported by ruby's Manticore HTTP client method => get url => "https://api.coingecko.com/api/v3/exchanges/upbit" headers => { Accept => "application/json" } } } request_timeout => 10 # Supports "cron", "every", "at" and "in" schedules by rufus scheduler schedule => { cron => "* * * * * UTC"} codec => "json" # A hash of request metadata info (timing, response headers, etc.) will be sent here metadata_target => "http_poller_metadata" } } output { elasticsearch { hosts => "localhost:9200" index => "covid_test" document_type => "search" #document_id => "%{[@metadata][_id]}" } stdout { codec => rubydebug } } |
파싱이 되어온다 indexer를 어떤 방식으로 쓴는지 공부해보자
'시스템' 카테고리의 다른 글
| docker-compse Elasticsearch 설치 (0) | 2023.05.18 |
|---|---|
| 키바나 설치 (0) | 2023.04.28 |
| iptime 공유기 & mesh linksys 콜라보레이션 무선 AP 구성 (0) | 2023.04.18 |
| MongoDB Replica Set (0) | 2023.01.10 |
| s3 권한이 없는 경우 (0) | 2022.05.17 |
iesay
