ELK批量删除索引

一、存在问题

用了一段时间elk发现如果索引长时间不删除，elk会越来越慢，重启elasticsearch服务器节点之前同步时间也会很长

二、解决方法（定期删除索引）

1.在elasticsearch节点上使用curl -XGET 'http://192.168.X.XX:9200/_cat/shards'查看索引

[root@--x-x scripts]# curl -XGET 'http://192.168.x.x:9200/_cat/shards' | more

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

                      --:--:-- --:--:-- --:--:--     0freetrip-2019.01.    r STARTED     59kb 192.168.x.x node3-192.168.x.x

freetrip-2019.01.                 p STARTED         59kb 192.168.x.x node1-192.168.x.x

freetrip-2019.01.                 p STARTED      .7kb 192.168.x.x node3-192.168.x.x

freetrip-2019.01.                 r STARTED      .7kb 192.168.x.x node1-192.168.x.x

freetrip-2019.01.                 p STARTED       .5kb 192.168.x.x node2-192.168.x.x

freetrip-2019.01.                 r STARTED       .5kb 192.168.x.x node1-192.168.x.x

freetrip-2019.01.                 r STARTED       .5kb 192.168.x.x node2-192.168.x.x

freetrip-2019.01.                 p STARTED       .5kb 192.168.x.x node3-192.168.x.x

freetrip-2019.01.                 p STARTED       .9kb 192.168.x.x node2-192.168.x.x

freetrip-2019.01.                 r STARTED       .9kb 192.168.x.x node3-192.168.x.x

2.将过滤出来要删除的索引存到临时文件夹里面

[root@--x-x ~]# curl -XGET 'http://192.168.x.x:9200/_cat/shards' | grep 192.168.x.x | grep  | awk '{print $1}' |uniq > elk-index.tmp

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

                142k       --:--:-- --:--:-- --:--:--  142k

[root@--- ~]# cat elk-index.tmp

otter-canal--2018.12.

logstash-nginx-chm-accesslog-2018.11.

logstash-nginx-oms-accesslog-2018.10.

laravel-chm248-online-2018.12.

otter-canal--2018.11.

logstash-nginx-oms-accesslog-2018.11.

otter-communication--2018.11.

otter-communication--2018.12.

otter-node--2018.12.

logstash-nginx-chvisa-accesslog-2018.10.

wanmei219-online-2018.10.

logstash-nginx-chm-accesslog-2018.11.

otter-canal--2018.11.

otter-canal--2018.11.

logstash-nginx-oms-accesslog-2018.11.

laravel-chm248-online-2018.11.

logstash-nginx-oms-accesslog-2018.12.

otter-manager--2018.12.

logstash-nginx-chvisa-accesslog-2018.11.

wanmei219-online-2018.12.

logstash-nginx-chvisa-accesslog-2018.11.

otter-communication--2018.11.

otter-communication--2018.11.

laravel-chm248-online-2018.11.

laravel-chm248-online-2018.10.

logstash-nginx-chm-accesslog-2018.12.

wanmei219-online-2018.11.

logstash-nginx-chvisa-accesslog-2018.12.

wanmei219-online-2018.11.

logstash-nginx-chm-accesslog-2018.10.

3. 删除过滤出来的索引文件

for i in `cat elk-index.tmp`

do

      curl -XDELETE  http://192.168.3.163:9200/$i

done

4. 使用脚本加定时任务，每一天删除前三天的索引

[root@--x-x ~]# cat /home/scripts/del_elasticseatch_index.sh

#!/bin/bash

curl -XGET 'http://192.168.x.x:9200/_cat/shards' |grep 192.168.x.x | awk '{print $1}' |grep `date -d "5 days ago" +%Y.%m.%d` | uniq > /tmp/index_name.tmp

for index_name in `cat /tmp/index_name.tmp`

do

   echo $index_name

    curl -XDELETE  http://192.168.x.x:9200/$index_name

    echo "${index_name} delete success" >> /home/scripts/del_elasticseatch_index.log

done

5. 定时任务

[root@--x-x ~]# crontab -l

  * * * bash /home/scripts/del_elasticseatch_index.sh

巴特西

ELK批量删除索引

一、存在问题

二、解决方法（定期删除索引）

最新文章

热门文章