elasticsearch & 记一次线上elasticsearch常用操作

elasticsearch & 记一次线上elasticsearch常用操作

五月 05, 2018

因线上服务产生的数据太多太大并且占用服务器内存,导致kibana经常查询失败和丢数据,将一部分不常用时间比较久的数据索引关闭,当运营需要查询某一天的数据时,将某一天的索引打开

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@ip 10.x.x.x ]# curl -X POST "http://10.x.x.x:9200/kbilogs-logics-gamex*2018.11.28/_open"
{"acknowledged":true,"shards_acknowledged":true}
[root@ip 10.x.x.x ec2-user]# curl -X POST "http://10.x.x.x:9200/kbilogs-logics-gamex*2018.11.28/_open" 10.x.x.x:9200/_cat/indices |grep 2018.11.28
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 close kbilogs-logics-crosservice-201.2018.11.28 SSJwkFtdRym7iFh9sqo7IQ
close kbilogs-logics-clientbilog.2018.11.28 yv10GRdrQeCsA9EsqVt4dw
close kbilogs-logics-multiplayer-200.2018.11.28 ZzZJrdRSQRet7ne8BQ7MWQ
100 close kbilogs-logics-pay-203.2018.11.28 qtt0qUuBTyCaoEdMeonJag
90083 close kbilogs-logics-multiplayer-204.2018.11.28 U1Ankh0ITuSmCXyyAcJieA
100 90083 0 0 70282 0 0:00:01 0:00:01 --:--:-- 70322
green open kbilogs-logics-gamex-200.2018.11.28 zV0dvfrQTe2W00MlCWaJyA 5 2 6399950 0 5.3gb 1.7gb
close kbilogs-logics-pay-200.2018.11.28 wRPBdg6EQcq28nZZ3Qqd8Q
close kbilogs-logics-crosservice-200.2018.11.28 oazIKDxlQTOqEfPbMd725w
close kbilogs-logics-multiplayer-202.2018.11.28 DuB4TE8gSS-s-y08lT3Kxg
close kbilogs-logics-pay-202.2018.11.28 YHbnaAOERXeg5WgN5h9Xvg
close kbilogs-logics-pay-201.2018.11.28 v72XCxeTQKqevBJbjV6FFw
close kbilogs-logics-anticheat_multiplayer.2018.11.28 K9nSkoVrR0WLXF9GrP6YuQ
close kbilogs-logics-multiplayer-201.2018.11.28 sSZYkoKjQUCXheozZIdcYA
close kbilogs-logics-crosservice-204.2018.11.28 20QOeBX6R0GDP3lY_3YTFg
close kbilogs-logics-pay-204.2018.11.28 ndGv-L2vRkynHkioT9XPDg
green open kbilogs-logics-gamex-203.2018.11.28 PnsR_1pfQd2ZFwqEapQ-uA 5 2 684706 0 566.1mb 188.7mb
close kbilogs-logics-crosservice-202.2018.11.28 kzLaaXyXTDGO8cUk0s-GdQ
close kbilogs-logics-auth.2018.11.28 8EqyrF4SQN2V-rDLjIDe3A
green open kbilogs-logics-gamex-204.2018.11.28 fbIgQVN6RXaeexRMfWib2w 5 2 48630621 0 41gb 13.6gb
green open kbilogs-logics-gamex-202.2018.11.28 f_slUsJ9SQC_sa_-Z5v1XQ 5 2 17299793 0 14.3gb 4.7gb
green open kbilogs-logics-gamex-201.2018.11.28 CfnnWRy2T52fmjcsanLUtA 5 2 18702652 0 15.5gb 5.1gb
close kbilogs-logics-crosservice-203.2018.11.28 ChIFyCgiRUSp8fVMbRCN1w

elasticsearch 常用的操作

  1. 查看当前集群的状态
    curl ‘http://ip:9200/_cat/health?v'

  2. 查看当前node
    curl ‘http://ip:9200/_cat/nodes?v'
    curl -XGET ‘http://ip:9200/_nodes/stats/os'
    curl -XGET ‘http://ip:9200/_nodes/stats/process'
    curl -XGET ‘http://ip:9200/_nodes/stats/indices'
    curl -XGET ‘http://ip:9200/_nodes/stats/http'
    curl -XGET ‘http://ip:9200/_nodes/stats/fs'
    curl -XGET ‘http://ip:9200/_nodes/stats/jvm'
    curl -XGET ‘http://ip:9200/_nodes/stats/thread_pool'
    curl -XGET ‘http://ip:9200/_nodes/stats/transport' ###Transport statistics about sent and received bytes in cluster communication
    curl -XGET ‘http://ip:9200/_nodes/stats/breaker' ##Statistics about the field data circuit breaker
    curl -XGET ‘http://ip:9200/_nodes/stats/process'

  3. 查看所有的index
    curl ‘http://ip:9200/_cat/indices?v'

  4. 创建一个index
    curl -XPUT ‘ip:9200/customer?pretty’

  5. 创建一个index 带有文档的
    curl -XPUT ‘ip:9200/customer/external/1?pretty’ -d ‘
    {
    “name”: “John Doe”
    }’

  6. 删除一个index
    curl -XDELETE ‘ip:9200/customer?pretty’
    curl -XDELETE ‘ip:9200/onland-?pretty’
    curl -XDELETE ‘ip:9200/yingxiong_gm.
    .05.2017’
    curl -XDELETE ‘http://ip:9200/bilogs-clientbilog-203.01.*.2018'

  7. update index
    This example shows how to update our previous document (ID of 1) by changing the name field to “Jane Doe”:
    curl -XPOST ‘ip:9200/customer/external/1/_update?pretty’ -d ‘
    {
    “doc”: { “name”: “Jane Doe” }
    }’

  8. elasticsearch 还原的case https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

  9. elasticsearch rolling update https://www.elastic.co/guide/en/elasticsearch/reference/current/rolling-upgrades.html

  10. elasticsearch full restart https://www.elastic.co/guide/en/elasticsearch/reference/current/restart-upgrade.html

  11. elasticsearch 2.2版本的变化 https://www.elastic.co/guide/en/elasticsearch/reference/current/float.html

  12. plugin 插件的版本
    curl ‘ip:9200/_cat/plugins?v’

  13. 查看集群的health
    curl -XGET ‘http://ip:9200/_cluster/health?pretty=true'

  14. 集群的 stats
    curl -XGET ‘http://ip:9200/_cluster/stats?human&pretty'

  15. cluster reroute 有一定的风险丢数据,虽然能让集群的状态变成green,慎用 https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-reroute.html

  16. cluster setting
    curl -XPUT ip:9200/_cluster/settings -d ‘{
    “persistent” : {

    "discovery.zen.minimum_master_nodes": 2

    }
    }’
    curl -XGET ip:9200/_cluster/settings

  17. 可以动态设置配置的 module A list of dynamically updatable settings can be found in the Modules documentation.

  18. dynamic index mapping https://www.elastic.co/guide/en/elasticsearch/reference/current/default-mapping.html

  19. elastcsearch cluster https://www.elastic.co/guide/en/elasticsearch/reference/current/shards-allocation.html
    curl ‘ip:9200/_cat/shards?v’
    curl -XGET http://ip:9200/_template | python -m json.tool
    可以直接恢复 elasticsearch 数据目录 删除掉 数据目录下面的 .kibana 文件夹 导入到新的集群