分布式ES：Elasticsearch安装与配置（四）

创建文件夹/application/tools，并下载Elasticsearch到该目录。下载地址：https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-linux-x86_64.tar.gz

[root@elasticsearch /]# mkdir /application/tools -p
[root@elasticsearch /]# cd /application/tools
[root@elasticsearch tools]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-linux-x86_64.tar.gz

解压软件包，移动到/application下，并创建data目录。（Elasticsearch需要data目录来存放数据，源中没该目录）

[root@elasticsearch tools]# tar -xf elasticsearch-7.6.1-linux-x86_64.tar.gz 
[root@elasticsearch tools]# mv elasticsearch-7.6.1 /application/
[root@elasticsearch tools]# cd /application/elasticsearch-7.6.1/
[root@elasticsearch elasticsearch-7.6.1]# mkdir data
[root@elasticsearch elasticsearch-7.6.1]# ll
total 548
drwxr-xr-x  2 root root   4096 Feb 29 08:18 bin
drwxr-xr-x  2 root root    148 Feb 29 08:18 config
drwxr-xr-x  2 root root      6 Mar 26 08:51 data
drwxr-xr-x  9 root root    107 Feb 29 08:18 jdk
drwxr-xr-x  3 root root   4096 Feb 29 08:18 lib
-rw-r--r--  1 root root  13675 Feb 29 08:13 LICENSE.txt
drwxr-xr-x  2 root root      6 Feb 29 08:18 logs
drwxr-xr-x 38 root root   4096 Feb 29 08:18 modules
-rw-r--r--  1 root root 523209 Feb 29 08:18 NOTICE.txt
drwxr-xr-x  2 root root      6 Feb 29 08:18 plugins
-rw-r--r--  1 root root   8164 Feb 29 08:13 README.asciidoc
[root@elasticsearch elasticsearch-7.6.1]#

## bin：可执行文件在里面，运行es的命令就在这个里面，包含了一些脚本文件等
## config：配置文件目录
## JDK：java环境
## lib：依赖的jar，类库
## logs：日志文件
## modules：es相关的模块
## plugins：可以自己开发的插件
## data：这个目录没有，自己新建一下，后面要用 -> mkdir data，这个作为索引目录

配置config下的elasticsearch.yml文件。

# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elasticsearch          #集群名
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: es-node1          #节点名
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /application/elasticsearch-7.6.1/data          #数据路径
#
# Path to log files:
#
path.logs: /application/elasticsearch-7.6.1/logs          #日志路径

  。。。

# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0           #绑定地址，0.0.0.0表示所有ip都可访问
#
# Set a custom port for HTTP:
#
#http.port: 9200                #端口保持默认
#
# For more information, consult the network module documentation.

  。。。

# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["es-node1"]    #初始化节点，保持与上面的节点名一致
#
# For more information, consult the discovery and cluster formation module documentation.
#

## 9200：Http协议，用于外部通讯
## 9300：Tcp协议，ES集群之间是通过9300通讯

#### 如果涉及到跨域问题，可以配置以下信息。（Network中） ####

# ---------------------------------- Network -----------------------------------
http.cors.enabled:true
http.cors.allow-origin:"*"

配置config下的jvm.options文件。

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms128m                #JVM的堆内存最小值（请根据当前系统内存进行适当的分配）
-Xmx128m                #JVM的堆内存最大值（请根据当前系统内存进行适当的分配）

Elasticsearch不允许采用root用户启动，我们需要添加一个esuser，并进行授权。

[root@elasticsearch elasticsearch-7.6.1]# useradd esuser
[root@elasticsearch elasticsearch-7.6.1]# cd ..
[root@elasticsearch application]# chown -R esuser:esuser elasticsearch-7.6.1/
[root@elasticsearch application]# ll elasticsearch-7.6.1/
total 552
drwxr-xr-x  2 esuser esuser   4096 Feb 29 08:18 bin
drwxr-xr-x  2 esuser esuser    178 Mar 26 09:14 config
drwxr-xr-x  2 esuser esuser      6 Mar 26 08:51 data
drwxr-xr-x  9 esuser esuser    107 Feb 29 08:18 jdk
drwxr-xr-x  3 esuser esuser   4096 Feb 29 08:18 lib
-rw-r--r--  1 esuser esuser  13675 Feb 29 08:13 LICENSE.txt
drwxr-xr-x  2 esuser esuser   4096 Mar 26 09:15 logs
drwxr-xr-x 38 esuser esuser   4096 Feb 29 08:18 modules
-rw-r--r--  1 esuser esuser 523209 Feb 29 08:18 NOTICE.txt
drwxr-xr-x  2 esuser esuser      6 Feb 29 08:18 plugins
-rw-r--r--  1 esuser esuser   8164 Feb 29 08:13 README.asciidoc

切换到esuser用户，启动Elasticsearch。

[root@elasticsearch application]# su esuser
[esuser@elasticsearch application]$ cd elasticsearch-7.6.1/
[esuser@elasticsearch elasticsearch-7.6.1]$ bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2020-03-26T09:19:49,140][INFO ][o.e.e.NodeEnvironment    ] [es-node1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [25.6gb], net total_space [27.9gb], types [rootfs]
[2020-03-26T09:19:49,143][INFO ][o.e.e.NodeEnvironment    ] [es-node1] heap size [123.7mb], compressed ordinary object pointers [true]
[2020-03-26T09:19:49,196][INFO ][o.e.n.Node               ] [es-node1] node name [es-node1], node ID [VgYQWHMvRP2g1BhWfSEvLw], cluster name [elasticsearch]
[2020-03-26T09:19:49,196][INFO ][o.e.n.Node               ] [es-node1] version[7.6.1], pid[16025], build[default/tar/aa751e09be0a5072e8570670309b1f12348f023b/2020-02-29T00:15:25.529771Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/13.0.2/13.0.2+8]
[2020-03-26T09:19:49,197][INFO ][o.e.n.Node               ] [es-node1] JVM home [/application/elasticsearch-7.6.1/jdk]
[2020-03-26T09:19:49,197][INFO ][o.e.n.Node               ] [es-node1] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=COMPAT, -Xms128m, -Xmx128m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-12739566337847309953, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=67108864, -Des.path.home=/application/elasticsearch-7.6.1, -Des.path.conf=/application/elasticsearch-7.6.1/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2020-03-26T09:19:51,528][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [aggs-matrix-stats]
[2020-03-26T09:19:51,528][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [analysis-common]
[2020-03-26T09:19:51,528][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [flattened]
[2020-03-26T09:19:51,528][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [frozen-indices]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [ingest-common]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [ingest-geoip]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [ingest-user-agent]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [lang-expression]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [lang-mustache]
[2020-03-26T09:19:51,529][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [lang-painless]
[2020-03-26T09:19:51,530][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [mapper-extras]
[2020-03-26T09:19:51,530][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [parent-join]
[2020-03-26T09:19:51,530][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [percolator]
[2020-03-26T09:19:51,530][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [rank-eval]
[2020-03-26T09:19:51,532][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [reindex]
[2020-03-26T09:19:51,532][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [repository-url]
[2020-03-26T09:19:51,532][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [search-business-rules]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [spatial]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [transform]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [transport-netty4]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [vectors]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-analytics]
[2020-03-26T09:19:51,533][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-ccr]
[2020-03-26T09:19:51,534][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-core]
[2020-03-26T09:19:51,534][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-deprecation]
[2020-03-26T09:19:51,534][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-enrich]
[2020-03-26T09:19:51,534][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-graph]
[2020-03-26T09:19:51,534][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-ilm]
[2020-03-26T09:19:51,535][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-logstash]
[2020-03-26T09:19:51,535][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-ml]
[2020-03-26T09:19:51,535][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-monitoring]
[2020-03-26T09:19:51,535][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-rollup]
[2020-03-26T09:19:51,536][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-security]
[2020-03-26T09:19:51,536][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-sql]
[2020-03-26T09:19:51,536][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-voting-only-node]
[2020-03-26T09:19:51,536][INFO ][o.e.p.PluginsService     ] [es-node1] loaded module [x-pack-watcher]
[2020-03-26T09:19:51,537][INFO ][o.e.p.PluginsService     ] [es-node1] no plugins loaded
[2020-03-26T09:19:55,700][INFO ][o.e.x.s.a.s.FileRolesStore] [es-node1] parsed [0] roles from file [/application/elasticsearch-7.6.1/config/roles.yml]
[2020-03-26T09:19:56,274][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [es-node1] [controller/16119] [Main.cc@110] controller (64 bit): Version 7.6.1 (Build 6eb6e036390036) Copyright (c) 2020 Elasticsearch BV
[2020-03-26T09:19:56,834][DEBUG][o.e.a.ActionModule       ] [es-node1] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-03-26T09:19:56,994][INFO ][o.e.d.DiscoveryModule    ] [es-node1] using discovery type [zen] and seed hosts providers [settings]
[2020-03-26T09:19:57,999][INFO ][o.e.n.Node               ] [es-node1] initialized
[2020-03-26T09:19:58,000][INFO ][o.e.n.Node               ] [es-node1] starting ...
[2020-03-26T09:19:58,168][INFO ][o.e.t.TransportService   ] [es-node1] publish_address {10.0.0.220:9300}, bound_addresses {[::]:9300}
[2020-03-26T09:19:58,508][INFO ][o.e.b.BootstrapChecks    ] [es-node1] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /application/elasticsearch-7.6.1/logs/elasticsearch.log
[2020-03-26T09:19:58,547][INFO ][o.e.n.Node               ] [es-node1] stopping ...
[2020-03-26T09:19:58,593][INFO ][o.e.n.Node               ] [es-node1] stopped
[2020-03-26T09:19:58,594][INFO ][o.e.n.Node               ] [es-node1] closing ...
[2020-03-26T09:19:58,652][INFO ][o.e.n.Node               ] [es-node1] closed
[2020-03-26T09:19:58,657][INFO ][o.e.x.m.p.NativeController] [es-node1] Native controller process has stopped - no new native processes can be started
[esuser@elasticsearch elasticsearch-7.6.1]$

##根据最后几行，我们会发现Elasticsearch并没有正常启动。

解决错误。
错误一：每个进程最大同时打开文件数太小。

[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

可通过下面2个命令查看当前数量

[esuser@elasticsearch elasticsearch-7.6.1]$ ulimit -Hn
4096
[esuser@elasticsearch elasticsearch-7.6.1]$ ulimit -Sn
1024
[esuser@elasticsearch elasticsearch-7.6.1]$

修改/etc/security/limits.conf文件，增加配置，用户退出后重新登录生效

[root@elasticsearch elasticsearch-7.6.1]# vim /etc/security/limits.conf 

#<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4
* soft nofile 65536
* hard nofile 131072

# End of file

错误二：elasticsearch用户拥有的内存权限太小，至少需要262144；

[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

修改/etc/sysctl.conf文件，增加配置vm.max_map_count=262144

[root@elasticsearch elasticsearch-7.6.1]# vim /etc/sysctl.conf 

vm.max_map_count=262144

执行命令sysctl -p生效

[root@elasticsearch elasticsearch-7.6.1]# sysctl -p
vm.max_map_count = 262144

（可能）错误三：问题同错误一，最大线程个数太低。

max number of threads [3818] for user [es] is too low, increase to at least [4096]

可通过命令查看

[root@elasticsearch elasticsearch-7.6.1]# ulimit -Hu
7859
[root@elasticsearch elasticsearch-7.6.1]# ulimit -Su
7859

修改配置文件/etc/security/limits.conf（和错误一是一个文件），增加配置。

[root@elasticsearch elasticsearch-7.6.1]# vim /etc/security/limits.conf 

* soft nproc 2048
* hard nproc 4096

重新切换到esuser用户，启动并访问。

[root@elasticsearch ~]# su esuser
[esuser@elasticsearch root]$ cd /application/elasticsearch-7.6.1/
[esuser@elasticsearch elasticsearch-7.6.1]$ bin/elasticsearch
  
  。。。

[2020-03-26T10:12:42,567][INFO ][o.e.l.LicenseService     ] [es-node1] license [c7ed25ac-4bdd-4c71-9951-580d62173dd6] mode [basic] - valid
[2020-03-26T10:12:42,569][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [es-node1] Active license is now [BASIC]; Security is disabled

通过10.0.0.220:9200在浏览器上访问。

##后台运行：bin/elasticsearch -d
##关闭进程：Jps，kill pid