Elasticsearch, 基于java的搜索引擎程序

 Free and Open, Distributed, RESTful Search Engine.

https://www.elastic.co/products/elasticsearch

Elasticsearch is the distributed, RESTful search and analytics engine at the heart of the Elastic Stack. You can use Elasticsearch to store, search, and manage data for:

• Logs

• Metrics

• A search backend

• Application monitoring

• Endpoint security

... and more!

To learn more about Elasticsearch’s features and capabilities, see our product page.

Get started

The simplest way to set up Elasticsearch is to create a managed deployment with Elasticsearch Service on Elastic Cloud.

If you prefer to install and manage Elasticsearch yourself, you can download the latest version from elastic.co/downloads/elasticsearch.

For more installation options, see the Elasticsearch installation documentation.

Upgrade

To upgrade from an earlier version of Elasticsearch, see the Elasticsearch upgrade documentation.

Build from source

Elasticsearch uses Gradle for its build system.

To build a distribution for your local OS and print its output location upon completion, run:

./gradlew localDistro

To build a distribution for another platform, run the related command:

./gradlew :distribution:archives:linux-tar:assemble
./gradlew :distribution:archives:darwin-tar:assemble
./gradlew :distribution:archives:windows-zip:assemble

To build distributions for all supported platforms, run:

./gradlew assemble

Distributions are output to distributions/archives.

To run the test suite, see TESTING.

Documentation

For the complete Elasticsearch documentation visit elastic.co.

For information about our documentation processes, see the docs README.

Contribute

For contribution guidelines, see CONTRIBUTING.

Questions? Problems? Suggestions?

• To report a bug or request a feature, create a GitHub Issue. Please ensure someone else hasn’t created an issue for the same topic.

• Need help using Elasticsearch? Reach out on the Elastic Forum or Slack. A fellow community member or Elastic engineer will be happy to help you out.

   from https://github.com/elastic/elasticsearch 

  (https://github.com/elastic/elasticsearch-definitive-guide,

  https://github.com/elasticsearch-cn/elasticsearch-definitive-guide)
  

  ------

  elasticsearch 安装笔记 

  什么是 Elasticsearch

  Elasticsearch是一个基于Lucene库的搜索引擎。它提供了一个分布式、支持多租户的全文搜索引擎，具有HTTP Web接口和无模式JSON文档。Elasticsearch是用Java开发的，并在Apache许可证下作为开源软件发布。官方客户端在Java、.NET（C#）、PHP、Python、Apache Groovy、Ruby和许多其他语言中都是可用的。根据DB-Engines的排名显示，Elasticsearch是最受欢迎的企业搜索引擎，其次是Apache Solr，也是基于Lucene。

  安装 JDK

• Elasticsearch是java应用，故依赖JDK

• yum install -y java-1.8.0-openjdk wget

安装 elasticsearch

• 下载 elasticsearch 并解压，这里创建一个软连接，方便以后切换版本

  
  

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.2.tar.gz wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.2.tar.gz.sha512 shasum -a 512 -c elasticsearch-6.7.2.tar.gz.sha512 tar -xzf elasticsearch-6.7.2.tar.gz -C /usr/share ln -s /usr/share/elasticsearch-6.7.2 /usr/share/elasticsearch

若出现 shasum: command not found 报错信息，安装 perl-Digest-SHA 即可

yum install -y perl-Digest-SHA

创建 elasticsearch 用户及组，为了安全考虑 elasticsearch 不能使用有 root 权限的用户进行启动

groupadd elasticsearch useradd elasticsearch -g elasticsearch -p elasticsearch

创建 elasticsearch 数据目录

mkdir -p /data/elasticsearch/{data,logs}

授予 elasticsearch 用户目录访问权限

chown -R elasticsearch:elasticsearch /usr/share/elasticsearch* chown -R elasticsearch:elasticsearch /data/elasticsearch

动态调整内核的运行参数

echo 'vm.max_map_count=262144' > /usr/lib/sysctl.d/elasticsearch.conf sysctl -p /usr/lib/sysctl.d/elasticsearch.conf

修改相关配置

• 修改 elasticsearch 配置

# vi /usr/share/elasticsearch/config/elasticsearch.yml cluster.name: dev node.name: node1 path.data: /data/elasticsearch/data path.logs: /data/elasticsearch/logs network.host: 0.0.0.0 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.16.227","192.168.16.226","192.168.16.225"] # 集群为3个节点，故最小master数量为 ⌊nodes / 2⌋ + 1 discovery.zen.minimum_master_nodes: 2

修改 JVM 配置

• $ vi /usr/share/elasticsearch/config/jvm.options -Xms4g -Xmx4g

添加 systemd 进程守护文件

$ vi /usr/lib/systemd/system/elasticsearch.service [Unit] Description=Elasticsearch Documentation=http://www.elastic.co Wants=network-online.target After=network-online.target [Service] RuntimeDirectory=elasticsearch PrivateTmp=true Environment=ES_HOME=/usr/share/elasticsearch Environment=PID_DIR=/data/elasticsearch WorkingDirectory=/usr/share/elasticsearch User=elasticsearch Group=elasticsearch ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet Restart=always # StandardOutput is configured to redirect to journalctl since # some error messages may be logged in standard output before # elasticsearch logging system is initialized. Elasticsearch # stores its logs in /var/log/elasticsearch and does not use # journalctl by default. If you also want to enable journalctl # logging, you can simply remove the "quiet" option from ExecStart. StandardOutput=journal StandardError=inherit # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=262144 # Specifies the maximum number of processes LimitNPROC=4096 # Specifies the maximum size of virtual memory LimitAS=infinity # Specifies the maximum file size LimitFSIZE=infinity # Disable timeout logic and wait until process is stopped TimeoutStopSec=0 # SIGTERM signal is used to stop the Java process KillSignal=SIGTERM # Send the signal only to the JVM rather than its control group KillMode=process # Java process is never killed SendSIGKILL=no # When a JVM receives a SIGTERM signal it exits with code 143 SuccessExitStatus=143 [Install] WantedBy=multi-user.target # Built for ${project.name}-${project.version} (${project.name})

设置防火墙

# 开启9200端口 $ firewall-cmd --permanent --zone=public --add-port=9200/tcp # 关闭防火墙(不建议) $ systemctl stop firewalld.service $ systemctl disable firewalld.service # 重启防火墙 $ firewall-cmd --reload # 开启防火墙 $ systemctl start firewalld.service

启动 elasticsearch

• # 设置开机自启 $ systemctl enable elasticsearch # 运行 elasticsearch $ systemctl start elasticsearch

参考文档

• https://github.com/elastic/elasticsearch/blob/master/distribution/packages/src/common/systemd/elasticsearch.service
  

 -----------------------------------------------------------------

Splainer 是开源的 Elatsticsearch 和 Solr 的沙箱，帮助理解搜索结果以及调整搜索设置、参数等因素。

GitHub地址：https://github.com/o19s/splainer

在线使用地址：http://splainer.io/ 

-------------------------------------------

安装ElasticSearch 

由于本次模拟的是服务器不能连接互联网的情况，所以全部安装步骤皆使用 RPM 或 tar 包的方式安装。本文主要记录安装 Elastic Search 的过程。

安装 JRE

首先这套平台是基于 Java 的，所以 Java 运行环境当然是不能少。但因为这上面不涉及 Java 的开发，所以不需要装 JDK，装 JRE 就够了，还能省下一些磁盘空间。我这里选择 JRE8u161。

我这次选择使用 RPM 包安装。

sudo rpm -ivh jre-8u161-linux-x64.rpm

安装完毕后，验证安装是否成功：

# 检验当前用户下是否安装成功 java -version # 检验sudo环境下是否安装成功 sudo java -version

若都输出如下内容则说明安装成功：

java version "1.8.0_161" Java(TM) SE Runtime Environment (build 1.8.0_161-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

至此 Java 环境配置完成

安装 Elastic Search

安装过程

使用 RPM 包安装

直接使用 rpm 命令安装该 RPM 包

sudo rpm --install elasticsearch-6.2.2.rpm

CentOS 7 使用 systemd 管理开机自启动项，而且安装过程已经配置好针对 systemd 的启动脚本，使用如下命令激活

sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service

使用 tar.gz 包安装

首先新建一个名为 elk 的用户，用于运行 ELK 平台

useradd -m elk

下载好 Elastic Search 的安装包，将其复制到 /opt 并解压，然后试运行

sudo cp elasticsearch-6.1.3.tar.gz /opt cd /opt sudo tar xvzf elasticsearch-6.1.3.tar.gz # 需要将Elastic Search目录的所有权设为将要运行该软件的用户 # Elastic Search不允许以root用户运行，安全方面亦不建议以root权限运行程序 sudo chown -R elk:elk elasticsearch-6.1.3 cd elasticsearch-6.1.3/bin ./elelasticsearch

启动成功后，在另一终端使用 curl 尝试连接 Elastic Search

curl http://127.0.0.1:9200

若有如下返回，则说明 Elastic Search 启动成功

{ "name" : "LWmSd17", "cluster_name" : "elasticsearch", "cluster_uuid" : "lkbXufQpQuiLaE5kzVKAeA", "version" : { "number" : "6.1.3", "build_hash" : "af51318", "build_date" : "2018-01-26T18:22:55.523Z", "build_snapshot" : false, "lucene_version" : "7.1.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }

安装后的配置

系统配置文件修改

Elastic Search 需要调整文件描述符大于 65535、最大线程数大于 4096、以及 vm.max_map_count 大于 262144。所以修改操作系统配置文件以满足此要求。

在 /etc/security/limits.conf 插入如下内容

* hard nofile 65536 * soft nofile 65536 * hard nproc 4096 * soft nproc 4096

在 /etc/sysctl.conf 中插入如下内容

vm.max_map_count=262144

然后执行 sysctl -p，并重新登录，使配置生效。若配置成功，则可见 Elastic Search 启动过程中相关的警告信息将不再出现。

Elastic Search 配置文件修改

以下文件位置根据安装方法不同而不同
若使用 RPM 包方式安装，则文件位于 /etc/elasticsearch
若使用 tar 包方式安装，则文件位于解压出来的目录的 conf 文件夹中

• 修改 cluster.name
  我们应当将集群名设置成一个能清晰地表明该集群的作用的名字，如 logging-prod。

• 修改 node.name
  为每个 Elastic Search 节点起一个清晰易懂的名字绝不会是一件坏事。
  节点名字可以是一个自定义的名字，如 prod-data-2，也可以使用 ${HOSTNAME} 来把本机的主机名作为该节点的节点名。

• 其他详细配置
  要想了解更多配置，可以参考 Elasticsearch Reference 的 Set up Elasticsearch 部分。

配置自动启动

如果使用 RPM 包方式安装，则此步可忽略。

若使用 tar 包方式安装，则进入 Elastic Search 的 bin 目录后运行

./elasticsearch -d -p ../logs/elasticsearch.pid

使 Elastic Search 以 daemon 模式启动并监控启动过程。

参考文献

• Elastic Search Installation Guide