Elastic Search is a popular open source search server that is used for real-time distributed search and analysis of data. When used for anything other than development, Elastic Search should be deployed across multiple servers as a cluster, for the best performance, stability, and scalability.
Demonstration:
OMegha Platform.
Image – Ubuntu-14.04
Prerequisites:
You must have at least three Ubuntu 14.04 servers to complete this, because an Elastic Search cluster should have a minimum of 3 master-eligible nodes. If you want to have dedicated master and data nodes, you will need at least 3 servers for your master nodes plus additional servers for your data nodes.
Install Java 8:
Elastic Search requires Java, so we will install that now. We will install a recent version of Oracle Java 8 because that is what Elastic Search recommends. It should, however, work fine with OpenJDK, if you decide to go that route.
Complete this step on all of your Elastic Search servers.
Add the Oracle Java PPA to apt:
$ sudo add-apt-repository -y ppa:webupd8team/java
Update your apt package database:
$ sudo apt-get update
Install the latest stable version of Oracle Java 8 with this command (and accept the license agreement that pops up):
$ sudo apt-get -y install oracle-java8-installer
Be sure to repeat this step on all of your Elastic Search servers.
Now that Java 8 is installed, let’s install Elastic Search.
Install Elastic Search:
Elastic Search can be downloaded directly from elastic.co in zip, tar.gz, deb, or rpm packages. For Ubuntu, it’s best to use the deb (Debian) package which will install everything you need to run Elastic Search.
$ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.2.deb
Then install it in the usual Ubuntu way with the dpkg command like this:
$ sudo dpkg -i elasticsearch-1.7.2.deb
This results in Elastic Search being installed in /usr/share/elastic Search/ with its configuration files placed in /etc/elastic Search and its init script added in /etc/init.d/elastic search.
$ sudo update-rc.d elasticsearch defaults
Be sure to repeat these steps on all of your Elastic Search servers.
Elastic Search is now installed but it needs to be configured before you can use it.
Configure Elastic search Cluster
Now it’s time to edit the Elastic search configuration. Complete these steps on all of your Elastic search servers.
Open the Elastic search configuration file for editing:
$ sudo vi /etc/elasticsearch/elasticsearch.yml
Set Cluster Name:
Next, set the name of your cluster, which will allow your Elastic search nodes to join and form the cluster. You will want to use a descriptive name that is unique (within your network).
Find the line that specifies cluster.name, uncomment it, and replace its value with the your desired cluster name. In this tutorial, we will name our cluster “elastic search_cluster”:
Set Node Name:
Next, we will set the name of each node. This should be a descriptive name that is unique within the cluster.
Find the line that specifies node.name, uncomment it, and replace its value with your desired node name. In this tutorial, we will set each node name to the host name of server by using the ${HOSTNAME}environment variable:
For Master Node:
For Master set the node.master as True and for node.data as False
For Data Node:
For Data set the node.master as False and for node.data as True
Network Host:
Set the network host as 0.0.0.0
Set Discovery Hosts:
Next, you will need to configure an initial list of nodes that will be contacted to discover and form a cluster. This is necessary in a unicast network.
Find the line that specifies discovery.zen.ping.unicast.hosts and uncomment it.
For example, if you have three servers node01, node02, and node03 with respective VPN IP addresses of 10.0.0.1, 10.0.0.2, and 10.0.0.3, you could use this line:
Save and Exit.
Your servers are now configured to form a basic Elastic search cluster. There are more settings that you will want to update, but we’ll get to those after we verify that the cluster is working.
Save and exit elasticsearch.yml.
Start Elastic search:
Now start Elastic search:
$ sudo service elasticsearch restart
Then run this command to start Elastic search on boot up:
$ sudo update-rc.d elasticsearch defaults 95 10
Be sure to repeat these steps (Configure Elastic search) on all of your Elastic search servers.
Testing:
By now, Elastic search should be running on port 9200. You can test it with curl, the command line client-side URL transfers tool and a simple GET request like this:
$ curl -X GET 'http://localhost:9200'
You should see the following response:
If you see a response similar to the one above, Elastic search is working properly. If not, make sure that you have followed correctly the installation instructions and you have allowed some time for Elastic search to fully start.
Check Cluster State:
If everything was configured correctly, your Elastic search cluster should be up and running. Before moving on, let’s verify that it’s working properly. You can do so by querying Elastic search from any of the Elastic search nodes.
From any of your Elastic search servers, run this command to print the state of the cluster:
$ curl -XGET 'http://localhost:9200/_cluster/state?pretty'
If you see output that is similar to this, your Elastic search cluster is running! If any of your nodes are missing, review the configuration for the node(s) in question before moving on.
Next, we’ll go over some configuration settings that you should consider for your Elastic search cluster.
Enable Memory Locking:
Elastic recommends to avoid swapping the Elastic search process at all costs, due to its negative effects on performance and stability. One way avoid excessive swapping is to configure Elastic search to lock the memory that it needs.
Complete this step on all of your Elastic search servers.
Edit the Elastic search configuration:
$ sudo vi /etc/elasticsearch/elasticsearch.yml
Find the line that specifies bootstrap.mlockall and uncomment it:
Save and exit.
Now restart Elastic search to put the changes into place:
$ sudo service elasticsearch restart
Cluster Health:
This API can be used to see general info on the cluster and gauge its health:
$ curl -XGET 'localhost:9200/_cluster/health?pretty'
Cluster State:
This API can be sued to see a detailed status report on your entire cluster. You can filter results by specifying parameters in the call URL.
$ curl -XGET 'localhost:9200/_cluster/state?pretty'
Conclusion:
Your Elastic search cluster should be running in a healthy state, and configured with some basic optimizations.