by Alex McLintock and Alan Duval of Alephant.co.uk, Sept 2017
The purpose of this page is to provide you with some basic tests to confirm that the Cloudera Distribution of Hadoop (CDH) is installed and functioning to some extent. It is NOT a full benchmark test suite.
As such, we presume that you have completed your own install, possibly by following our previous post, Doing the Install: Cloudera CDH5.12 on CentOS7.3
Cloudera Manager Admin Console
To test the success of your install, start off by accessing the Cloudera Manager Admin Console.
To do this type, the following in to your web browser:
Where servername is the name you've given to the server that Cloudera Manager Admin Console is installed on. You will need to log in as the admin user created previously.
If you can't access the console at all then either it is not running, or you may have a network issue. Solving that is left as an exercise for the reader.
Checking for "Good Health"
When the console pops up, you'll see a list of the services that are running and hopefully, to the left of these, a green circle with a tick next to each service, indicating that it is running as expected.
If, however, there are other indicators instead you may need to investigate resolving the issues. These maybe a solid orange circle, or a red circle with an exclamation mark inside.
To the right of the service name you will see orange or red circles, both with exclamations inside, and to the right of these there maybe a spanner icon.
Note - sometimes the spanner icon will appear with no exclamation between it and the service name (see Hive in the image below).
The spanner icon indicates that there is a configuration issue to be resolved.
For the most part, if there is an error or configuration issue icon to the right, this will cause the icon to the left to reflect this. There are, however, exceptions:
- In the case of Hive in the image above, there are two configuration issues, but they are minor, so Hive has the green tick of Good Health.
- In the case if Zookeeper in the image above, there are no errors or configuration issues, but the health of the service is of concern. This may mean that the service's warning threshold is set very conservatively, whilst the service itself is within normal parameters, or it may be the first in a cascade of issues - so it's best to find out which.
Clicking on the exclamation mark to the right of the server (with a number denoting the total number of issues) will bring up a 'Health Issues" dialog (see image below), with links to logs, a link to the relevant Cloudera Manager page for the faulty service, etc.
Clicking on the exclamation mark to the left of the service will take you straight to the Cloudera Manager page for the faulty service (below).
Likewise, clicking on the solid orange circle to the left of the service name will take you straight to the Cloudera Manager page for the service of concern (below).
In the Cloudera Manager Admin Console select Hosts tab, then select All Hosts.
The sixth column from the left is headed Last Heartbeat, and the value displayed should be less than 15 seconds.
Note - this test doesn't re-poll while you watch, so you will need to reload the page, or re-select Hosts - All Hosts if you have this page open and want to re-check your cluster's status after taking some action.
Running a MapReduce Job
Log into a single machine in the cluster. We went back to Hosts > All Hosts and confirmed the machines with most roles (which were machines 1 and 2), and ran the following command on both.
We viewed the the job by selecting Clusters > yarn Applications - the relevant output looked like this:
Note - Cloudera's instructions suggest selecting Clusters > ClusterName > yarn Applications. This is a little confusing as you don't select 'ClusterName' (which is to say, the imaginatively titled 'Cluster 1', in the image below). The yarn Application link is listed under Cluster 1 as an available selection (in the right hand column of the image below). If you do click on Cluster 1, you go to the cluster summary, and there is no YARN Applications link there.
Cloudera Manager Health Tests
The basic smoke tests are a bit like holding up a mirror to someone's mouth to see if they are still breathing. If you want to investigate each individual service in more detail then you run Cloudera Manager Health Tests for each service. You can continue reading the documentation for those on the Cloudera website.
Here are the links for the individual health tests listed in the above URL:
- Active Database Health Tests
- Active Key Trustee Server Health Tests
- Activity Monitor Health Tests
- Alert Publisher Health Tests
- Beeswax Server Health Tests
- Cloudera Management Service Health Tests
- DataNode Health Tests
- Event Server Health Tests
- Failover Controller Health Tests
- Flume Health Tests
- Flume Agent Health Tests
- Garbage Collector Health Tests
- HBase Health Tests
- HBase REST Server Health Tests
- HBase Thrift Server Health Tests
- HDFS Health Tests
- History Server Health Tests
- Hive Health Tests
- Hive Metastore Server Health Tests
- HiveServer2 Health Tests
- Host Health Tests
- Host Monitor Health Tests
- HttpFS Health Tests
- Hue Health Tests
- Hue Server Health Tests
- Impala Health Tests
- Impala Catalog Server Health Tests
- Impala Daemon Health Tests
- Impala Llama ApplicationMaster Health Tests
- Impala StateStore Health Tests
- JobHistory Server Health Tests
- JobTracker Health Tests
- JournalNode Health Tests
- Kafka Broker Health Tests
- Kafka MirrorMaker Health Tests
- Kerberos Ticket Renewer Health Tests
- Key Management Server Health Tests
- Key Management Server Proxy Health Tests
- Key-Value Store Indexer Health Tests
- Lily HBase Indexer Health Tests
- Load Balancer Health Tests
- Logger Health Tests
- MapReduce Health Tests
- Master Health Tests
- Monitor Health Tests
- NFS Gateway Health Tests
- NameNode Health Tests
- Navigator Audit Server Health Tests
- Navigator Luna KMS Metastore Health Tests
- Navigator Luna KMS Proxy Health Tests
- Navigator Metadata Server Health Tests
- Navigator Thales KMS Metastore Health Tests
- Navigator Thales KMS Proxy Health Tests
- NodeManager Health Tests
- Oozie Health Tests
- Oozie Server Health Tests
- Passive Database Health Tests
- Passive Key Trustee Server Health Tests
- RegionServer Health Tests
- Reports Manager Health Tests
- ResourceManager Health Tests
- SecondaryNameNode Health Tests
- Sentry Health Tests
- Sentry Server Health Tests
- Service Monitor Health Tests
- Solr Health Tests
- Solr Server Health Tests
- Sqoop 2 Health Tests
- Sqoop 2 Server Health Tests
- Tablet Server Health Tests
- TaskTracker Health Tests
- Tracer Health Tests
- WebHCat Server Health Tests
- Worker Health Tests
- YARN (MR2 Included) Health Tests
- ZooKeeper Health Tests
- ZooKeeper Server Health Tests