If Cloudera 5.7 was installed in a process similar to the sequence described in: https://community.emc.com/community/products/isilon/blog/2016/07/07/cloudera-and-isilon-implementation-part-2

 

Then the Impala Service may not start or stay running correctly, a few additional configuration changes will need to be made to get this service running. Also take a look at the Cloudera doc: Using Impala with Isilon Storage

 

 

1.png

 

On further review, the Impala Daemon will not stay started correctly.

2.png

 

The process will continue to exit post start.

3.png

 

 

On reviewing the Impalad log we see the following info and errors.

 

-Short-circuit reads are not enabled

-block tracking is not properly enabled because of dfs.client.file-block-storage-locations.timeout.millis timeout

 

 

5.png

 

 

In order to correct these, from Cloudera Manager search for: dfs_client_read_shortcircuit

 

And select the Isilon : Enable HDFS Short-Circuit Reads

 

6.png

 

1.

Enable HDFS Short-Circuit Read on the Gateway Default Group

 

7.png

 

2.

In HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml and the Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml properties for the Isilon service

 

Add a new value of the dfs.client.file-block-storage-locations.timeout.millis property to 10000

 

hdfs-site.xml

8.png

 

9.png

 

 

core-site.xml

10.png

 

Also add In the Isilon Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml property for the Isilon service

 

3.

Set the value of the hadoop.security.token.service.use_ip property to FALSE.

11.png

 

Save and return to Cloudera Manager Dashboard, since configuration changes have been made we need to redeploy configuration and restart services.

 

12.png

 

Select, to Deploy Client Configuration

13.png

 

 

Deploy,

14.png

 

 

Monitor the deploy

15.png

 

 

We now need to restart the services, affected by the configuration redeployment

 

16.png

 

Restart all the services,

17.png

 

 

Restart,

18.png

 

 

Monitor the restarts,

19.png

 

 

 

You may still see alarms on services based on prior alert triggers

20.png

 

The alarms can be viewed for their status,

21.png

 

 

The alarms will timeout and the status of Impala will return to healthy green.

 

22.png

 

The Impala service is now started and operational.

 

 

 

russ_stevenson

Isilon

Using Hadoop with Isilon - Isilon Info Hub