This blog post continues where Cloudera and Isilon Implementation Part 1 leaves off:

 

 

In order to deploy a Hadoop Cluster with Isilon, we need to select the Custom Services install, this allows us to select the required components.

 

z.png

 

Initially no services are selected,

aa.png

 

Select the Hadoop services you wish to deploy;

 

DO NOT SELECT HDFS, since we are deploying Isilon as the storage & effective NN we don't need the Cloudera HDFS Service.

 

MapReduce is also not recommended, as MapReduce2 is included in Yarn

 

But, if your application is legacy and not written for Yarn MRv1 can be enabled and used.

bb.png

 

Having selected the services continue and assign roles. Since this is a single host all roles are deployed on the same host. Consult the Cloudera documentation for best practices regarding role assignments.

 

cc.pngdd.pngee.png

 

Leave the Isilon role, as default. No need to change it.

ff.png

 

Continue,

 

gg.png

 

hh.png

 

Again this is test host, so all defaults are selected, consult the Cloudera documentation for best practices regarding database.

Test the Database Connections,

ii.png

 

Continue,

 

On the Cluster Setup page, we need to assign the Isilon to the following two parameters, note the ports are different.

 

default_fs_name        hdfs://smartconnectzonename:8020

webhdfs_url                http://smartconnectzonename:8082/webhdfs/v1

 

jj.pngkk.png

ll.png

mm.png

nn.png

 

Assign the two Isilon paramters

oo.png

 

All other settings can be left as default currently,

 

Continue,

 

Cluster Setup will start,

pp.png

 

Review the setup as it runs,

 

qq.png

 

rr.png

Setup will complete, additional details can be seen by opening the specific services

 

The setup will complete,

ss.png

 

Continue, the Hadoop cluster deployment has finished.

tt.png

uu.png

 

Finish and return to the main Cloudera Manager dashboard, review the status

 

 

 

It is not uncommon to see alarms and service down issues on the dashboard. Review the alarms and services and triage as needed. Some services may just need restarting, follow standard protocols in starting these services.

-start the service

-monitor and review logs as needed

-reviewing the Isilon /var/log/hdfs.log  (remember Isilon is a clustered system, so all node logs need reviewing)

 

123.png

 

 

It is also common to see Configuration issues, address and make the required changes as needed to resolve each issue.

124.png

 

Restart services to resolve Alarms following configuration changes.

 

234.png

 

 

The Hadoop cluster and Services are now fully operational and can be tested.

 

vv.png

 

Basic functionality can be tested.

 

Browse the Isilon HDFS root:

hadoop fs -ls /

ww.png

 

Write to the Isilon HDFS root:

xx.png,

 

Run some basic smoketest jobs; PI or teragen/teravalidate/terasort to test mapreduce.

 

xxx.png

 

 

 

With Cloudera 5.7, you may notice that Impala service is not started fully, some additional configuration changes are needed to get this Service started.

 

345.png

 

 

 

The steps needed to get Impala running can be found here  ---- >  Get Cloudera 5.7 Impala starting with Isilon

 

 

Hopefully this doc gives the high level overview of getting Cloudera CDH up and running against Isilon OneFS.

 

 

 

 

russ_stevenson

Isilon

Using Hadoop with Isilon - Isilon Info Hub