In order to deploy a Hadoop Cluster with Isilon, we need to select the Custom Services install, this allows us to select the required components.
Initially no services are selected,
Select the Hadoop services you wish to deploy;
DO NOT SELECT HDFS, since we are deploying Isilon as the storage & effective NN we don't need the Cloudera HDFS Service.
MapReduce is also not recommended, as MapReduce2 is included in Yarn
But, if your application is legacy and not written for Yarn MRv1 can be enabled and used.
Having selected the services continue and assign roles. Since this is a single host all roles are deployed on the same host. Consult the Cloudera documentation for best practices regarding role assignments.
Leave the Isilon role, as default. No need to change it.
Again this is test host, so all defaults are selected, consult the Cloudera documentation for best practices regarding database.
Test the Database Connections,
On the Cluster Setup page, we need to assign the Isilon to the following two parameters, note the ports are different.
All other settings can be left as default currently,
Cluster Setup will start,
Review the setup as it runs,
Setup will complete, additional details can be seen by opening the specific services
The setup will complete,
Continue, the Hadoop cluster deployment has finished.
Finish and return to the main Cloudera Manager dashboard, review the status
It is not uncommon to see alarms and service down issues on the dashboard. Review the alarms and services and triage as needed. Some services may just need restarting, follow standard protocols in starting these services.
-start the service
-monitor and review logs as needed
-reviewing the Isilon /var/log/hdfs.log (remember Isilon is a clustered system, so all node logs need reviewing)
It is also common to see Configuration issues, address and make the required changes as needed to resolve each issue.
Restart services to resolve Alarms following configuration changes.
The Hadoop cluster and Services are now fully operational and can be tested.
Basic functionality can be tested.
Browse the Isilon HDFS root:
hadoop fs -ls /
Write to the Isilon HDFS root:
Run some basic smoketest jobs; PI or teragen/teravalidate/terasort to test mapreduce.
With Cloudera 5.7, you may notice that Impala service is not started fully, some additional configuration changes are needed to get this Service started.