This article walks through the upgrade of a deployment of Hortonworks Data Platform (HDP) using Ambari 2.2 when Isilon OneFS is used as the data storage for HDFS.

 

Choosing an upgrade guide

 

This guide applies to OneFS 7.2.1.3 or higher, OneFS 8.0.0.1 or higher, and later releases of OneFS.

 

There have been two previous articles on this topic. Last November we described how to upgrade HDP, which required manual changes to work. Use that guide if you are stuck on OneFS 7.2.1.1 or lower. In January OneFS was updated, making the process much easier. Use that guide if you are not yet on OneFS 7.2.1.3 or 8.0.0.1, or if you are not ready to upgrade to Ambari 2.2.

 

A table will help illustrate.

 

OneFSAmbariGuide
  • 7.2.1 family, equal or greater than 7.2.1.3
  • 8.0.0 family, equal or greater than 8.0.0.1
  • Any later family
2.2 or higherCurrent page
7.2.1 family equal or greater than 7.2.1.22.1 family. But, not 2.1.2 because of JIRA 13414January 2016 guide
7.2.1.0 (with patch) or 7.2.1.12.1 familyNovember 2015 guide
7.2.0.3 or 7.2.0.42.1.0No upgrade possible to HDP 2.3

 

Ambari 2.2 mainstreamed the previously experimental skip of service checks that we used in the January 2016 guide. Also Ambari 2.2 introduced a new workflow, Express Upgrade, which speeds up the process when there is no problem for your data scientists if you disrupt jobs. This guide describes the improved Rolling Upgrade experience, and what to expect from Express Upgrade.

 

The Upgrade

Getting started

 

OneFS still needs to report to Ambari Server that it is at the target version. This step is unchanged. Here's how I described target version last time:

 

"This is the 4-number release version and 4 digit build number of HDP. The three HDP releases of 2.3 that are available today are 2.3.0.0-2557, 2.3.2.0-2950, and 2.3.4.0-3485. This can be determined from the release notes of the HDP build that you are upgrading to, or running the command 'hadoop version' on an existing deployment."

 

Currently we're at 2.4.2.0-258. Here's how to make the change:

 

isi hdfs settings modify --odp-version=[version] --zone=[zone]

 

Rolling Upgrade

 

Begin Ambari's upgrade steps and complete installation of packages. Click the Perform Upgrade button (it may have changed to a link named Upgrade: In Process; if so, click on this).

 

The Upgrade Options box will open:

 

HDP upgrade dialog.jpg

 

Click the blue link to take a look at list of failed pre-upgrade checks. There will be two Requirements that fail because of the OneFS host configuration. Ambari expects NameNode High Availability enabled, and expects no Secondary NameNode. Those are okay because OneFS provides continuous availability internally. Hadoop's NNHA and Secondary NameNode are not truly applicable. You can ignore these errors.

Ambari upgrade warning and requirements.jpg

 

If the odp-version setting in OneFS was not previously set, this warning may appear. It can be ignored as well. It could also be corrected by setting odp-version on OneFS to the initial HDP version and then restarting the HDFS service in Ambari.


HDP version error Ambari 2-2.jpg


If there are any other required checks or warning messages, resolve those issues before continuing.

 

Back at the Upgrade Options, select the "Skip all Service Check failures" checkbox, then choose Rolling Upgrade and Proceed.

 

Skip Service Checks Ambari 2-2.jpg

 

 

There may be other actions necessary to complete HDP installation, but the OneFS and HDFS aspects will be complete.

 

Express Upgrade

 

No special instructions are needed for HDP upgrade using Express Upgrade with OneFS in the cluster. You may encounter the warning about the HDFS components not having version stamps, but that can be ignored.

 

Work through the HDP upgrade wizard and a satisfying "Upgrade finished" dialog is your reward!

 

Finalize upgrade Ambari 2-2.jpg.png