A couple of weeks back, OneFS 8.2.2 was launched, adding a bevy of new features to the Isilon portfolio. We’ll take a peek at this new functionality over the course of the next few blog articles, kicking it off with large file support.


The largest file size that OneFS currently supports is raised to 16TiB in the new OneFS 8.2.2 release. This is a fourfold increase over previous OneFS versions, up from a maximum of 4TiB in prior releases.

 

16TB_file_1.PNG.png

 

This helps enable additional applications and workloads that typically deal with large files, for example videos & images, seismic analysis workflows, as well as a destination or staging area for backups and large database dumps.


So let’s take a quick look at this new functionality...


Firstly, large file support is available for free. No special license is required to activate large file support in OneFS 8.2.2 and, once enabled, files larger than 4TiB may be written to and/or exist on the system. However, large file support cannot be disabled once enabled.


In order for OneFS 8.2.2 to support files larger than 4TiB, adequate space is required in all of a cluster’s disk pools in order to avoid a potential performance impact. As such, the following requirements must be met in order to enable large file support:

 

 

Large File Support Requirement

Description

Version

A cluster must be running OneFS 8.2.2 in order to enable large file support.

Disk Pool

A maximum sized file (16TiB) plus protection can consume no more than 10% of any disk pool. This translates to a minimum disk pool size of 160TiB plus protection.

SyncIQ Policy

All SyncIQ remote clusters must be running OneFS 8.2.2 and also satisfy the restrictions for minimum disk pool size and SyncIQ policies.

 

Note that the above restrictions will be removed in a future release, allowing support for large (>4TiB) file sizes on all cluster configurations.

The following procedure can be used to configure a cluster for 16TiB file support:

 

16TB_file_2.PNG.png

 

Once a cluster is happily running OneFS 8.2.2, the ‘isi_large_file -c’ CLI utility will verify that the cluster’s disk pools and existing SyncIQ policies meet the requirements listed above. For example:

 

# isi_large_file -c

Checking cluster compatibility with large file support...

 

NOTE:

Isilon requires ALL clusters in your data-center that are part of

any SyncIQ relationship to be running on versions of OneFS compatible

with large file support before any of them can enable it.  If any

cluster requires upgrade to a compatible version, all SyncIQ policies

in a SyncIQ relationship with the upgraded cluster will need to resync

before you can successfully enable large file support.

 

* Checking SyncIQ compatibility...

- SyncIQ compatibility check passed

 

* Checking cluster disk space compatibility...

- The following disk pools do not have enough usable storage capacity to support                   large files:

 

Disk Pool Name    Members Usable  Required  Potential Capable  Add Nodes

-----------------------------------------------------------------------------

h500_30tb_3.2tb-ssd_128gb:2  2-3,6,8,10-11,13-16,18-19:bay3,6,9,12,15   107TB                     180TB      89T N        X

h500_30tb_3.2tb-ssd_128gb:3 2-3,6,8,10-11,13-16,18-19:bay4,7,10,13,16  107TB                     180TB      89T N        X

h500_30tb_3.2tb-ssd_128gb:4 2-3,6,8,10-11,13-16,18-19:bay5,8,11,14,17  107TB                     180TB      89T N        X

h500_30tb_3.2tb-ssd_128gb:9  1,4-5,7,9,12,17,20-24:bay5,7,11-12,17      107TB                     180TB      89T N        X

h500_30tb_3.2tb-ssd_128gb:10 1,4-5,7,9,12,17,20-24:bay4,6,10,13,16 107TB 180TB      89T      N X

h500_30tb_3.2tb-ssd_128gb:11 1,4-5,7,9,12,17,20-24:bay3,8-9,14-15 107TB 180TB      89T      N X

 

The cluster is not compatible with large file support:

  - Incompatible disk pool(s)

 

Here, the output shows that none of the pools meet the 10% disk pool rule above and contain insufficient storage capacity to allow large file support to be enabled. In this case, additional nodes would need to be added.

 

The following table explains the detail of output categories above:

 

Category

Description

Disk Pool Name

Node pool name and this disk pool ID.

Members

Current nodes and bays in this disk pool .

Usable

Current usable capacity of this disk pool.

Required

Usable capacity required for this disk pool to support large files.

Potential

The max usable capacity this disk pool could support at the target node count

Capable

Whether this disk pool has the size of disk and number per node to support large files

Add Nodes

If this disk pool is capable, how many more nodes need to be added

 

Once the validation confirms that the cluster meets the requirements, the following CLI command can then be run to enable large file support:

 

# isi_large_file -e

 

Upon successfully enabling large file support, the ‘cluster full’ alert threshold is automatically lowered to 85% from the OneFS default of 95%. This is to ensure that adequate space is available for large file creation, repair, and restriping. Additionally, any SyncIQ replication partners must also be running OneFS 8.2.2, adhere to the above minimum disk pool size, and have the large file feature enabled.

 

Any disk pool management commands that violate the large file support requirements are not allowed. Once enabled, disk pools are periodically checked for compliance and OneFS will alert if a disk pool fails to meet the minimum size requirement.

 

If Large File Support is enabled on a cluster, any SyncIQ replication policies will only succeed with remote clusters that are also running 8.2.2 and have Large File Support enabled. All other SyncIQ policies will fail until the appropriate remote clusters are upgraded and have large file support switched on.


16TB_file_3.PNG.png

 

Be aware that, once enabled, large file support cannot be disabled on a cluster – regardless of whether it’s a SyncIQ source or target, or not participating in replication. This may impact future expansion planning for the cluster and all of its SyncIQ replication partners.

 

Also note that, after enabling large file support, the ‘cluster full’ alert threshold is automatically lowered to 85% from the default of 95%. This helps ensure that adequate space is available for large file creation, repair, and restriping.


When the maximum file size is exceeded, OneFS typically returns an ‘EFBIG’ error. This is translated to an error message of “File too large”. For example:

 

# dd if=/dev/zero of=16TB_file.txt bs=1 count=2 seek=16384g

dd: 16TB_file.txt: File too large

1+0 records in

0+0 records out

  0 bytes transferred in 0.000232 secs (0 bytes/sec)