There has been considerable interest recently around how to estimate the storage overhead, or the amount of raw disk space, that OneFS will consume for a particular level of protection.

 

For example:

 

If I have a cluster comprising five 108TB nodes, protected with +2d:1n, and containing a dataset with medium and large files, how much general usable capacity I would have available?”

 

The protection policy of +2d:1n on this cluster means that it can survive two simultaneous drive failures or one entire node failure without data loss or unavailability.


The chart below answers such storage overhead questions across a range of OneFS 7.2 protection level options and node counts.

 

For each field in this chart, the storage overhead numbers are calculated by dividing the sum of the two numbers by the number on the right.

 

x+y => y/(x+y)

 

So, for the 5 nodes @ +2d:1n example above, the chart shows that it’s an 8+2 layout (see yellow highlighted field below).

 

8+2 => 2/(8+2) = 20%

 

 

Number of nodes

[+1n]

[+2d:1n]

[+2n]

[+3d:1n]

[+3d:1n1d]

[+3n]

[+4d:1n]

[+4d:2n]

[+4n]

3

2 +1 (33%)

4 + 2 (33%)

6 + 3 (33%)

3 + 3 (50%)

8 + 4 (33%)

4

3 +1 (25%)

6 + 2 (25%)

9 + 3 (25%)

5 + 3 (38%)

12 + 4 (25%)

4 + 4 (50%)

5

4 +1 (20%)

8+ 2 (20%)

3 + 2 (40%)

12 + 3 (20%)

7 + 3 (30%)

16 + 4 (20%)

6 + 4 (40%)

6

5 +1 (17%)

10 + 2 (17%)

4 + 2 (33%)

15 + 3 (17%)

9 + 3 (25%)

16 + 4 (20%)

8 + 4 (33%)

  

These x+y numbers in each field in the table also represent how files are striped across a cluster for each node count and protection level.

 

For example, with +2d:1n protection on a 5-node cluster, OneFS will write a double stripe across all 5 nodes (total of 10 stripe units), and use two of these stripe units for parity/ECC and eight for data. This is illustrated in the following diagram:


 

protection_layout_dwg.jpg

For OneFS 7.2, the general protection overhead will look something like the percentages in the table below.


Note: The estimated protection overhead percentage (% in brackets) is a very rough guide and will vary considerably across different datasets, depending on quantities of small files, etc.

  

Number of nodes

[+1n]

[+2d:1n]

[+2n]

[+3d:1n]

[+3d:1n1d]

[+3n]

[+4d:1n]

[+4d:2n]

[+4n]

3

2 +1 (33%)

4 + 2 (33%)

6 + 3 (33%)

3 + 3 (50%)

8 + 4 (33%)

4

3 +1 (25%)

6 + 2 (25%)

9 + 3 (25%)

5 + 3 (38%)

12 + 4 (25%)

4 + 4 (50%)

5

4 +1 (20%)

8 + 2 (20%)

3 + 2 (40%)

12 + 3 (20%)

7 + 3 (30%)

16 + 4 (20%)

6 + 4 (40%)

6

5 +1 (17%)

10 + 2 (17%)

4 + 2 (33%)

15 + 3 (17%)

9 + 3 (25%)

16 + 4 (20%)

8 + 4 (33%)

7

6 +1 (14%)

12 + 2 (14%)

5 + 2 (29%)

15 + 3 (17%)

11 + 3 (21%)

4 + 3 (43%)

16 + 4 (20%)

10 + 4 (29%)

8

7 +1 (13%)

14 + 2 (12.5%)

6 + 2 (25%)

15 + 3 (17%)

13 + 3 (19%)

5 + 3 (38%)

16 + 4 (20%)

12 + 4 (25%)

9

8 +1 (11%)

16 + 2 (11%)

7 + 2 (22%)

15 + 3 (17%)

15 + 3 (17%)

6 + 3 (33%)

16 + 4 (20%)

14 + 4 (22%)

5 + 4 (44%)

10

9 +1 (10%)

16 + 2 (11%)

8 + 2 (20%)

15 + 3 (17%)

15 + 3 (17%)

7 + 3 (30%)

16 + 4 (20%)

16 + 4 (20%)

6 + 4 (40%)

12

11 +1 (8%)

16 + 2 (11%)

10 + 2 (17%)

15 + 3 (17%)

15 + 3 (17%)

9 + 3 (25%)

16 + 4 (20%)

16 + 4 (20%)

6 + 4 (40%)

14

13 +1 (7%)

16 + 2 (11%)

12 + 2 (14%)

15 + 3 (17%)

15 + 3 (17%)

11 + 3 (21%)

16 + 4 (20%)

16 + 4 (20%)

10 + 4 (29%)

16

15 +1 (6%)

16 + 2 (11%)

14 + 2 (13%)

15 + 3 (17%)

15 + 3 (17%)

13 + 3 (19%)

16 + 4 (20%)

16 + 4 (20%)

12 + 4 (25%)

18

16 +1 (6%)

16 + 2 (11%)

16 + 2 (11%)

15 + 3 (17%)

15 + 3 (17%)

15 + 3 (17%)

16 + 4 (20%)

16 + 4 (20%)

14 + 4 (22%)

20

16 +1 (6%)

16 + 2 (11%)

16 + 2 (11%)

16 + 3 (16%)

16 + 3 (16%)

16 + 3 (16%)

16 + 4 (20%)

16 + 4 (20%)

14 + 4 (22%)

30

16 +1 (6%)

16 + 2 (11%)

16 + 2 (11%)

16 + 3 (16%)

16 + 3 (16%)

16 + 3 (16%)

16 + 4 (20%)

16 + 4 (20%)

14 + 4 (22%)