This blog post is the third in a series covering basic multiprotocol concepts in OneFS. Here, we’ll expand on how OneFS determines a user’s authoritative identity in a multiprotocol environment: the on-disk identity.


See Multiprotocol Concept Series Part 1: Overview for a high-level introduction to multiprotocol data access concepts.


The cornerstone of Isilon’s multiprotocol permissions approach is the unified permissions model. OneFS combines all user identities from all directory services into a single entity and evaluates file permissions against a standard permissions model. This model is different from other network-attached storage (NAS) systems that handle multiprotocol permissions with user mapping or separate permissions.


Let’s review the relationship between authentication, identity management, and authorization (AIMA) functionality and multiprotocol elements. On-disk identity is key to AIMA and to multiprotocol data access.




Any single user can have different identities on different directory services. OneFS uses the user mapper, ID mapping, and on-disk identity to generate a unified access token that represents a user’s persona to the Isilon cluster. ID mapping records how a particular identity’s SID and UID/GID relate to each other. See Multiprotocol Concepts Series Part 2: Access Tokens, User Mapping, and ID Mapping for details about how access tokens, user mapping, and ID mapping works.


The importance of on-disk identity

The OneFS file system supports UIDs, GIDs, and SIDs, but it stores only a single type of identity on disk. As you can see in the following sample user access token, each identity contains both an SID and UID/GID. The reciprocal lookup of these identities to each other is handled by ID mapping, and the persistent mappings are stored in the ID mapping database on the Isilon cluster.



(To see a larger version, click the screen capture.)

Because OneFS stores only a single type of identity on disk and a user’s access token always contains a reciprocal mapping of any identity, file access can always be evaluated against a UID/GID or an SID. So why is on-disk identity so important?


The stored on-disk identity can potentially affect the portability, scalability, and flexibility of the files on the file system. Suppose that a user writes a local identity such as a local OneFS-created UID and GID on a file and then copies that file to a different file system. The file’s metadata, including owner, permissions, and Access Control Lists (ACLs) copied with the file will contain references to the original, local accounts from the source cluster, not the account IDs from a (centrally managed) directory service. This situration will cause issues and should be avoided.


The goal of on-disk identity is to place an identity that is stored and maintained in a directory service. This way, if you write a directory service real identity on disk, any system that’s joined to that directory service has access to those identities and can evaluate the metadata. This ensures that the system can enumerate all real identities and permissions consistently and enforce them in a standard way across storage platforms. For example, a real SID from Windows Active Directory is accessible to any system that’s joined to that Active Directory. Isilon’s on-disk identity setting determines which of the user’s identities OneFS will store on-disk when a user writes a file. Understanding this is critical for maintaining consistent access across the cluster.


Identity modes

Isilon has three modes for choosing the identity that OneFS stores on disk:

  • Native: The cluster makes the appropriate choice based on ID mapping (Default)
  • UID: Stores only UNIX identities (UIDs and GIDs)
  • SID: Stores only Windows identities (SIDs)

Native is the default. Native mode means that the cluster will store a directory service identity that the cluster determines is the most appropriate based on ID mapping. The cluster never stores a local identity on disk if a directory service identity exists.


Important: Do not change the default (native) mode unless you have a very, very good reason and you fully understand the implications of changing it. Changing the on-disk mode is not likely to fix actual file permission issues.

Real and generated identities

OneFS requires that all identities have Windows and UNIX identities (SIDs and UNIX UID/GIDs). Every user access token contains the SID and UID/GIDs that it obtains from the ID mapping performed by OneFS – even if the cluster has a single directory service/single protocol environment. In that environment, OneFS can obtain an identity only from the single directory service provider. So what does OneFS do about the “missing” identity?


To meet the requirement that all identities have an SID and UID/GIDs, even in a single directory service/single protocol environment, OneFS generates an internal identity. This means that, in a single directory service/single protocol environment, it’s very likely that either the SID or the UID/GIDs are real and that one is an internal identity generated by OneFS. That internally-generated identity enables OneFS to complete the ID mapping and to ensure that all identities have the required reciprocal, matching SID and UID/GIDs.


There are three primary possible environments:

  1. Clusters joined only to Active Directory. There’s a real SID and OneFS generates the equivalent internal UID/GIDs.
  2. Clusters joined only to LDAP. The UID/GIDs is real and OneFS generates the equivalent internal SID.
  3. Clusters are joined to both Active Directory and LDAP. This is the most likely multiprotocol environment. There’s a real SID from Active Directory, and a real UID/GIDs from LDAP for mapped users.

Scenario: Clusters joined only to Active Directory

Suppose that the cluster is only joined to Active Directory. OneFS can get a real user SID from Active Directory, but because OneFS is UNIX-based, it still needs a matching user UID for that SID. To meet the requirement to have reciprocal, matching identities, OneFS will generate a UID with a value in the default range of one to two million, map the generated UID to the real SID, and then store this mapping in the ID mapping database. It does the same for group SIDs with GIDs.

Scenario: Clusters joined only to LDAP

Suppose that the cluster is joined only to LDAP. OneFS can get real UID/GIDs from LDAP, but cannot get SIDs from valid Active Directory credentials. So OneFS generates SIDs internally. OneFS has an equivalent internal range for generating SIDs for LDAP-only environments.

Scenario: Clusters joined to Active Directory and to LDAP

If the cluster is joined to both Active Directory and LDAP, OneFS can look up a user’s real SID and real UID/GIDs directly from each directory service for mapped users, and explicitly map those identities to each other. Both identities are real ("real – real" mapping).

Choosing the authoritative identity

In Native On-disk mode, where the cluster decides which identity to use for on-disk identity, the cluster always tries to set the most appropriate, valid identity as the on-disk identity on the file. If the SID is real, OneFS stores the SID on disk. If the UID is real, OneFS stores the UID on disk. If the SID and the UID are real, OneFS stores the UID as the on-disk identity. That’s because OneFS is a UNIX file system under the hood, and it’s slightly faster to use UIDs on disk. Ultimately, it doesn’t matter how the ID is stored. The goal is always to get a real ID on disk.


The following diagram shows the three environments and, for each environment, the identity that the cluster selects to becomes the on-disk identity in Native mode.



The other two on-disk modes of the cluster are much more straightforward and you can see the impact of these modes on getting a real or internal identity written to disk :


  • SID On-disk:
  • Active Directory only - On-disk SID (real)
  • LDAP Only - On-disk SID (internal)
  • Active Directory + LDAP - On-disk SID (real)


  • UNIX On-disk
    • Active Directory only - On-disk UID (internal)
    • LDAP Only - On-disk UID (real)
    • Active Directory + LDAP -  On-disk UID (real)


You can see how the selected on-disk mode of the cluster can affect which identity is placed on disk and how you can potentially end up with the  less preferred (internal) identity on disk if you select the incorrect mode. We recommend the default Native mode setting unless you have good reason to change it and you understand the implications. If you do decide to change the default Native mode, be sure to test to validate the behavior before you move into production.


Note: On-disk mode is a cluster wide setting only. You cannot set it at any other level within the file system.


The following screenshot shows how to set the On-Disk identity on the cluster WebUI.


Incorrect or invalid on-disk identities can be fixed using the OneFS permission repair job. We'll provide more details about that in a future post.

Will the real identity please stand up?

You can examine a user access token to see the on-disk identity selected for each user. The isi auth mapping token command dumps the token:

#isi auth mapping token --user=<user> --zone=<zone>


Let’s look at the access token for the foo domains administrator user. In this example, the cluster’s on-disk mode is in Native mode.



(To see a larger version, click the screen capture.)


What’s in this token:

  • The highlighted entry, On Disk, is the authoritative, on-disk identity for this user.
  • The real SID from Windows. The 500 at the end indicates an administrative account in this case.
  • The cluster is not joined to LDAP, so OneFS generates an internal ID of one million.

The real identity is the SID, so that’s the one that becomes the native on-disk identity.  You can see that the values for On Disk and user’s SID are identical.

Viewing the on-disk identity for a file

OneFS employs the user access token and ID mapping of the users and groups to determine the on-disk identity in order to write on a file during file creation or whenever the file’s permissions are modified. The on-disk identity written on the file constitutes the file’s metadata: only one of the identities is written on the file. This is why ID mapping and storing the appropriate on-disk identity is critical to maintaining the consistency and portability of file access permissions.


You can determine the on-disk identity that is written for a particular file by examining its file permissions.

  • ls -le shows the extended file permissions with user and group names

You need to add the -len switches to view the actual on-disk identities written SID or UID/GIDs

  • ls -len shows the actual SID and UID/GIDs written to disk


We’ll explore these commands further in an upcoming blog post.


Here’s the output for a UNIX file called posix2.txt, and a Windows file called acl2.txt.


(To see a larger version, click the screen capture.)

In the UNIX On-Disk example, the file has posix permissions. You can see that the posix permissions, owner, and the group are represented with UID/GIDs - both 0 in this case - representing root and wheel. The synthetic generated access control list (ACL) that is used for SMB access also contains UID/GIDs, but this isn’t an issue due to valid reciprocal mappings in the user token.  In the SID On-Disk example, this file has real ACLs. Look at the ACL values on the file. You can see that OneFS is writing SIDs on-disk. We’ll explore this further in a future post.


You can see how Isilon OneFS stores only one identity on disk. OneFS evaluates that single identity by following the multiprotocol story, top to bottom: Directory services lead into user mapping, user mapping ties into ID mapping, that intersection enables token creation, tokens define reciprocal mappings, and reciprocal mappings enable permissions to be evaluated correctly and the correct identity to be written to disk.



Here are some highlights:

  • On-disk identity is the authoritative identifier for a user or group, which will be stored on disk.
  • The cluster has three modes for on-disk identity: SID, which always writes only SIDs to disk; UNIX, which always writes UIDs/GIDs; and Native, the default, which selects the most appropriate identity. We recommend that you do not change the default mode. If you do, you must first fully understand why and the implications of making the change. Changing the on-disk mode is not likely to fix actual file permission issues.
  • OneFS stores only a single entity on disk. The ID mappings make reciprocal conversion of identities possible, when needed. You should store only real directory service identities on disk because internally generated identities can present issues.
  • You can dump the access token to see a user’s on-disk identity.
  • Use the command ls -len to see the actual on-disk identity that is written on a particular file and to see how it’s implemented.



Coming soon

The next blog post in this series will describe how OneFS checks a user’s file access.


Additional posts will address how OneFS stores file and share permissions, POSIX permissions, access control list (ACL) policies, and how to check permissions. The following common multiprotocol commands will also be covered in more detail:

-ls –le/ls -led

-ls –len/ls -lend



Let us know

Tell us what you think of this article. Was this level of information useful? Do you have questions that you would like us to cover in future blog posts? Let us know by leaving a comment.