There have been a couple of recent inquiries from the field around SMB opportunistic locking so it seemed like an appropriate topic to dig into a bit in an article.
Under certain conditions, opportunistic locks, or oplocks, can enable a storage device and client to aggressively cache data – helping to boost performance. More specifically, oplocks allow a Windows client to cache read-ahead data, writes, opens, closes, and byte-range lock acquisitions.
With SMB2.1 and onward, in addition to oplocks, Microsoft also introduced the concept of leases. These provide more flexible and granular caching for clients, while also allowing for lock upgrades.
Here’s a brief rundown on how SMB and NFS support locks and leases:
Oplocks are defined and used in the SMB1 protocol. These are fully supported in OneFS.
Oplocks are still supported but leases are also included in the protocol. These offer a number of improvements over oplocks. These are fully supported in OneFS.
No provision in the protocol for leases or oplocks.
Optional support for file and directory delegations which are similar to SMB leases. These are not currently supported by OneFS.
When a Windows client attempts to open a file, it can request no oplock, request a batch, or request an exclusive oplock. Once the open has passed the access and share mode checks, OneFS must do one of the following:
1) Grant the client its requested oplock on the file (exclusive or batch).
2) Grant the client a lower-level (level II) oplock on the file.
3) Not grant an oplock on the file at all.
The various oplocks types, ranked from the least to the most amount of caching, include:
Level II (shared)
Level II oplocks, also referred to as shared oplocks, grant clients the ability to cache the results of read operations. This means a client can prefetch data that an application may want to read, as well as retain old read data, allowing its reads to be more efficient. Multiple clients can hold level II oplocks at the same time, but all existing level II oplocks are broken when a client tries to write data to the file.
Exclusive oplocks grant clients the ability to retain read data, like level II oplocks, but also allow clients to cache data and metadata writes and byte-range lock acquisitions. Unlike level II oplocks, a client cannot be granted an exclusive oplock if the file is already opened. If a client is granted an exclusive oplock, it is able to cache writes to the file, cache metadata changes (such as timestamps, but not ACLs) and cache range locks of the file via byte-range locking. As soon as there is another opener, either from the same client or a different client, the cluster must request to break the exclusive oplock, in order to guarantee the second opener has access to the most up-to-date data.
Batch oplocks are identical to exclusive oplocks, except that they allow clients to cache open/close operations. The origins of this type of oplock are from the days of DOS batch files; batch files were opened and closed for every line of the script to be executed.
There are two types of oplock breaks: level I breaks and level II breaks. An oplock break occurs when an oplock is contended, due to a conflicting file operation. In OneFS, contention occurs when an operation on one File ID (a File ID, or FID, is the ‘handle’ that SMB uses to refer to a file) conflicts with a currently held oplock on a different FID, either on the same or a different client. When an oplock contends with an operation, the oplock is broken. The OneFS rules governing oplock contention are:
A level II oplock contends with modifying operations, such as writes and truncates, as well as byte-range lock acquisitions.
An exclusive oplock contends with an open operation, except for stat-only opens.
A batch oplock contends with an open, delete, or rename operation.
Contention can occur if the operations are from either the same or a different Windows client. However, an operation on a file ID does not contend against the FID’s own oplock since FIDs must be different to contend. However, opening the same file a second time will typically contend with the first opening of the file, since the second opening will return a different FID.
The two level I oplocks, exclusive and batch, are broken in different ways. An exclusive oplock is broken when the file it pertains to has been requested for opening. Batch oplocks are broken when the same file is opened from a different client or when the file is deleted or renamed.
When OneFS needs to break a level I oplock, it allows the client a chance to perform all of the operations that it has cached. Before the cluster can respond to the open request from the second client, it waits for an acknowledgment of the oplock break from the first client. The first client now has the chance to either flush cached metadata or data or send byte-range locking requests.
Once the client has flushed its cached operations, it relinquishes its oplock either by closing the file or acknowledging to the cluster that it has downgraded its oplock. When a client decides to downgrade its oplock it either accepts a level II oplock or informs the cluster that it does not require an oplock at all. After the client has acknowledged the oplock break, OneFS is free to respond to the open request from the second client. It may also give the second client a level II oplock, allowing it to cache read data. Since OneFS waits up to 30 seconds for an acknowledgment of its oplock break request, after which it considers the client unresponsive and times out.
A FID’s level II oplock is broken when a modifying operation or a byte range lock acquisition is performed on a different FID. The cluster informs the first FID that its oplock has been broken and that it can no longer cache read data. Unlike the exclusive oplock break, OneFS does require an oplock break acknowledgment from the client and can continue processing the write request right away.
Leases are similar to, and compatible with, oplocks, but superior in a number of areas:
- Leases contend based on a client key, rather than a FID, so two different applications on a client accessing the same file can share a lease whereas they cannot share an oplock.
- There are more lease types, namely:
A Read lease (shared) indicates that there are multiple readers of a stream and no writers. This supports client read caching (similar to Level II oplock).
A Read-Handle lease (shared) indicates that there are multiple readers of a stream, no writers, and that a client can keep a stream open on the cluster even though the local user on the client machine has closed the stream. This supports client read caching and handles caching (level II plus handle caching).
A Read-Write lease (exclusive) allows a client to open a stream for exclusive access and allows the client to perform arbitrary buffering. This supports client read caching and write caching (Level I Exclusive).
Read-Write-Handle (RWH) lease (exclusive) allows a client to keep a stream open on the cluster even though the local accessor on the client machine has closed the stream. This supports client read caching, write caching, and handle caching (Level I Batch).
To globally enable oplocks on a cluster’s shares from the WebUI, navigate to Protocols > Windows Sharing (SMB) > Default Share Settings > Advanced Settings > Oplocks and check the ‘Enable oplocks’ checkbox.
The same can be done for individual shares, from the desired share’s advanced configuration menu.
From the CLI, the syntax to enable oplocks on a share named ‘test’ is:
# isi smb shares modify test --oplocks Yes
To verify the configuration:
# isi smb shares view test | grep -i oplocks
Similarly, the syntax to disable oplocks on the ‘test’ share is:
# isi smb shares modify test --oplocks No
To re-enable oplocks, the following command can be used:
# isi smb shares modify test --revert-oplocks
The following gconfig syntax can be used to disable leases:
# isi_gconfig registry.Services.lwio.Parameters.Drivers.srv.smb2.EnableLeases=0
Note that the above oplocks configuration is unaffected by this config change to leases.
Similarly, to re-enable leases, the following command can be used:
# isi_gconfig registry.Services.lwio.Parameters.Drivers.srv.smb2.EnableLeases=1