Bitly URL:


Tweet this document:

New #XtremIO blog: Using Adv. Format 4 #Oracle databases. Great overview sure 2 improve the performance of database


Related content:

EMC XtremIO Snapshots are Different!


Virtual Storage Zone: Getting the Best Oracle Performance on XtremIO


XtremIO for Oracle -- Low Latencies & Better Use of Datacenter Resources


Features and Benefits of Using Oracle in XtremIO Array Part 1


Features and Benefits of Using Oracle in XtremIO Array Part 2


Features and Benefits of Using Oracle in XtremIO Array Part3


EMC XtremIO Introduction - Scalable Performance for Oracle Database


Follow us on Twitter:


XtremIO Best Practices for Oracle using Advanced Format

Architecting a database on an All Flash Array (AFA) like EMC’s XtremIO is best done by reviewing practices to optimize I/O performance. One consideration is the use of Advanced Format and how it impacts the performance of the database Redo logs. Advanced Format refers to a new physical sector size of 4096 bytes (4KB) replacing original 512 byte standard. The larger 4KB physical sector size has these benefits:

  • Greater storage efficiency for larger files (but conversely less efficiency for smaller files)
  • Enablement of improved error correction algorithms to maintain data integrity at higher storage densities [1]


A DBA might be very interested in using a 4KB physical sector size to gain the efficiencies and improved error correction but there are a few considerations to review. For example, some applications and databases do not recognize the newer 4KB physical sector size. At EMC we have extensively tested Oracle on XtremIO following recommendations by Oracle support in Support Note, “4K ASM” (1133713.1).” To address the problem of a database or application not recognizing the new 4KB physical sector size there is the option to use 512-byte (512e) emulation mode.



Emulation mode uses a 4KB physical sector with eight 512 byte logical sectors. A database expecting to update (read and write) a 512 byte sector can accomplish this by using the logical block address (LBA) to update the logical sector. This means the 4KB physical sector size is transparent to the database as it can write to the 512 byte logical sector and thus backwards compatibility is maintained.


Picture 1: 512-Byte Emulation Mode


Unfortunately, there is the possibility of misaligned 4KB operations: one 512 byte update causing two 4K physical sectors to be updated. Before exploring the impact of misaligned operations on the Oracle database we need to how writes are managed in emulation mode.

Picture 2: Writes in 512-Byte Emulation Mode


Shown above writes are managed by:

  • The entire 4KB physical sector is read from disk into memory
  • Using the LBA the 512 byte logical sector is modified
  • The entire 4KB physical sector is written to disk


The key point of this read-modify-write process is the entire 4KB physical sector is modified. A request to modify one 512 bytes logical sector means reading 4KB into memory and writing the 4KB physical sector back to disk. For optimal efficiency it would be ideal to update multiple logical sectors belonging to one physical sector in one operation. When properly aligned writes to logical sectors are a one-to-one match to the physical sector and do not cause excessive I/O.


Misalignment is caused when incorrectly partitioning the LUN. To quote Thomas Krenn’s Wiki on Partition Alignment:

  • Partitioning beginning at LBA address 63 as such is a problem for these new hard disk and SSDs[2]
  • If partitions are formatted with a file system with a typical block size of four kilobytes, the four-kilobyte blocks for the file system will not directly fit into the four-kilobyte sectors for a hard disk or the four-, or eight-, kilobyte pages for an SSD. When a four-kilobyte file system block is written, two four-kilobyte sectors or pages will have to be modified. The fact that the respective 512-byte blocks must be maintained simply adds to the already difficult situation, meaning that a Read/Modify/Write process will have to be performed. [2]

Picture 3: Negative Impact of Misalignment


Quick side note:

I enjoyed reading the blogs “4k Sector Size” and “Deep Dive: Oracle with 4k Sectors” by flashdba. Although flashdba works for a competitor many of the recommendations apply to all Oracle users.


The solution is not to partition the LUN but present the unpartitioned device to ASM. There is an excellent blog by Bart Sjerps (Dirty Cache) called, “Fun with Linux UDEV and ASM: UDEV to create ASM disk volumes” that provides steps on using unpartitioned devices with ASM. In the blog Linux UDEV is reviewed as a solution to eliminate the misalignment:

  • We completely bypassed the partitioning problem, Oracle gets a block device that is the whole LUN and nothing but the LUN[3]
  • We assigned the correct permissions and ownership and moved to a place where ASM only needs to scan real ASM volumes (not 100s of other thingies) [3]
  • We completely avoid the risk of a rookie ex-Windows administrator to format an (in his eyes) empty volume (that actually contains precious data). An admin will not look in /dev/oracleasm/ to start formatting disks there[3]

Reading through the blog Bart points to the fact using UDEV and maintaining the ASM rules can be hard work and so he created a script called, “asm” and a RPM called, “asmdisks” to automate the use of UDEV with ASM. Highly recommended reading and look to the bottom of the blog for the link to download the RPM. Why not use ASMlib? Bart goes into detail on some of the challenges in using ASMlib in the same blog so I’m not going to list them here but rather encourage you to review the ASMlib section.


Here are a few examples of how to determine if you are using emulation mode as detailed in the Unix & Linux website under “How can I find the actual size of a flash disk?”

Using sgdisk:

sgdisk –print <device>


Disk /dev/sdb: 15691776 sectors, 7.5 GiB

Logical sector size: 512 bytes

The output shows the number of sectors and the logical sector size.


Using the /sys directly:
For the number of sectors:

cat /sys/block/<device>/size


For the sector size:

cat /sys/block/<device>/queue/logical_block_size


Using udisks:

udisks outputs the information directly.

udisks –show-info <device> | grep size

Using blockdev:

Get physical block sector size:

blockdev –getpbsz <device>


Print sector size in bytes:

blockdev –getss <device>


Print device size in bytes:

blockdev –getsize64 <device>


Print the size in 512-byte sectors:

blockdev –getsz <device>


Beyond using unpartitioned devices for ASM to bypass the misalignment issue is there any other recommendations? Interestingly, the database online redo log files by default have a block size of 512 bytes. For optimal online redo log efficiency it would be ideal to change from 512 byte block size to 4096 byte block size. As of version 11.2 this can be changed by specifying the BLOCKSIZE clause to values of 512 (default), 1024, or 4096 bytes.

Picture 4:Online Redo Log Blocksize Recommendation


For example, in recent XtremIO testing we used:



Before creating new online redo logs with the 4K blocksize there is a known issue with emulation mode. Emulation mode makes the 4K physical sector size transparent to the database so when creating online redo log files the database checks the sector size and finds 512 byte sectors. Unfortunately, discovering 512 byte sectors when attempting to write 4096 byte blocks results in an error like:


ORA01378: The logical block size (4096) of file +DATA is not compatible with the disk sector size (media sector size is 512 and host sector size is 512) [4]


The solution is using a hidden database parameter _DISK_SECTOR_SIZE_OVERRIDE to TRUE. This parameter overrides the sector size check performed with creating optimally sized redo log files. This parameter can be changed dynamically.




If you creating new online redo logs with the 4K blocksize then you might have to drop the original 512 byte redo log files groups.



Summary of 512e emulation mode best practices:

Below is a summary of the recommendations in this blog. Time for a disclaimer: I would also encourage you to review Oracle support note 1681266.1 “4K redo logs and SSD based storage” as a good place to start in determining if these recommendations are a good fit for your databases. Test these recommendations in a copy of production and validate the impact. Now that the disclaimer is over here are the steps.

  • Create your LUNs
  • Validate use of emulation mode
    • Example: blockdev –getss <device>
  • Do NOT partition the LUNs
  • Use UDEV to create ASM disk volumes (See Dirty Cache blog)
  • Set the database initialization parameter “_DISK_SECTOR_SIZE_OVERRIDE”=”TRUE”
  • Create Online Redo Logs with using the BLOCKSIZE clause for 4096 bytes

4KB Native Mode

In native mode the physical sector size and the logical sector size are the same: 4 KB. If planning on using advanced format native mode the DBA will have to create 4 KB block size redo logs. Outside of the redo logs there are a few other considerations for 11gR2 and higher.

Picture 5: 4K Native Mode


Unfortunately, I haven’t the time to fully explore 4K native mode but promise a follow-up for my next blog. I did want to provide this summary table below because it highlights Oracle’s recommendation to use 4KB online redo logs for emulation mode and native mode. In native mode there is no option to use 512 byte redo logs so in a good way Oracle automatically directs the DBA into using the optimal 4KB blocksize for the redo logs.

Summary table: Supported and Preferred Modes

Mode Type

512-Byte Redo Logs

4 KB Redo Logs


Emulation Mode




Emulation Mode



Native Mode




In the above summary table we see that emulation mode will support both 512 KB and 4 KB redo log block sizes but 4 KB is preferred. The overall recommendation is to use 4 KB block size for your redo logs.


Next Blog: Exploring the 4K native mode and insights into XtremIO testing using Advanced Format.


Table of References

[1] Advanced Format from Wikipedia, URL:

[2] Thomas Krenn Wiki on Partition Alignment, URL:


[3] Bart Sjerps (Dirty Cache) called, “Fun with Linux UDEV and ASM: UDEV to create ASM disk volumes” URL:


[4] Oracle Support Note 1681266.1 entitled, “4K redo logs and SSD – based Storage”