Find Communities by: Category | Product

In the past I was kind of depending on the service times reported for db file parallel write waits via AWRs. The reason for looking at db file parallel write service times was to help in determining if there were write I/O service time issues.


I brought up looking at db file parallel write times in a conversation with Kevin Closson and he gave me his         “Two Cents Worth” on the topic and suggested I read Frits Hooglands blog accessed via: 


So, I read Frits' blog which was nicely written!

The bottom line, from Frits’ blog, for sync I/O:  basically, db file parallel write waits are not timed at all.  And I/Os are executed sequentially.


The bottom line, from Frit’s blog, for async I/O:  db file parallel write waits shows the response time for the minimal number of I/O requests from the I/O completion queue. For example, via io_submit system call, 32 async write I/Os are submitted. Via io_getevents system call, it tries to reap/read at least a minimum number of I/O events (ie. 2) from the completion queue, but not necessarily the 32 write requests. So if it reads/reaps completion times of only 2 of the write I/Os, what is the value of the I/O timing provided? Well, really none.   


Yet another lesson learned.

A pithy blog on Oracle VCA, Trusted Partitions, Hard Partitions, OVM Live Migration & Oracle Licensing

I recently had my 1st collision with the Oracle VCA, and competing with a vBlock.

Customer Situation & Question

Customer says “Oracle tells me, if I use an Oracle VCA and I use Trusted Partitions, I can limit the number of CPU cores my Oracle DB runs on and therefore I reduce my Oracle Licensing costs. How does a vBlock compare to this?”.

Not having an understanding of Trusted Partitions or VCA, I had to do some research to try and answer their question.

The Bottom Line

I was not able to determine, after spending a few hours googling, if:

-using Trusted Partitions on a VCA requires one to hard partition a VM to specific CPUs(ie. CPU pinning)

-and, if OVM Live Migration works with Trusted Partitions

So, I am left to my gut feel which is: Trusted Partitions requires hard partitioning/CPU pinning of a VM to reduce core usage on a physical host & therefore reduce Oracle licensing of the physical host. And, …

  • by doing this, Live Migration will not work, but OVM HA will work
    • and
  • Oracle Rep will not tell the customer Live Migration will not work, but just tells them they can reduce their licensing spend by using Trusted Partitions

Maybe someone can set me straight on this topic, I sure want to learn the answer!   Not sure why Oracle Corp has made it difficult to get to the bottom of answering my question.

And, by the way, VMware ESX has had the ability to pin a VM to certain CPUs for years now. But, by pinning a VM to certain CPUs, operations like VMotion, DRS & DPM do not work. But ESX HA does work.

My Research on these technologies

VCA – Virtual Compute Appliance (an “Engineered” system)

  • How many Compute appliances is this now for Oracle?
  • An appliance built to enable virtualization of Oracle products. Kind of like a VCE vBlock.
  • Supports Oracle Trusted Partitions capability

Oracle Trusted Partitions

  • An Oracle supported approach which gives customers the flexibility to license Oracle software on a vCPU basis (well, somewhat).  You still license by the CPU core granularity.
  • I could not find a document that tells me how to configure Trusted Partitions on a VCA. So, could not find out any considerations or limitations.
  • From a licensing perspective in a Trusted Partition, 2 vCPUs = 1 physical core
  • Licenses must be procured in increments of 2 physical cores
  • Customer is required to license the highest number of vCPUs running at any given time
  • The customer must use Enterprise Manager 12c
  • The customer must deploy the 12c agent on the guest OS running the VMs in the trusted partition
    • They can run OEM in connected mode or disconnected mode. Connected mode means OEM has a connected link to My Oracle Support(MOS) so usage can be captured & uploaded. Disconnected mode means there is no constant connection MOS, but customers are required to upload usage reports quarterly, maintain backups of the reports & keep 2 years records of the reports.

OVM Live Migration

  • Similar capability to VMware vMotion, which is a reactive or proactive way to move a running VM from 1 physical host to another.
  • If you use CPU pinning with OVM & try to use Live Migration, you lose hard coded CPU mapping info in the guest vm.cfg file. So, on the new host you need to re-apply the CPU pin info in the vm.cfg on the new physical host.
    • Per an Oracle document:  “Live Migration of CPU pinned virtual machines to another Oracle VM server is not permitted under the terms of the Hard Partitioning license. Consequently, for Oracle VM3, DRS & DPM policies should not be enabled for server pools containing CPU pinned guests.
Matthew Kaberlein

Oracle 12c Flex ASM

Posted by Matthew Kaberlein Mar 17, 2014

Flex ASM in a RAC Config

In Oracle 12c, a new capability (or option) was introduced call Flex ASM.  Flex ASM comes with some new components which will be discussed:  ASM Network & ASM Cluster.  This new capability does not impact any of our DB Storage layout guidelines/best practices we have created over the years.  But…

•       What is Flex ASM?

–      Flex ASM allows the ASM instance a DB instance interfaces with to run on a separate server/host than the DB instance(s)

–      Only a subset of the RAC nodes run an ASM instance, ie. 3 of 4 RAC nodes

•       Why deploy a Flex ASM config?

–      Less ASM homes to Patch & Upgrade, especially in a “wide” RAC

–      If the ASM instance fails on a host, DB clients of the ASM instance auto access another ASM instance

•       In pre-12c, the DB instance(s) would fail on the host where the ASM instance failed

o    Helps in high DB consolidated deployments

–      If the ASM host fails, CRS starts another ASM instance on another host

–      Reduces the CPU & Memory requirements for the RAC nodes where the ASM instance is NOT running

•       Though I don’t buy in to this value add much since the host RAM requirements for ASM is about 250MB per host and I have never heard nor read that ASM drains the CPU

New ASM Components:  ASM Network & ASM Cluster

•       ASM Network (within 12c Clusterware)

–      Public network, which connects the RAC nodes to the company public network

–      Private IP network, ASM Clients use to connect to ASM instances (Figure 1, RAC nodes 1 & 2)

•       Should use same RAC inter-connect…Light traffic between ASM instances

•       ASM Cluster

–      Enables ASM instances to communicate with each other

–      Multiple DB instances can share the same ASM disk groups

•       Requires Clusterware installed on each node

–      Requires an ASM network to communicate between Hub nodes

–      Requires Password file for ASM instance to authenticate DB instance

•       Stored in +DATA disk group


The default ASM instance creation cardinality is 3. Which means in a 5 node RAC (see Figure 1), 3 of the nodes run an ASM instance. The DB instance on RAC node 1 connects to the ASM instance on RAC node 2 to enable it to issue IOs against the ASM disk groups. And the same applies to RAC node 5 & 4 (see Figure 1).



Figure 1 - Normal state of Flex ASM


In Figure 2, there is a failed ASM instance on RAC node 2. So, RAC node 1 now interfaces with RAC node 4’s ASM instance to get ASM metadata info and to be able to issue IOs to the storage array. And the DB instance on RAC node 2 interfaces with RAC node 3’s ASM instance and a 3rd ASM instance is spawned on RAC node 5 due to a cardinality of 3 being set.



Figure 2 - Failed state of Flex ASM Instance

RAID-5 vs. Ditka  -  Who Wins in a 3 game match?


Ditka wins 85-15, 90-10 and 95-5. 

Being a Chicagoan and a Bears fan I thought I would use the Ditka theme.  These scores are important for later on in this discussion.

RAID-5 vs RAID-10 is still a topic of discussion and a bit of religion with DBAs.

Several times over the past few weeks I was talking to different customer DBAs about performance issues with one of their DBs and they told me they really want (or need) RAID-10 for their DB and redo logs. And, flash drives are needed for the redo logs.

I had AWRs in 15 minute snap intervals for many hours for their respective DBs and reviewed them before meeting with the DBAs.  I noticed no sluggish performance from the storage array for redo log writes (ie. log file parallel write, average write time was 2ms-4ms) and read I/O service times averaged 3ms – 6ms (ie. db file sequential read). Nor were there any service time issues when viewing the AWR histograms for these wait events.

I asked the DBAs why they feel they need RAID 10 and the answer was something like “because it offers the best IO performance”.

As Oracle folks we’ve been hounded in to believing RAID-10 is the only RAID config to achieve good physical IO performance for an Oracle DB.

For a single DB, when the read/write ratio is 85/15, 90/10, 95/5 the RAID5 parity overhead to the DB & end user is minimal to none on an intelligent storage platform like an EMC VMAX or VNX. One of the reasons is, we do not write directly to disk and therefore do not incur parity calculations at the time of LGWR writes for example. Instead, our writes are absorbed in DRAM cache slots and the I/O acknowledgement is sent back to the host. Then, later on (minutes to many minutes, depending on fall through rate) we de-stage the cache slots to disk and do the parity calculations. So at this point the writes are out of transactional scope.

Now, we will deploy RAID-10 for Oracle when many, many Oracle DBs are consolidated on to a single storage platform. Reason being, the backend IOPS needed for RAID 5 parity calculation can be amplified quite a bit by the sheer number of many, many DBs being deployed on a single storage array.  IF, the read/write profile is high for the DBs, ie. 85/15 or 80/20.  So as usual, the “multiplier effect” impact occurs.

In the end, given the right current & future IO performance metrics, we can design a high performing RAID-10, RAID-5 and even (crazy as it may be) RAID-6 configurations for an Oracle DB.

DBWR & LGWR write example illustration:


Filter Blog

By date:
By tag: