Find Communities by: Category | Product

As All Flash Arrays begin to accelerate their expanding footprint and mindshare in corporate data centers, a large percentage of customers are asking about the value of placing Exchange databases onto EMC's Scale-Out All Flash Array -- XtremIO. This is an interesting paradoxical scenario.  At first blush, you might believe that Exchange needs "lots of capacity" and at the same time, the latest versions (2010, 2013, and 2016) have relatively "low performance" requirements compared to earlier versions.  In fact, when running in Cached Mode (connections made and managed via Client Access Servers), the IO requirements seem to be down-right unimportant.  Why would customers be interesting in an all-flash storage option for Exchange?  The better question is, how can we help them understand the value of using EMC's Scale-Out All Flash Array to support their Exchange implementation?

In this blog post, I will share:

  1. The state of the On-Premises Exchange environment -- challenges, issues, considerations
  2. IDC's analysis of All Flash Arrays in the Messaging & Collaboration marketplace
  3. How EMC's XtremIO Array solves IT and Business challenges for Exchange implementations
  4. Next steps and two trusted analysis techniques for predicting potential success with XtremIO for specific environments
  5. A sample Total Cost of Ownership model based on 10,000 Exchange users


Here's what we know so far:

1. Retention is the IT term for "Keep everything forever".  Storage is cheap, unless you need 144TB of it... or 1.4PB...  For every e-mail that is retained in the Active database, it is also stored in the database copies.  Studies show that mail storage is growing at a rate of more than 30% YOY.  In one customer example of storage growth, the migrated capacity from Exchange 2003 was 72TB back in 2010.  That same organization now requires 158TB of capacity after only three years.

2. IT administrators have soured to the notion of a) managing racks-upon-racks of Direct Attached Storage shelves, and b) Exchange Continuous Replication (Database Availability Groups/ DAG) consumes between 2x and 6x the capacity required to store the production copy of email/calendars/contacts.  In fact, IT Administrators used to enjoy Single Instance Storage (SIS) provided by versions of Exchange prior to 2010.  Most IT Admins report that capacity requirements have multiplied by a factor of 6x after migrating from Exchange 2003 to version 2010.  The mail that used to consume 12TB of space in an Exchange 2003 Single Copy Cluster, now consumes 72TB in a three-copy DAG.  This explosion of capacity has left most IT Admins disillusioned with keeping mail "on prem"

3. Distribution Groups (DLs) have become a de facto method for transmitting messages.  Coupled with the fact that nearly 70% of all email messages contain attachments and the fact that every attachment is stored repeatedly in every DL member's inbox, mailboxes and mailbox databases are larger than ever before.

4. Not all customers and not all end-users connect to their mailboxes via Cached Mode connection.  In fact more than 10% of a typical Exchange user-base is forced to run in Online Mode for several reasons, including VDI, Security/Governance (cannot use OSTs), Workflow applications (cannot tolerate cached versions of mailbox items, and HIPPA regulations.  And yet, in a portion of our customer accounts, 100% of email users are forced to use Online Mode for connecting to their mailboxes.  Online Mode increases the IO requirements compared to Cached Mode by 270%.


A study by IDC, published in March of 2015 outlined the issue of retaining primary and "backup" copies of data within corporate environments.  They call it the "Copy Data Problem".  IDC's advice is simple, "use the most advanced storage array technologies that support real-time thin provisioning, de-duplication, compression, and snapshot capabilities to reduce your overall storage footprint".  AFA for MandC_Marketplace_IDC.pngIDC also completed an analysis of the All Flash Array marketplace in May, 2015.  Their findings were compelling to say the least.  The bulk of organizations deploying All-Flash Arrays for use in Messaging & Collaboration workloads are EMC's core customers.


EMC has responded to all of these storage concerns with one comprehensive offering.  XtremIO has been tested and evaluated by dozens of customers including EMC IT.  Initial findings are reporting impressive and compelling efficiency rates ranging from 2.8:1 to more than 16:1 depending on the number of mailboxes and the number of DAG copies.  In the vast majority of installations customers deploying two copies of their Exchange databases on XtremIO are observing 7:1 efficiency rates. Again, we are observing an average reduction in actual storage usage for the Active copy of 60%.  In real numbers, this means that 12TB of Exchange databases typically consumes only 4.8TB of space on an XtremIO array.  Of course, every environment will be different.  In the last section of this post, we'll discuss two valid methods of analysis that will allow you to understand how your specific environment will benefit from the power of XtremIO.

Here's how it works:

XtremIO offers a radically innovative storage platform for Exchange Administrators.  Let’s take a deep dive into how XtremIO works with Microsoft Exchange databases.   XtremIO works like no other storage array ever created.  According to Wikibon, XtremIO is a Generation 4 flash array -- it is specifically built to make optimum use of flash drives.   It uses a massive ultra-high-speed metadata table to store and track all of the block locations and fingerprints of all of the data that is written to the array.

XtremIOContentAddressing.png

This metadata table is like a massive File Allocation Table with a list of each and every 8K block on the array along with a numeric representation of the data it contains -- every block has a hash, like a serial number or a fingerprint.  Every time a new 8K block is written to the array, a numeric hash is generated.  When additional data is ingested, hashes are generated for those blocks too.  If the hash of a new block matches the hash of a previously written block, only the metadata is updated, not the underlying flash drives.

XtremIOContentAddressingStorage.png


This radical new approach to storage reduces writes to the flash, reduces storage usage and dramatically increases write performance compared to any other storage array.

 

Let’s see how this technology – already producing sub-millisecond response times for more than 1000 enterprise database applications – benefits a Microsoft Exchange environment.  We'll start by drawing a logical representation of the host volume, the LUN. Inside this, we'll place your Exchange database.

 

Inside that, we'll place your data – the messages, the attachments, the contacts, the headers. Keep in mind, XtremIO has no awareness of file, file structures, or databases -- it only knows about 8KB blocks and whether or not it's seeing them for the first time or a subsequent time.  If an inbound block is identical to a previously written block, the XtremIO metadata table references the previously written block and reports the second occurrence in metadata.  As far as the host is concerned, all of the data that it has written to the volume has been logged into that volume's FAT or GPT and all of the data blocks have been committed to disk -- in this case, the XtremIO array and its metadata table.  Exchange on XtremIO Efficiency Explained.pngBut here, the orange and smallest blue cylinder inside the data, we'll talk about some of the compression and deduplication technologies the XtremIO array brings – and how those fingerprints play a role in reducing our capacity usage.  Based on actual customer data – including an extensive study of EMC’s own Exchange environment, we are finding that an average of 60% LESS storage capacity is used compared to DAS and traditional storage arrays.  And the more mailboxes and copies of mailboxes you place on XtremIO, the higher the savings becomes.  Here why. Let’s follow the bits as we store Exchange data on our XtremIO array.


First, the LUN is created on the XtremIO and is presented to your Mailbox server.  From this point forward, the Exchange server is making use of a block-storage device.  Windows formats the volume, a drive letter or mount point is assigned, and Exchange is able to utilize this new storage volume.  Then, you either create an empty mailbox database file or, you can copy an existing database file to the XtremIO array. As the database is ingested into the XtremIO, each database page is mapped into the XtremIO’s metadata.  For every database page, there are four hashes created.  Remember, only blocks with unique hashes are candidates to be written to the flash drives.  If a block containing zeros is discovered, only the metadata is updated.

 

What actually gets copied onto XtremIO’s flash drives is the actual data, not the strings of zeroes within the database – but this is only the first of three steps as the data is written into the array, there’s more data reduction before the data actually lands on the flash drives.  Within Exchange databases, a large percentage of the database pages are blank – filled with zeros – either waiting for new message data or erased after deleted items have been removed.  Unlike traditional storage technologies, when databases are placed on XtremIO, none of these blank databases pages consume any space on the array.  XtremIO saves space by remembering the content of each and every page that it’s already ingested.  If it sees a duplicate of that data, it simply references that “already written” block in its metadata table.  XtremIO does this operation in the data pathway – it never allows duplicate pages to get onto the flash drives – and it never performs any post-ingest deduplication process.

 

ExchangeDatabaseOnXtremIO.png

The last thing that happens before any data is passed onto the flash drives is we remove any repeating characters within the 8K blocks, just like you would typically see in a compression program.  It would remove any repeating sevens or eights or repeating sequences, or zeroes. No! Not zeroes!  Zeroes never get written to the flash drives.  What you're left with is just the unique data, and that's how XtremIO creates an incredibly efficient storage platform for all databases, but especially for Exchange.   You might think that all this In-Line data processing would slow-down the IOs, but it has just the opposite effect.  Even under extreme load, XtremIO predictably delivers sub-millisecond response times.  XtremIO is a match for every Exchange environment, but makes OnLine or Non-Cache Mode users and VDI users especially happy and productive.

 

Within your database, you have mailboxes. Mailboxes are loaded with zeroes. Through Background Database Maintenance, Exchange strives to keep about 20% of the database filled with black pages at any one point in time. Remember, Exchange pages are 32K in size.  [Animate messages being added to each mailbox, various colors]  For the XtremIO array, that means it takes four 8K blocks in order to store one page of Exchange data. This means that any partially-filled Exchange page will also benefit from this deprovisioning.  And, when anything is deleted [show white replacing already written messages] from Exchange, those pages are blanked or zeroed , so those pages become de-provisioned, as well. Contrary to common belief, attachments stored in an Exchange system are not compressed before they're placed into individual users' mailboxes. Keep in mind that an e-mail that's been routed to dozens of people is stored dozens of times inside the Exchange database. Because those attachments and those messages are placed into each user's mailbox, XtremIO records their placement in each user’s mailbox, yet stores the blocks associated with that attachment onto its flash drives only once. When a given user opens the message, he or she finds it in his or her mailbox as if it were stored dozens of times on the array. Through real-world observations, we have learned how XtremIO actually benefits real Exchange environments; we don’t pretend that JetStress databases represent real-world scenarios.

XtremIOEfficiency.png

In the many customers we have studied, we've learned that through thin provisioning techniques, we can save as much as 40% of the necessary capacity to store our Exchange databases. By not writing zeroes, we've learned that we can save an additional 20% to 40%. We've learned that because we don't duplicate data blocks, we can save another 20-30%. We also learned that because we can remove repeating characters and strings of characters, we can remove another 20% from our storage requirements.  All in all, this tallies up to about 2.8-to-1 storage efficiency.


Let's recap

EfficiencySummary.png

Off the top, our volume savings is 40%; this is our LUN tail space.   Because we don’t write pages filled with zeros, we save another 30%. Because we de-duplicate repeating blocks within the Exchange mailbox databases, we save another 30%, and then we compress the remaining blocks that haven't already been duplicated by another 20%. This gives us a typical single-copy efficiency of 2.8-to-1. That’s a reduction of 64%!

And remember, if we duplicate the database for use in a Database Availability Group, not only are the databases protected by more than five-nines of availability, but, those copies come for free -- they are block-for-block duplicates. That means for every copy of the databases we make, our efficiency number doubles.   And because we use XtremIO snapshots to seed those copies, we can generate DAG copies in seconds instead of hours.


Exchange administrators need predictable, reliable performance.  They rely on XtremIO to deliver sub-millisecond performance, ease of administration and, reduced costs 24-hours a day.  Unlike other All-flash arrays, XtremIO delivers absolutely consistent performance.  In fact, EMC’s tests reveal that a single XtremIO X-Brick handles 45 million messages a day with over 300,000 mailboxes at 150 messages per user, per day.


Analysis -- How will XtremIO fit into my environment?

Through our own analysis process, we have developed two different and complimentary methods for discovering your expected efficiency rates.

 

Checksum Analysis

Screen Shot 2015-04-08 at 8.09.26 PM.jpgExamining your Exchange databases is simple, but can be time-consuming.  Create a snapshot of your database(s) and mount that snapshot to an alternate host or VM.  You will need to install the Exchange utilities onto your alternate server.  In a command window, run the Exchange utility called ESEUTIL /k [database name].  This will scan the database at a rate or 200GB/hr and look for failed checksums in each of the database’s pages.  At the end of the operation, ESEUTIL will present a short report.  One of the fields in the report shows the number of uninitialized pages in the database.  This number, when multiplied by 32K will tell you the amount of space that the XtremIO array will NOT use in storing that database on the array.  Take the total database size as it appears on the Windows volume and subtract the uninitialized space from it.  This is the space needed to store your database on XtremIO (prior to compression and de-duplication!).

 

EMC MiTrend Efficiency Analysis

https://app.mitrend.com/emc/#instructions/XtremIO_Reduction contains detailed instructions for how the XtremIO Efficiency Analyzer works and how to get dependable results. If you clicked the MiTrend link and were challenged for credentials you could not provide, please contact your EMC Partner, Rep, or Systems Engineer for assistance.  Many Exchange admins are reluctant to have scanners running on their production databases despite that fact that we have successfully scanned dozens of customers' production volumes with the tool.  For Exchange admins who "worry", simply restore a representative sampling of databases to an alternate server (non-production VM) and run the MiTrend tool against those volumes.  Once you have the MiTrend data collection, you can have your Partner or EMC SE submit your collection to the MiTrend site.  A report will be produced in short-order and you can begin to make decisions.

 

TCO Comparison

Let's stop to ask what this is going to cost us.  We are faced with questions that we have previously NOT needed to answer.  XtremIO takes all traditional storage methods, mechanisms, and technical structures and tosses them aside; we are left asking, "how can XtremIO save me money?" or "how can XtremIO cost less than traditional storage?"  To assist you in understanding how XtremIO will compare financially to the storage you are using today, we have assembled a "straw man TCO" based on 10,000 seats, 2GB average mailbox size, with 150 messages sent/received per user, per day.  The TCO includes all of the various aspects of installing, managing, cooling, supporting, and paying for facilities costs that you would typically find in any other TCO model.  The model in this blog post does not attempt to represent ANY cost savings derived from simplified management of the storage device -- all personnel costs are held even across all systems analyzed.

The "Comparative Costs of XtremIO" graph shows three Exchange 2010 implementations based on three different storage devices.  All other aspects of the Exchange implementation are held as constant as possible.  10000-seatTCO_comparing_XtremIO_to_VNX_MS-PreferredArchitecture.pngFor example, same number of servers, Ethernet ports, admins, mailboxes, databases and database copies -- everything is the same except the storage and costs associated with the storage such as maintenance, installation, facilities costs, power and cooling.  Long-story short, all three storage configurations resulted in average mailbox costs that were within 25-cents of each other.  And, based on the "deal" you might get from your hardware vendor, could actually be the same price.  The prices used in this TCO are the typical "street" prices seen in the open marketplace.  There are no special discounts, no list prices, no one-time offers.  These are real prices of real configurations obtained in April of 2015.

 

Think Exchange Think XtremIO.png

Conclusion

EMC XtremIO is a thoroughly tested, production proven, and enterprise-class array.  We know you will want to analyze your Exchange environments today.  We are delighted to say, "Welcome to the rapidly expanding community of Exchange admins who would not trade their XtremIO arrays for all the DAS in the world."

 

When you think of Exchange, think of XtremIO. It's an amazing TCO.

Data Protection for Exchange & SharePoint

 

Most Exchange administrators look at data protection and yawn.  SharePoint administrators look at backup and cringe.  For Exchange admins, backups are boring; actually, VERY boring, time consuming, and frustrating to schedule, monitor, and remediate backups.  For SharePoint Admins, backups are complicated.  SharePoint Farms are multi-server octopus-like entities with dependencies, linkages, and fragile indexes that need each other in order to function properly.

 

A complete contrast to backups, there is nothing more anxiety producing than working through a failed data restoration issue. Restores are exciting, but often for all the wrong reasons. What is EMC doing to make backups more exciting and restores less anxiety producing?  The engineers at EMC have developed a suite of tools to address backup reliability, restoration confidence, and make monitoring and remediation of data protection fun.  Yep, I said "fun".

 

I know what you are thinking… “this guy is crazy!”. How could protecting data possibly be fun?!  He must be a geek… Well… I might be a geek, but let me show you what I'm talking about in a demo. At the center of the EMC Data Protection Suite is a software package called Data Protection Advisor.

The demo is about eight minutes long so grab a drink and stay focused.  This demo may change your entire perspective on backup and recovery.

 

https://www.youtube.com/watch?v=JsCEzGYZWLI

 

In a couple of weeks, I'll post a follow-up to this where we'll talk about restoring data elements — like individual emailmessages, SharePoint content (files) and how to index all of it so your legal team will be buying you lunch every Friday!  Talk to you then.  Enjoy.

My fellow blogger @Noel Wilson recently posted part one of a three-part series that hopes to explain how EMC observes the Private, Hybrid, and Public scenarios that our customers deal with every day.  Unlike Noel's excellent overview of Cloud scenarios, this blog entry is part two of my multipart series that focuses, not on constructs, concepts, and scenarios, but on the actual tires on the proverbial road.  Back on the 11th of September, I posted a blog about "Corporate Availability.  I mentioned that the white paper would be published soon.  Well — it's out and it's more detailed than even I expected.  It's so detailed that we will be releasing a series of video demos to explain all of the aspects the white paper covers. Just in case someone forwarded this blog to you and the links got stripped-out, the paper is on emc.com (http://www.emc.com/collateral/white-paper/h13360-cont-avail-ms-apps-wp.pdf).  So here is a brief summary of what you'll find in this mammoth white paper:

 

Microsoft Business ApplicationsEMC TechnologiesVMware Technologies

Microsoft Exchange Server 2013

Microsoft SharePoint Server 2013

Microsoft SQL Server 2012

EMC VNX

EMC RecoverPoint

EMC VPLEX Metro

EMC VNX

EMC RecoverPoint

EMC VPLEX Metro

Here is the basic visual representation of what the paper explains:

Screen Shot 2014-10-03 at 1.10.38 PM.jpg

The idea is that a total of three data center (DC) locations will be deployed -- two within 60-miles of each other (the Primary pair), the third (Tertiary) is as far away as possible and feasible -- a typical separation between Primary and Tertiary data centers would be 600-900 miles (about 1000km-1500km).  The two "near proximity" data centers will handle the production load in a shared resources model -- either data center can use the resources of the other and workloads can migrate at a moment's notice from one DC to the other.  The third (tertiary) data center is only put into production when the pair of Primary DCs is offline for network, electrical, or physical reasons.  Please note that the tertiary data center will be "moments behind" (in replication terms) from the two Primary data centers, so Reporting, test, development, analytics, cube builds, and the like could all be housed and operational at the tertiary location.  There is one additional workload that would be running at the tertiary data center by default: Backup!  Operational backups should always be made in the building that DID NOT just catch fire...  If backups can be taken AT the offsite location, all the better.  Why backup, then replicate?  Why not replicate, then backup?

In this three site configuration, there are two sites coupled via Ethernet — and the Fibre Channel Fabric is extended between these two Primary sites — there are several ways to accomplish this (none of which are expensive or complex here in the year 2014...) -- and of course the Ethernet is also extended to both Primary sites.  The third site, however, is only connected via Ethernet as there is no need to extend Fibre Channel over a distance of 900-miles!.  Your exact bandwidth requirements will be set based on the actual traffic that is "write oriented" -- only the changed blocks ever travel to the third site (under normal operations).  Only when the third site is brought into Production, does the read traffic ever "back flow" through the Ethernet to either site A or Site B — unless of course you have a tertiary iNet point of presence at that third site for use during failures.  In that case, the tertiary site can me come the sole remaining resource (furnishing both Primary workloads as well as all backup infrastructure).

The two Primary data centers would be connected like this:

Screen Shot 2014-10-03 at 1.20.26 PM.jpg

Anyway… the scenarios that the paper explain are:

  • A single VM failure (and its recovery)
  • A vSphere host failure (and the resulting recovery)
  • An entire Storage Array outage (like it got hit by a swinging gorilla or backhoe — hey, it could happen!!) and the resulting recovery
  • An entire site failure (like it got taken away by an alien space ship — or backhoe) and the resulting recovery
  • An entire regional outage that takes both primary data centers offline — like a massive grid failure (unfortunately these actually do happen) and the resulting recovery

In all but the last scenario, recovery of services happens within minutes — actually, that's not true… in ALL scenarios, recovery happens within minutes — it's just that the dual DC failure takes about an hour to restart everything at the tertiary site… so "minutes" is more like "tens of minutes"… BUT, in all cases, recovery is COMPLETELY AUTOMATED.  I'm serious.  No hot-line, no midnight con calls with 47 people from four states and nine cities.  It's all automated — and wonderfully dynamic.

 

Please scamper (ok , don't scamper)… Just click.  The white paper is http://www.emc.com/collateral/white-paper/h13360-cont-avail-ms-apps-wp.pdf and demos will be following in the next few weeks.  We'll do SQL Server first!

 

Please — comment below if you like it — and please submit your criticisms if you can muster any — seriously.  This is the first paper of this magnitude to be published by a major Cloud enabler — we want your feedback so we can continue to bring you the solutions you've asked for!

 

Cheers!

I can't tell you the number of Windows Clusters I've implemented over that past fourteen years because of patch Tuesday, but let's just say it was more than I could keep track of.  If these words ring true for you, you'll be very excited to hear that EMC is releasing a massive Solution white paper that discusses how to bring your systems, not just up to local availability, but to Corporate Availability.  We are bringing you abilities to not just patch whenever you want, but to do whatever you want, whenever you want — at all levels of the Storage/Compute/Network stack.  We are bringing you products and techniques that protect your data and your systems better and more accessibly than ever before.

 

But what the heck is Corporate Availability?  It's the level of application availability that business units have learned to expect and demand.  It means that the systems they use to generate revenues, to manufacture products, to maintain contact with their customers will be available -- constantly available.  Corporate Availability means that local High Availability techniques are no longer "good enough".  It means that patching needs to occur without a testing cycle -- it means that the systems that have been patched need to be able to go backwards in time to "undo" a patch that my have negatively impacted the system.  Corporate Availability means that, no matter what is happening in Server Administration, Network Management, or Storage Management, every application owner can focus on running their applications.  There's no more tolerance for scheduling outages.  There's no more "agreeing to wait for them".  Application owners need to have access to their application infrastructure constantly and consistently.  That's what Corporate Availability demands.

 

There once was a time when applications had "maintenance windows".  I remember when I was a Technical Account Manager (TAM) at Microsoft back in the last decade, the customers I supported were large, international financial services firms.  Their IT shops, just like today, need to support hundreds of SQL Server instances, dozens of Exchange 2003 servers, and hundreds of Windows File Servers.  Every month, they would wait for Patch Tuesday.  At about 10:00am PST, the email blast would come from Bellevue or Redmond announcing to the organization, which groups would need to begin their systems tests based on the latest hotfixes for the "following products are affected" list… and the fire drill would begin.  Who would be affected, how long would the systems be down, who would work late, what impact would these updates have on related systems.  How would we role back?

 

Many of the issues related to "Patch Tuesday" have been satisfied by Clustering techniques and by snapshotting technologies — both VM's as well as datasets.  Hardware VSS has made going backwards in time a commonplace concept.  But what about the rest of the stack?  Can application availability be totally satisfied by snapshots?  We all know the answer is "NO".

 

Networking and the "connectedness" of our user base is so complex, and so entirely required for everything we do, that we need to provide layered techniques to maintain access to our valued datasets.  These datasets are, of course, the reason that IT organizations and IT infrastructure exists.  Each data element, whether it's a customer order for a pair of black and white checkered Vanns skateboarding sneakers or an email from your HR department outlining the new holiday schedule for 2015, those data elements are the key to an entire system's existence.  Allowing the consumer, the end-user, to access those data elements is the stuff this new EMC Solution is all about.

 

In the Solution, we gathered your favorite applications -- ok, so "favorite" might not be the right adjective… -- we gathered your most popular applications -- Sharepoint, Exchange, and SQL Server.  We implemented them in the way that you would most likely chose.  We even put the SQL Servers into Failover Clusters, not AlwaysOn Availability Groups.  And we demonstrate how to maintain application availability across THREE sites. EMC's architecture does not REQUIRE three sites, but it allows you the opportunity to explore what EXTREME availability can look like.  And it's pretty amazing how simple the whole thing really is.  It's not simple because it's 2014; it's simple because it's EMC.  The folks you've trusted since 1990 when we brought you Symmetrix, the world's first storage supercomputer.

 

Please take some time to glance through the new white paper -- it'll be launched very soon -- it'll show you the exact steps to take to create an availability infrastructure unlike anything you've seen before.  It takes advantage of products that you have come to know and trust -- components such as EMC VNX, EMC RecoverPoint, and EMC VPLEX.  It uses new code versions that allow them to all know about each other and do things that you've wish they could do.  Things such as replicating from a stretched Windows Failover Cluster (located in two synchronously-connected data centers) to a third data center that's asynchronously connected (three states away) AND automatically fail to it if the network access links get a visit from the Grim Backhoe.

 

This blog will be updated when the paper is published, but for now, please post your comments below.  Please let me know if you are as excited about this paper as I am!

Written by James Cordes and John DeLuca


Azure is "Cloud"; it's not "applications".  As IT Professionals, we bring our applications to The Cloud.

But WHICH applications should we bring to The Cloud?  All, some, none, the big ones, the small ones, the ones that can't go down, the ones that CAN go down??  How do we choose? How can we know if we were right?  How do we know where to start?!

One of the most considered candidates is Messaging -- Good 'ol email.  Looking into the past, we have seen messaging providers come and go.  Back in 1998, we had early multi-tenancy offered by a handful of ISPs or ASPs as we called them… Exchange was a different animal back then.  Microsoft had no intention of making Exchange a multi-tenent application for Cloud.  Heck, "Cloud" didn't even exist back in 1998.  But Microsoft has become a leader in responding to customer demand.  They are about the best there is, from a pure software perspective.  The Exchange product group took it upon themselves to rewrite the Exchange Mailbox role to reduce the number of Critical Situations (a term coined by Microsoft's Premier Alliances Group back in 2001/2002 following the horrific events of 9/11.  Critical Situations occurred when a customer was facing a serious "service down" scenario.  The complete outage of an Exchange environment, for example.  Microsoft Product Support Services and Microsoft Premier worked arduously to determine the root causes of their Exchange CritSits.  They came up with a list of five things.  SAN Zoning, Multi-Homed Server NICs, Windows Clustering Configuration Errors, Unrecoverable Hard Disk Errors, SCSI Bus resets related to backup devices residing on the same SCSI network as the Cluster Disks.  The point is, Microsoft's Exchange Product Group worked for more than seven years to re-engineer the Exchange Mailbox Role to eliminate these CritSits.  Long story short, Microsoft decided that a hosted Exchange environment would give customers the most ideal environment with the highest customer satisfaction rate.  A hosted Exchange environment meant that Microsoft, itself, would install and operate Exchange.  Ten years later, we have the result: Office 365.

Many of my customers ask the same questions that you're asking about Office 365, and they usually start with "Why wouldn't I do this? It seems to make sense."  So let's look at the facts.

1) Microsoft is running this service;  how bad could it be?

2) I have a clear definition of what's being offered

3) I can free up my existing staff to do what they want to do instead of running exchange servers

4) I get 50 GB mailboxes! That's way more than I could possibly offer my users today

5) it comes with an SLA that's three nines; I would have to spend a lot of money in order to get to that level of service.

6) it comes with Office for iPad?! I can't do that today!

7) I can include Lync and Sharepoint services too?! All on the same monthly bill?! I'm having trouble figuring out why I wouldn't do this."

Screen Shot 2014-06-05 at 12.53.02 PM.jpg

Here are some factors you may not have considered as you look at this seemingly great list of features.  Of the things to consider, we divide them into three basic categories. The categories are Trust, Cost, and Features.

Trust comes in a number of forms in this scenario.  The first one is who's looking at my data and do I know if they are. Keep in mind that your data will be housed at a public facility. This means that, not only can Microsoft look at your data, but also any law enforcement organization can submit a subpoena to Microsoft.  This action, allowing the law organization to look at your data and neither Microsoft nor the law organization has to tell you that they are doing so. Furthermore since your data is in the public facility, we can only imagine the number of attacks that happen at Microsoft Azure data centers every day. Simply do a Google search and you'll see the number of reports that leak out for Microsoft every day.  If you were at all concerned about who's looking at your email, your Sharepoint information, and your Lync traffic, this could be a concern for you.

From a cost perspective, it gets complicated quickly.  Running a messaging environment includes hard costs and soft costs, operational costs and capital expenditures.  When comparing the cost of an Office 365 monthly license, it's easy to forget all of the expenses that you incur in your current messaging environment. It's simple to tally up salaries and racks and power and cooling and cabling and disk drives and repair bills and software updates but what about the things that are difficult to account for?  What about the new spam filter that I just spent $50,000 on? That was a shared expense. Can I still use that for other things or does that cost go into this bucket? And my cool new F5 load balancer cluster? What am I going to do with that?  Do I count that as sunk cost?  Consider such factors such as: Did I just purchased a new VBLOCK that I should be installing Exchanging into?  How much time will my Exchange administrators need to build-out the Active Directory Federation?  Since Office 365 doesn't come with any sort of Helpdesk, how many people will I need to keep on staff to support my users? When there is an outage at Office 365, Microsoft will give me money back, but what about the loss of productivity? Who pays for that?

And lastly let's talk about Flexibility.  Sometimes we talk about flexibility in the form of functional requirements. For example, do I have third-party applications that plug into my current Exchange environment? ..that might plug into my current SharePoint environment? …there might be gateways to my Mail environment, are they the same ones used to archiving mail into a secondary failure domain by using SourceOne. Does that link to Office 365. If it doesn't, have I evaluated, and do I like the native archiving that Microsoft provides me in Office 365? Many consumers of Office 365 come back with the answer that they don't like...

At the end of the day, the services that you offer your end-users are all that matter. When Joe from accounting is using his iPad to check his mail when he's at his daughter's soccer game, he could care less which server model you installed his mailbox on. He does care, however, that there are messages in his inbox from an undesirable website. He also cares about the mail that he forwarded to the automated mail dispatcher that's linked to Sharepoint, and that the workforce application actually notifies his manager that he needs to be on vacation the week of July 14.

Let's face it, IT is complicated these days and chances are very good that you've created a fairly complex system of interworking parts to service the needs of your users. If your desire is to get out of the email business, this is easily accomplished by working with one of EMC's Service Providers. We can get you what you need without sacrificing any of what you want.  And, the costs will be in-line with or less than the long-term costs of Office 365.

Exchange is a critical application for you and having control over messaging applications and data is important to the end users of messaging systems. EMC Exchange Solutions dictate that Exchange and its data stays within your control.  This means that data sovereignty and data governance is not at the mercy of another organization. Exchange data and its access remains under your control, within your security and privacy mandates, and is compliant by being auditable, attestable, and discoverable.  Finally, control means streamlined infrastructure management, automation, and proven best practices delivered through EMC Exchange Solutions.

Many IT organizations may believe that outsourcing Exchange to Microsoft Office 365 is their low-cost option.  That is usually not true.  Earlier this year, Wikibon demonstrated for any organization with more than 300 Exchange seats, Office 365 is more expensive than well-managed on-premises Exchange deployments.  And, larger enterprises needing high-availability Exchange from Microsoft can expect to pay 47% more than high-availability in-house deployments.  Customers deploying converged infrastructure can realize even greater levels of economic benefit.  Furthermore, an well-known SLA can ensure you even greater cost controls as you grow. Similar cost benefits can be had over Google and many other hosted Exchange service providers.

Flexibility is central to many messaging architects. This means that you are not limited in your choice of add-on software.  Because you are not limited to the options of a service provider, you can add third-party messaging tools and deeper application integration.  This means that Exchange is not disconnected from other critical business applications and the organization is better able to integrate and expedite their business processes.

Finally, protection and availability are what your end users demand.  In a mobile-connected world, systems must be available constantly.  By owning and controlling Exchange infrastructure, you are assured that disaster recovery and business continuity meets your business requirements.  Privacy and security is managed in-house as opposed to trusting another organization.  High-performance and high-availability is assured as opposed to promised by others.

Some organization, however want to "get out of the email business entirely".  This means that they want to use email, they simply don't wish to run email.  Certainly, you have more Exchange hosting options than ever before, but these options are not limited to the list gathered by Gartner.  There are compelling, value-based arguments to deploy EMC Exchange Solutions on-premises, but some organizations will be dead-set on outsourcing Exchange.  For these companies, EMC Partners are trusted service providers.  These partners are distinguished by using the same EMC infrastructure and uphold these same value propositions around your customer’s critical Exchange environment.  Often times, these EMC Powered SPs can actually take your existing infrastructure and either manage it for you, or transfer it to their Co-located Data Centers and run it there.  It's not often that you get to have your cake, and eat it too.

When looking at your options for messaging, there are few reasons to settle for less than what you want.  EMC is the company who can help you achieve your messaging goals at the lowest cost, highest levels of trust, with unmatched flexibility.

The entire Microsoft Global Solutions Marketing (GSM) Team attended Microsoft TechEd 2014: 6 of us in total.  Another 10 EMCers attended from EMC PreSales and EMC Product Groups.  The following summary was created and compiled by the GSM team in a cooperative effort to understand and interpret Microsoft's Technical and Marketing content that we gathered from TechEd 2014 in Houston, Texas May 11-15, 2014.

 

Looking at what Microsoft was positioning to its customers at TechEd 2014, we have ascertained the following:

The main message this year for Microsoft's TechEd conference was "Cloud-First, Mobile-First". As part of IT transformation, Cloud strategy is the key for Microsoft's customers.  In Microsoft's terms, Cloud strategies will come with a variety of questions and decisions in order to achieve a successful deployment: build your HA and DR plans, management, security and education for your technical teams.


Cloud is the understood enabler for all things Mobile.  Therefore, Cloud-First, because Mobile is the most important aspect of an individual user's work style.  A mobile user needs a broad array of devices in order to accomplish the work-life balance he/she needs, and those devices need Cloud in order to be of value to their owners.  Educating customers came in two basic scenarios: 1) IT Infrastructure, and 2) End-User Capabilities.  Microsoft's curriculum for TechEd2014 was specifically tailored to IT Professionals who's main purpose was to meet the needs of a mobile workforce.


We categorized our findings into the four following topics:

  • Azure Public Cloud Services including Office 365: The Keynotes, breakout sessions, everything… was about Azure Cloud; Microsoft delivered a very clear cloud category definition: Public In-Cloud (Azure) and SP/On-Prem (Windows Azure Pack, System Center, Windows Server 2014/Hyper-V) which formed the whole hybrid cloud diagram from MS’s cloud vision.
    • New Azure product offerings are coming to market every quarter: Azure Remote App, Azure Site Recovery, Azure Express Route, Azure Compute-intensive "A8" & "A9" VMs that will provide a variety of tools to integrate, manage and orchestrate customer applications in the Azure Cloud.  According to Microsoft, more than 57% of Fortune 500 companies are using Azure, but at this point we don’t know which kind of applications or development are using in this environment.
    • Microsoft Public Cloud has been approved by European Union for: Office 365, Azure, Microsoft Dynamics CRM and Windows Intune.
    • VM Mobility with Live Migration SMB Live Migration, Live Storage Migration and Share Nothing Live Migration
    • Scaling for Mission Critical Workloads
      1. Highest level of performance for Microsoft workloads

      2. SAP is certified to run on Azure

      3. Oracle software supported on Windows server Hyper-V and Azure!
    • SharePoint Server SP1 and SharePoint Online offer a Hybrid Cloud Solution for your organization, using Single-On feature providing Directory Synchronization in the entire SharePoint solution.
  • Visual Studio is the central focus for all mobile development efforts: Microsoft is embracing choice (as is EMC).  While not using the "Agile" label for their developer framework, clearly Microsoft is skillfully responding to the radically improved development process that an Agile development framework offers.  Microsoft is leading with "choice" by allowing .NET developers a broad range of target devices, languages, and browser types.  In fact, the only choice Microsoft doesn't offer is whether or not you can use Visual Studio to create your applications in or Windows Server to run your applications on!  Microsoft has worked arduously over the past fifteen years to build-out its .NET development framework to include a broad array of mobile device targets and multiple browser types and vendors.  You can even write in a variety of languages ranging from C to Ruby.
  • Windows Server and the Microsoft Cloud OS:  Centered deeply on the capabilities of System Center to achieve the full potency of Windows and Azure, Microsoft presented a rich and functional operational plan to achieve the needs of a mobile workforce. 
  • SQL Server: Microsoft SQL Server 2014 bringing new functionalities delivering better performance in memory built-in Hybrid cloud solutions. Lots of interest and emphasis in Business Intelligence and Big Data; how to discover, manage, process and display data. Power BI and analytics managed by Office 365 for critical business applications.  The combination of SQL Always-On replica and Hyper-V Replica will cover different DR scenarios between SQL On premises and Azure. At this time they have to work to improve performance and reduce times in the replication, but the solution is very interesting.


So what about "the competition"?

This year VMware was on the Expo Floor — even awarded TOP VIRTUALIZATION at the show. And some breakout sessions had comparison slides, talking about the advantage of Hyper/System Center against VMware/vSphere/vCenter/vCAC, more about specs and features. Azure Cloud is a much grander diagram when compared against VMware’s cloud capabilities (on-prem only — no comparison were made to vCHS!). Azure is now becoming the key differentiator when talking about "the competition". Example here is the MSFT storage solution (storsimple + Azure), the DR solution (MASR to Azure), etc.To no one's surprise, very little was said about AWS or Google.  It was very clear that Microsoft Azure, backed by the capabilities of System Center, was an entirely different ballpark.


Unexpected Lessons from TechEd:

  1. Microsoft is openly supporting all  device types — they have given-up on attempting to rule the world by having Windows on every device and in every home.  With developer tools supporting every viable mobile device, Chrome, Mozilla/Firefox, even Safari, Microsoft he's learned that it will not win a world dominated by it's own products.
  2. EMC did not have a booth on the Expo Floor; customers were confused by this.  When told that EMC had enlisted its Federation-member, VMware, to exhibit on the Expo Floor, this confused them more...
  3. Cloud and Mobile are one thing.  With a customer-base that demands to be "constantly productive", Microsoft has determined a licensing model that allows them to bring Office to nearly any device.  iPads can now run Office, Web browsers can run Office, even Android phones run Office.  Microsoft has learned that Office is what people want to create documents with and it has learned that the device/mobility is more important than Office.  The value-add for any device is the ability to become productive with it — Office makes that happen.
  4. This year, MSFT focused on step-by-step deployment/migration to MS cloud. Many sessions described exact deployment steps; very straight forward. Microsoft introduced several key advantages regarding the choice of Azure Cloud: Azure Active Directory, Application development and deployment considerations; Azure RemoteApp (Desktop as a service) from mobility perspective; Microsoft Azure Site Recovery and other features for security, etc. The most important thing was that Microsoft had introduced its huge investment of the Azure Data Center all over the world, this is amazing data, showing MSFT’s ambition on its Cloud strategy and this investment is very convinced when selling Azure to customer.
  5. Lastly, it is difficult to ignore how much "bashing of SAN" went on during the week. It wasn’t until the second day that we managed to get through an entire session without the speaker making a leading comment about the "high cost" of SAN. There was even one session where the presenter said “…and if that is on EMC SAN, it is going to be pretty costly.”  Microsoft is attempting to teach its customers what Software Defined Storage is.  Microsoft is has usurped the term "Software Defined".  It is attempting to teach its customers that Microsoft's definition of Software Defined is complete and accurate.  The fact is that Microsoft is teaching customers that reducing the length and complexity of the IO path will increase performance and drive costs out. This lesson has little to do with Software Defined Storage, but we trust that customers already know that fact.

Conclusions:

It's been two weeks since TechEd began.  In that time we have discussed many aspects of Azure, System Center, and EMC's role in our customers' data centers.  Azure is evolving to meet Microsoft's customers' needs. It is an impressive collection of products and services.  It is also limited in what in can do, and what it allows its customers accomplish.

EMC leads with choice.  Giving customers choice is built into everything EMC does, thinks about, and presents.  Simply look at the vast number of storage choices that EMC brings to market, not to mention the number of options available to backup, replicate, restore, control, secure, and expose your data.  Look at the number of hypervisors EMC supports.  Most customers believe that choice is the reason that EMC is #1.  EMC expects CHOICE to dominate our market presence and we look forward to providing choice to you as you transform your data centers into the enabling technologies that your customers and consumers demand.

I was just asked if EMC Marketing had a competitive slide deck covering the differences between Windows File Services in Windows Server 2012R2 and the EMC File Portfolio of products.  Hmmm, I thought… What ARE the differences?  There are so many, I'm not sure where to start.  So let's do this: let's start with an old expression; "the mother of invention is necessity".  In this context, that means, "what was the initial set of requirements?"  Let's take a look at a potential list:

  1. Store "home" directories
  2. Store high-performance VHDx files in a Hyper-V Server Cluster
  3. Replicate advertising content (videos, PowerPoint docs, and graphics files used for placing ads in our magazines and web sites) to another site 120-miles away so they can use the unified file space
  4. Link a bunch of file servers together using DFS-R (Distributed File System - Replication) Services
  5. Store a single department's files in a single remote location
  6. Store user files so that they can access them on any mobile device anywhere in the world without using VPN

Ok… that's a diverse set of requirements.  So many use cases, but all of them using file servers to reach the goals.  What should we do?

Here's where it gets easy, but also gets interesting. By clearly defining what you need to accomplish, you can easily match needs to features or offerings.  For example: 1. Store Home Directories: how large are they?, are there capacity quotas?, are they scanned for corporate governance compliance (data that might contain credit card numbers)?, are the directories synchronized to laptops using My Documents redirection?  If all these are true, you may need the advanced features offered by an Isilon Scale Out file system. Check out Isilon Family - Big Data Storage, Scale-out NAS Storage - EMC

What about #2. Store high performance VHDx files for use in a Hyper-V Cluster?  Well, now we are in a completely different place, eh? All of a sudden we have concerns for performance, availability, clustering support, metadata handlers, snapshots, recoveries, potential replication concerns, we need to arrange VSS backups of individual files.  It's like we aren't talking about a file server anymore.  We are finding more and more customers clamoring for the advanced, scale-to-fit, extremely efficient, and trusted VNX Family - Unified Storage Hardware and Software for Midrange - EMC.

#3 - Replicated and shared file spaces have been a challenge for IT professionals since the dawn of IT.  Microsoft's Distributed File System Replication is an extremely evolved solution for highly specific use cases.  Sharing replicated file spaces is a tricky task, Windows Server 2012R2 delivers a unique and highly optimized set of tools to allow small sets a data to be replicated among sites efficiently.  There are complex setup steps, but Virtualizing your Windows File Servers can ease management headaches and reduce costs!  Our paper on Microsoft Private Clouds is great starting place for your journey. http://www.emc.com/collateral/whitepaper/h11228-management-integration-cloud-wp.pdf  Likewise, #4, specifically calls for DFS-R -- EMC technologies, when working WITH Microsoft technologies will lower costs, reduce downtime, reduce support issues, and allow you to reach your business goals faster.

#5 -- Store a single departments files in a single location? This is one of the requirements that can go in so many directions because it seems as if the requirement is simple.  The problem here is latent -- it doesn't present itself at first.  At first blush, you might think, "hey, no problem: stick a VM out at the remote office.  Back it up with Avamar - Backup and Recovery, Data Deduplication - EMC , and I'm done.  Maybe you are, maybe you're not.  What if the remote location has power issues, physical security issues, the staff constantly deletes files and needs point-in-time recoveries routinely?  The simple problem just consumed your week.  EMC knows that these little "ankle biter issues" are detailers for IT shops. Handling remote file usage has become the bane of so many IT shops and IT managers.  EMC sees that there is more to remote file access than placing a VM at a remote location; that's why we introduced leading technologies to assist you and your users get what they want when they want it.  My ol' pal Paul Galjan has posted an article to get you thinking about the possibilities: http://flippingbits.typepad.com/blog/2014/05/mobilize-sharepoint-with-syncplicity.html

#6.  Access anywhere files.  Oddly, sometimes Dropbox just isn't good enough.  That's why EMC launched Syncplicity.  Please take a moment to see all you can do for your increasing group of remote and mobile users. Features

The summary to all this is that File doesn't mean File Server anymore.  It means storage.  Every storage scenario is different and that's why EMC has a proud portfolio of offerings.  Not just services, not just products, not just software.  EMC has become the answer to an ever increasing number of questions and scenarios.  Thanks for reading.

This is part one… part two next week!

SQL Server: Where is Flash most effective?  Everywhere!

 

I am, what is commonly referred to as, a “reluctant blogger”.  I’m the guy that drives my Marketing department crazy.  They keep telling me, “Jim! You know stuff!  You need to share it!”  It’s not that I disagree, it’s that I’m fully aware that the Internet is loaded with content created by people who know lots of stuff. This time, however, I have information that I know won’t be somewhere else on the Internet.  This time, I can offer content that will make a difference. I’ll begin by telling you a bunch of stuff that you already know… In an effort to establish a baseline, a starting place…

SQL Server is a complex platform, but not so complex that we can’t talk about big concepts.  I’ll attempt to take the significant areas of SQL Server and explain the details that matter most to this discussion.  In this post, I’ll explain three things:

  1. Three various common types of SQL Server workloads,
  2. How SQL Server processes data, uses cache, the log stream, and the database files themselves, and
  3. How EMC Xtrem technology ties into SQL Server and into each various workload.

Three typical SQL Server workloads:

  1. OLTP – this is your run-of-the-mill on-line transaction processing database.  The kind that stores transactions like orders you place at Amazon or Zappo’s or Geeks.com or even Home Depot.  The transaction contains an order number, the items you purchased, and how you paid – maybe even your credit card number.  A typical OLTP database will Read 80% of the time, Write 20% of the time, and IO sizes of 64KB to the database with 8KB Writes to the log file.
  2. DSS/BI/Reporting – most SQL admins will resent the fact that I just globbed all three of those “clearly different” workloads into one category, but stick with me for a minute: all three have generally similar workload patterns.  All three ask for heavy use of Indexes to increase query speeds, all three generally receive updates in a dribble or data stream manner. Occasionally, there will be a nightly load of data but the newly imported/incoming dataset is generally small compared to the overall size of the existing data.  The most important element these three share is the use of TEMPDB. They all make extensive use of outer joins to answer queries.  This use of joins dictates the use of TEMPDB.  As we know, there is only one TEMPDB per SQL Instance, so it’s very difficult (read “impossible”) to dedicate TEMPDB resources to an individual database.
  3. OLAP – These are SQL Server applications that build cubes on a nightly basis – these cubes care a collection of pre-analyzed data that allows business units to make decisions based on “what the data shows”.  These cubes are not ad-hoc queries – they are a predefined “view” of the data with multiple dimensions.  They allow a business unit analyst to quickly answer questions such as “How many white plastic forks did we sell in Dayton after we ran the flyer in the Sunday newspaper where we advertised a sale on picnic blankets?”. Cube builds are hard.  Depending on how they are executed, they can read every bit of data in an entire database more than once.  A cleverly engineered cube build will run in twelve minutes.  A complex set of data can cause a SQL Server to work on the problem for several hours.
  4. Now let’s talk about how SQL processes data and why these various tasks create singular issues for each SQL server they run on.  SQL Server has three basic parts that make it work:
    1. The SQL Cache – this is the space in SQL memory set-aside to hold actual database data that SQL Server is using to fulfill queries, update rows/fields, delete data elements, and generally go about its business.  SQL Cache is finite – limited ultimately by the available RAM in the SQL Server OS.  (This will change in SQL Server 2014… teaser…).  The important concept is that everything a SQL Server does goes through the cache and the cache is limited.ACID.png
    2. The Log Writer – every SQL Server instance has one (and only one) log writer. Regardless of the number of databases and log files within an instance, each SQL Server instance has and uses a single log writer.  What does this mean?  Log writes are not parallelized.  They are serialized.  They happen one at a time.  The Log Writer is arguably the essence of data integrity and database consistency. Clearly, ACID (Atomic, Consistent, Isolated, and Discrete) uses the order, finite nature, and time element aspects of log writes to fulfill its mission.  In order for SQL to “continue processing”, each data element that denotes a change to “the database”, must result in a posting of the operation to the log file.  Everything goes into the log file.  As the operation and the data hit the log file, the data/metadata is considered “hardened”.  As data is replicated between servers, it is the write-order consistency that allows ACID to hold.  Without write-order integrity, ACID breaks.
    3. The database itself: comprised of .mdf files and .ndf files.  As the dBA adds files for the storage of database pages, the pages (in eight-page extents) are stored in a round-robin fashion amongst the database files. The dBA or even storage adin is free to place these database files on a single volume or separate them across a multitude of volumes.  Each volume can then be placed on a single storage device or may separate storage devices: on one disk, one RAID group, one storage array, or many of those – even across separate datacenters.  As long as the database server can access the volume (even SMB!), the file can be placed there.  Pages are fetched from the database files as the database server fails to find the pages it needs in SQL cache.  If the page is not in cache, it goes to the file that is supposed to have the page – a disk request is issued, and the page is returned (along with seven other pages in the extent).  Data is stored (written) into the files when SQL Server has “Spare cycles”.  The Lazy Writer grabs processor slices when they come available and “commits” the data that was hardened into the log file into the database files.  The very interesting part of this process is that SQL Server aggregates the writes before it makes the disk requests.  That means that SQL Server is very clever at assembling and sorting and consolidating the data that it “flushes-out” to the disk.  Detailed analysis shows that SQL Server’s writer optimizes data-to-disk operations by as much as 3.5X.  This means that the raw IO that SQL Server “wants” to write to disk as it comes into the database is actually reduced by a whopping 67%!  The evidence of this is seen in black and white when you look at the relationship between Primary SQL Server write IOs and the write IOs of its Database Mirroring partner.  The secondary server will display between 2 and 3.5 as many IOPS to the disk volumes as observed on the primary server.  The reason is that the Mirror, the secondary, cannot aggregate database writes as the primary does.  It is so fixated on getting the data into the database, that the aggregation code has been disabled!
    4. TEMPdB – Now, I said there were only three parts of a SQL Server – and there are – but number 4, here, is added for clarification.  TEMPdb exists just once for every SQL Server Instance – keep in mind that any given Server OS installation can contain a number of SQL Instances – this is simply the number of times the SQL engine is running on a given server OS. Why would you want more than one? Clustering is one reason – you may want to allow SQL databases to run on all nodes of your OS cluster.  Another reason is TEMPdB.  As was just mentioned, a single TEMPdB is allowed to operate for any single instance of SQL Server.  TEMPdB can be distributed among multiple database files, just like any database.  Logging occurs just like any other database, but not exactly the same way.  For example, you cannot back up TEMPdB. There is no reason to, and you cannot restore it either.  TEMPdB exists for SQL Server to store temporary data while it’s working on the final answer to queries that require external joins.  The data from these table-joins lands in TEMPdB and is then used again as the source for the question that was actually asked.  After the query is completed, TEMPdB ditches all the data that was fetched into it.  It is not uncommon for Ad-Hoc queries to make extensive use of TEMPdB. It is also not uncommon to see a 40/60 Read/Write ratio, where more than half of the data that SQL Server fetches is discarded – only to be fetched again for another Ad-Hoc query.RoleOfTEMPDB.png

Ok…  Now, I realize that many dBAs who are reading this are thinking, “Dude, that is the most simplified, stupid, made-for-idiots explanation ever.”  And perhaps it is, but, as simple and basic as the explanation is, it’s a solid and factual representation of what happens inside a SQL Server.

With this information, we can start to understand how accelerating each of these areas can help a SQL Server go faster.  The idea is that “Performance” is not inexpensive. Performance can be achieved in a number of ways – beginning with cleverly assembled queries, indexes, and insert operations.  Addressing ways to make “good” code run faster, though, we can begin to apply various hardware technologies to SQL Server architectures to improve their speed.

There are three significant Xtrem technologies at this time (more coming in 2014 and 2015!).  The three are:

  1. Xtrem SF (Server Flash) – a hardware device that fits directly into a server.  The device is a multi-lane PCIe card that brings ultra-fast storage to a single server.  By itself, it is High Speed Storage.  It is available in a multitude of capacities and at least two NAND cell types.
  2. XtremSW Cache (Software) – device driver and management interface that allows a single host to store previously fetched data on a local storage device (it does not need to be XtremSF! But there are management benefits if it is…).  The caching software allows a SQL Server (or any application: Exchange, Sharepoint, Dynamics, etc.) to fetch a recently used and discarded data page without going all the way out to the rotating disk. The intention is that a relatively large dataset can reside on a local flash device (200ns!) but be protected by enterprise-class storage (3-6ms).  It’s a matter of bringing milliseconds down to nanoseconds.
  3. XtremIO (Storage Device) – this is an array device specifically built from the ground-up to benefit from high-speed disk drives.  Everything about the XtremIO device is based on the operating aspects of SSDs – but is not specifically built for SSDs.  If you can look into a crystal ball and see “the next thing” that’s faster than an SSD, XtremIO will use it.  It can absorb data like nobody’s business.  And the more you pump at it, the faster it seems to go.

Based on these ten elements, let’s begin to create linkages. So, just for review: we have three types of workloads, three types of SQL Server components (with a special mention for a fourth, TEMPdB), and three Xtrem technologies.  Where do each of these technologies fit each of these various workloads and/or SQL components? And why?

Combo number 1) OLTP makes heavy use of a small number of data elements as it processes orders/transactions.  These data elements could be a customer list, a parts list, a table of ZIP codes, shipping costs based on the number of shipping zones that a package needs to traverse on its way to a customer’s location.  The idea is that the SQL Server needs to have these lists handy at all times.  The traditional approach to keeping these data elements “close” is to increase server RAM.  This is a great plan, but can have two potential downsides:

1) RAM is ultimately finite – if I virtualize, I am taking and potentially reserving pooled RAM for a single server and I might actually exhaust all of the RAM that a physical server has available. There are many Intel-based servers that “max-out” at 96GB or even 192GB of physical RAM.  If I have a large number of VMs “fighting” for that RAM, I may run out. And,

2) assigning RAM to a VM is not all upside. As I add RAM, I also need to facilitate paging file capacity on physical disk.  The more I add, the more physical hard disk capacity I burn.  Now, disk is “cheap” so this is not a huge concern, but it needs to be addressed as I add server RAM.

 

 

 

Part 2 next week!

Not often discussed, but very interesting… Microsoft Windows Azure offers, among all sorts of other things, the ability to create and join a DC to your corporate domain!  You can imagine the possibilities.

Install a replica domain controller in Windows Azure

You can imagine the benefits… the question becomes, "is Azure the best place for me to keep one of my Active Directory Domain Controllers?".  There are a multitude of local and national EMC-Powered Infrastructure-as-a-Service Providers who can offer not only DC's in the cloud, but all sorts of other customizable options as well.  A quick look at EMC Cloud Service Providers - Americas.

Filter Blog

By date:
By tag: