Condusiv Technologies Blog

Condusiv Technologies Blog

Blogging @Condusiv

The Condusiv blog shares insight into the issues surrounding system and application performance—and how I/O optimization software is breaking new ground in solving those issues.

MS-SQL Performance Woes Worsen Despite Massive Hardware Investments, Survey Confirms

by Brian Morin 26. February 2018 06:36

We just completed our 4th annual I/O Performance Survey from 870 IT Professionals. This is one of the industry’s largest studies of its kind and reveals the latest trends in applications driving performance demands, and how IT Professionals are responding.

The survey results consist of 20 detailed questions designed to identify the impact of I/O growth in the modern IT environment. In addition to multiple choice questions, the survey included optional open responses, allowing a respondent to provide commentary on why they selected a particular answer. All of the individual responses have been compiled across 68 pages to help readers dive deeply into any question.

The most interesting year over year take away when comparing the 2018 report against the 2017 report is that IT Professionals cite worsening performance from applications sitting on MS-SQL despite larger investments in multi-core servers and all-flash storage. More than 1 in 4 respondents cite staff and customer complaints regarding sluggish SQL queries and reports. No Condusiv customer was included in the survey.

Below is a word cloud representing hundreds of answers to visually show the applications IT Pros are having the most trouble to support from a performance standpoint.

To see what has either grown or shrunk, below is the same word cloud from 2017 responses.

 

The full report can be accessed by visiting http://learn.condusiv.com/IO-Performance-Study.html

As much as organizations continue to reactively respond to performance challenges by purchasing expensive new server and storage hardware, our V-locity® I/O reduction software offers a far more efficient path by guaranteeing to solve the toughest application performance challenges on I/O intensive systems like MS-SQL. This means organizations are able to offload 50% of I/O traffic from storage that is nothing but mere noise chewing up IOPS and dampening performance. As soon as we open up 50% of that bandwidth to storage, sluggish performance disappears and now there’s far more storage IOPS to be used for other things.

To learn more about how V-locity I/O reduction software eliminates the two big I/O inefficiencies in a virtual environment see 2-min Video: Condusiv I/O Reduction Software Overview

 

How to Achieve 2X Faster MS-SQL Applications

by Brian Morin 8. November 2017 05:31

By following the best practices outlined here, we can virtually guarantee a 2X or faster boost in your MS-SQL performance with our I/O reduction software.

  1) Don’t just run our I/O reduction software on the SQL Server instances but also on the application servers that run on top of MS-SQL

- It’s not just SQL performance that needs improvement, but the associated application servers that communicate with SQL. Our software will eliminate a minimum of 30-40% of the I/O traffic from those systems.

  2) Run our I/O reduction software on all the non-SQL systems on the same host/hypervisor

- Sometimes a customer is only concerned with improving their SQL performance, so they only install our I/O reduction software on the SQL Server instances. Keep in mind, the other VMs on the same host/hypervisor are interfering with the performance of your SQL instances due to chatty I/O that is contending for the same storage resources. Our software eliminates a minimum of 30-40% of the I/O traffic from those systems that is completely unnecessary, so they don’t interfere with your SQL performance.

- Any customer that is on the core or host pricing model is able to deploy the software to an unlimited number of guest machines on the same host. If you are on per system pricing, consider migrating to a host model if your VM density is 7 or greater.

  3) Cap MS-SQL memory usage, leaving at least 8GB left over

- Perhaps the largest SQL inefficiency is related to how it uses memory. SQL is a memory hog. It takes everything you give it then does very little with it to actually boost performance, which is why customers see such big gains with our software when memory has been tuned properly. If SQL is left uncapped, our software will not see any memory available to be used for cache, so only our write optimization engine will be in effect. Moreover, most DB admins cap SQL, leaving 4GB for the OS to use according to Microsoft’s own best practice.

- However, when using our software, it is best to begin by capping SQL a little more aggressively by leaving 8GB. That will give plenty to the OS, and whatever is leftover as idle will be dynamically leveraged by our software for cache. If 4GB is available to be used as cache by our software, we commonly see customers achieve 50% cache hit rates. It doesn’t take much capacity for our software to drive big gains.

  4) Consider adding more memory to the SQL Server

- Some customers will add more memory then limit SQL memory usage to what it was using originally, which leaves the extra RAM left over for our software to use as cache. This also alleviates concerns about capping SQL aggressively if you feel that it may result in the application being memory starved. The very best place to start is by adding 16GB of RAM then monitor the dashboard to see the impact. The software can use up to 128GB of DRAM. Those customers who are generous in this approach on read-heavy applications get into otherworldly kind of gains far beyond 2X with >90% of I/O served from DRAM. Remember, DRAM is 15X faster than SSD and sits next to the CPU.

  5) Monitor the dashboard for a 50% reduction in I/O traffic to storage

- When our dashboard shows a 50% reduction in I/O to storage, that’s when you know you have properly tuned your system to be in the range of 2X faster gains to the user, barring any network congestion issues or delivery issues.

- As much as capping SQL at 8GB may be a good place to start, it may not always get you to the desired 50% I/O reduction number. Monitor the dashboard to see how much I/O is being offloaded and simply tweak memory usage by capping SQL a little more aggressively. If you feel you may be memory constrained already, then add a little more memory, so you can cap more aggressively. For every 1-2GB of memory added, another 10-25% of read traffic will be offloaded.

 

Not a customer yet? Download a free trial of Condusiv I/O reduction software and apply these best practice steps at www.condusiv.com/try

 

New Dashboard Finally Answers the Big Question

by Brian Morin 25. October 2017 04:38

After surveying thousands of IT professionals, we’ve found that the vast majority agree that Windows performance degrades over time – they just don’t agree on how much. Unbeknown to most is what the problem actually is, which is I/O degradation as the size of writes and reads become excessively smaller than they should. This inefficiency is akin to moving a gallon of water across a room with dixie cups instead of a single gallon jug. Even if you have all-flash storage and can move those dixie cups quickly, you are still not processing data nearly as fast as you could.

In the same surveys, we’ve also found that the vast majority of IT professionals are aware of the performance penalty of the “I/O blender” effect in a virtual environment, which is the mixing and randomizing of I/O streams from the disparate virtual machines on the same host. What they don’t agree on is how much. And, they are not aware of how the issue is compounded by Windows write inefficiencies.

Now that the latest Condusiv in-product dashboard has been deployed across thousands of customer systems who have upgraded their Condusiv I/O reduction software to the latest version, customers are getting their first-ever granular view into what I/O reduction software is doing for their systems in terms of seeing the exact percentage and number of read and write I/O operations eliminated from storage and how much I/O time that saves any given system or group of systems. Ultimately, it’s a picture into the size of the problem – all the I/O traffic that is mere noise – all the unnecessary I/O that dampens system performance.

In our surveys, we found IT professionals all over the map on the size of the performance penalty from inefficiencies. Some are quite positive the performance penalty is no more than 10%. More put that range at 20%. Most put it at 30%. Then it dips back down with fewer believing a 40% penalty with the fewest throwing the dart at 50%.

As it turns out, our latest version has been able to drop a pin on that.

There are variances that move the extent of the penalty on any given workload such as system configuration and/or workload behavior. Some systems might be memory constrained, some workloads might be too light to matter, etc.

However, after thousands of installs over the last several months, we see a very consistent range on the vast majority of systems in which 30-40% of all I/O traffic is being offloaded from underlying storage with our software. Not only does that represent an immediate performance boost for users, but it also means 30-40% of I/O headroom is handed back to the storage subsystem that can now use those IOPS for other things.

The biggest factor to consider is the 30-40% improvement number represents systems where memory has not been increased beyond the typical configuration that most administrators use. Customers who offload 50% or more of I/O traffic from storage are the ones with read heavy workloads who beef up memory server-side to get more from the software. For every additional 1-2GB of memory added, another 10-25% of read traffic is offloaded. Some customers are more aggressive and leverage as much memory as possible server-side to offload 90% or more I/O traffic on read-heavy applications.

As expensive as new all-flash systems are, how much sense does it make to pay for all those IOPS only to allow 30-40% of those IOPS to be chewed up by unnecessary, noisy I/O? By addressing the two biggest penalties that dampen performance (Windows write inefficiencies compounded by the “I/O blender” effect), Condusiv I/O reduction software ensures optimal performance and protects the CapEx investments made into servers and storage by extending their useful life.

Tags:

Disruption, Application Performance, IOPS | virtualization | V-Locity

The Revolution of Our Technology

by Rick Cadruvi, Chief Architect 18. October 2017 12:38

I chose to use the word “Revolution” instead of “Evolution” because, with all due modesty, our patented technology has been more a series of leaps to stay ahead of performance-crushing bottlenecks. After all, our company purpose as stated by our Founder, Craig Jensen, is:

“The purpose of our company is to provide computer technology that enormously increases

the production and income of an area.”

We have always been about improving your production. We know your systems are not about having really cool hardware but rather about maximizing your organization’s production. Our passion has been about eliminating the stops, slows and stalls to your application performance and instead, to jack up that performance and give you headroom for expansion. Now, most of you know us by our reputation for Diskeeper®. What you probably don’t know about us is our leadership in system performance software.

We’ve been at this for 35 years with a laser focus. As an example, for years hard drives were the common storage technology and they were slow and limited in size, so we invented numerous File System Optimization technologies such as Defragmentation, I-FAAST®1 and Directory Consolidation to remove the barriers to getting at data quickly. As drive sizes grew, we added new technologies and jettisoned those that no longer gave bang for the buck. Technologies like InvisiTasking® were invented to help maximize overall system performance, while removing bottlenecks.

As SSDs began to emerge, we worked with several OEMs to take advantage of SSDs to dramatically reduce data access times as well as reducing the time it took to boot systems and resume from hibernate. We created technologies to improve SSD longevity and even worked with manufacturers on hybrid drives, providing hinting information, so their drive performance and endurance would be world class.

As storage arrays were emerging we created technologies to allow them to better utilize storage resources and pre-stage space for future use. We also created technologies targeting performance issues related to file system inefficiencies without negatively affecting storage array technologies like snapshots.

When virtualization was emerging, we could see the coming VM resource contention issues that would materialize. We used that insight to create file system optimization technologies to deal with those issues before anyone coined the phrase “I/O Blender Effect”.

We have been doing caching for a very long time2. We have always targeted removal of the I/Os that get in your applications path to data along with satisfying the data from cache that delivers performance improvements of 50-300% or more. Our goal was not caching your application specific data, but rather to make sure your application could access its data much faster. That’s why our unique caching technology has been used by leading OEMs.

Our RAM-based caching solutions include dynamic memory allocation schemes to use resources that would otherwise be idle to maximize overall system performance. When you need those resources, we give them back. When they are idle, we make use of them without your having to adjust anything for the best achievable performance. “Set It and Forget It®” is our trademark for good reason.

We know that staying ahead of the problems you face now, with a clear understanding of what will limit your production in 3 to 5 years, is the best way we can realize our company purpose and help you maximize your production and thus your profitability. We take seriously having a clear vision of where your problems are now and where they will be in the future. As new hardware and software technologies roll out, we will be there removing the new barriers to your performance then, just as we do now.

1. I-FAAST stands for Intelligent File Access Acceleration Sequencing Technology, a technology designed to take advantage of different performing regions on storage to allow your hottest data to be retrieved in the fastest time.

2. If I can personally brag, I’ve created numerous caching solutions over a period of 40 years.

University of Illinois Doubles SQL and Oracle Performance on All-Flash Arrays with V-locity® I/O Reduction Software

by Brian Morin 19. May 2017 11:00

The University of Illinois, had already deployed an all-flash Dell Compellent storage array to support their hardest hitting application, AssetWorks AiM, that runs on Oracle. After a year in service, performance began to erode due to growth in users and overall workload. Other hard hitting MS-SQL applications supported by hybrid arrays were suffering performance degradation as well.

The one common denominator is that all these applications ran on Windows servers – Windows Server 2012R2.

“As we learned through this exercise with Condusiv’s V-locity I/O reduction software, we were getting hit really hard by thousands of excessively small, tiny writes and reads that dampened performance significantly,” said Greg Landes, Manager of Systems Services. “Everything was just slower due to Windows Server write inefficiencies that break writes down to be much smaller than they need to be, and forces the all-flash SAN to process far more I/O operations than necessary for any given workload.”

Landes continued, “When you have a dump truck but are only filling it a shovelful at a time before sending it on, you’re not getting near the payload you should get with each trip. That’s the exact effect we were getting with a surplus of unnecessarily small, fractured writes and subsequent reads, and it was really hurting our storage performance, even though we had a really fast ‘dump truck.’ We had no idea how much this was hurting us until we tried V-locity to address the root-cause problem to get more payload with every write. When you no longer have to process three small, fractured writes for something that only needs one write and a single I/O operation, everything is just faster.”

When testing the before-and-after effect on their production system, Greg and his team first measured without V-locity. It took 4 hours and 3 minutes to process 1.57TB of data, requiring 13,910,568 I/O operations from storage. After V-locity, the same system processed 2.1TB of data in 1 hour and 6 minutes, while only needing to process 2,702,479 I/O operations from underlying storage. “We processed half a terabyte more in a quarter of the time,” said Landes.

“Not only did V-locity dramatically help our write-heavy MS-SQL and Oracle Servers by increasing performance 50–100% on several workloads, we saw even bigger gains on our read heavy applications that could take advantage of V-locity’s patented DRAM caching engine, that put our idle, unused memory to good use. Since we had provisioned adequate memory for these I/O-intensive systems, we were well positioned to get the most from V-locity.”

“We thought we were getting the most performance possible from our systems, but it wasn’t until we used V-locity that we realized how inefficient these systems really are if you’re not addressing the root cause performance issues related to the I/O profile from Windows servers. By solving the issue of small, fractured, random I/O, we’ve been able to increase the efficiency of our infrastructure and, ultimately, our people,” said Landes.

Tags:

General | SAN | Success Stories | virtualization | V-Locity | Windows Server 2012

RecentComments

Comment RSS

Month List

Calendar

<<  July 2018  >>
MoTuWeThFrSaSu
2526272829301
2345678
9101112131415
16171819202122
23242526272829
303112345

View posts in large calendar