Condusiv Technologies Blog

Condusiv Technologies Blog

Blogging @Condusiv

The Condusiv blog shares insight into the issues surrounding system and application performance—and how I/O optimization software is breaking new ground in solving those issues.

Financial Sector Battered by Rising Compliance Costs

by Dawn Richcreek 15. August 2018 08:39

Finance is already an outlier in terms of IT costs. The industry devotes 10.5% of total revenue to IT—and on average, each financial industry IT staffer supports only 15.7 users, the fewest of any industry.

All over the world, financial services companies are facing skyrocketing compliance costs. Almost half the respondents to a recent Accenture survey of compliance officers in 13 countries said they expected 10% to 20% increases, and nearly one in five are expecting increases of more than 20%.

Much of this is driven by international banking regulations. At the beginning of this year, the Common Reporting Standard went into effect. An anti-tax-evasion measure signed by 142 countries, the CRS requires financial institutions to provide detailed account information to the home governments of virtually every sizeable depositor.

Just to keep things exciting, the U.S. government hasn’t signed on to CRS; instead we require banks doing business with Americans to comply with the Foreign Account Tax Compliance Act of 2010. Which requires—surprise, surprise—pretty much the same thing as CRS, but reported differently.

And these are just two examples of the compliance burden the financial sector must deal with. Efficiently, and within a budget. In a recent interview by ValueWalk entitled “Compliance Costs Soaring for Financial Institutions,” Condusiv® CEO Jim D’Arezzo said, “Financial firms must find a path to more sustainable compliance costs.”

Speaking to the site’s audience (ValueWalk is a site focused on hedge funds, large asset managers, and value investing) D’Arezzo noted that finance is already an outlier in terms of IT costs. The industry devotes 10.5% of total revenue to IT, more than government, healthcare, retail, or anybody else. It’s also an outlier in terms of IT staff load; on average, each financial industry IT staffer supports only 15.7 users, the fewest of any industry. (Government averages 37.8 users per IT staff employee.)

To ease these difficulties, D’Arezzo recommends that the financial industry consider advanced technologies that provide cost-effective ways to enhance overall system performance. “The only way financial services companies will be able to meet the compliance demands being placed on them, and at the same time meet their efficiency and profitability targets, will be to improve the efficiency of their existing capacity—especially as regards I/O reduction.”

At Condusiv, that’s our business. We’ve seen users of our I/O reduction software solutions increase the capability of their storage and servers, including SQL servers, by 30% to 50% or more. In some cases, we’ve seen results as high as 10X initial performance—without the need to purchase a single box of new hardware.

If you’re interested in working with a firm that can reduce your two biggest silent killers of SQL performance, request a demo with an I/O performance specialist now.

 

For an explanation of why your heaviest workloads are only processing half the throughput they should from VM to storage, view this short video.

 

Windows is still Windows Whether in the Cloud, on Hyperconverged or All-flash

by Brian Morin 5. June 2018 04:43

Let me start by stating two facts – facts that I will substantiate if you continue to the end.

Fact #1 - Windows suffers from severe write inefficiencies that dampen overall performance. The holy grail question as to how severe is answered below.

Fact #2, Windows is still Windows whether running in the cloud, on hyperconverged systems, all-flash storage, or all three. Before you jump to the real-world examples below, let me first explain why.

No matter where you run Windows and no matter what kind of storage environment you run Windows on, Windows still penalizes optimal performance due to severe write inefficiencies in the hand-off of data to storage. Files are always broken down to be excessively smaller than they need to be. Since each piece means a dedicated I/O operation to process as a write or read, this means an enormous amount of noisy, unnecessary I/O traffic is chewing up precious IOPS, eroding throughput, and causing everything to run slower despite how many IOPS are at your disposal.

How much slower?

Now that the latest version of our I/O reduction software is being run across tens of thousands of servers and hundreds of thousands of PCs, we can empirically point out that no matter what kind of environment Windows is running on, there is always 30-40% of I/O traffic that is nothing but mere noise stealing resources and robbing optimal performance.

Yes, there are edge cases in which the inefficiency is as little as 10% but also other edge cases where the inefficiency is upwards of 70%. That being said, the median range is solidly in the 30-40% range and it has absolutely nothing to do with the backend media whether spindle, flash, hybrid, hyperconverged, cloud, or local storage.

Even if running Windows on an all-flash hyperconverged system, SAN or cloud environment with low latency and high IOPS, if the I/O profile isn’t addressed by our I/O reduction software to ensure large, clean, contiguous writes and reads, then 30-40% more IOPS will always be required for any given workload, which adds up to unnecessarily giving away 30-40% of the IOPS you paid for while slowing the completion of every job and query by the same amount.

So what’s going on here? Why is this happening and how?

First of all, the behavior of Windows when it comes to processing write and read input/output (I/O) operations is identical despite the storage backend whether local or network or media despite spindles or flash. This is because Windows only ever sees a virtual disk - the logical disk within the file system itself. The OS is abstracted from the physical layer entirely. Windows doesn’t know and doesn’t care if the underlying storage is a local disk or SSD, an array full of SSDs, hyperconverged, or cloud. In the mind of the OS, the logical disk IS the physical disk when, in fact, it’s just a reference architecture. In the case of enterprise storage, the underlying storage controllers manage where the data physically lives. However, no storage device can dictate to Windows how to write (and subsequently read) in the most efficient manner possible.

This is why many enterprise storage controllers have their own proprietary algorithms to “clean up” the mess Windows gives it by either buffering or coalescing files on a dedicated SSD or NVRAM tier or physically move pieces of the same file to line up sequentially, which does nothing for the first penalized write nor several penalized reads after as the algorithm first needs to identify a continued pattern before moving blocks. As much as storage controller optimization helps, it’s a far cry from an actual solution because it doesn’t solve the source of the larger root cause problem - even with backend storage controller optimizations, Windows will still make the underlying server to storage architecture execute many more I/O operations than are required to write and subsequently read a file, and every extra I/O required takes a measure of time in the same way that four partially loaded dump trucks will take longer to deliver the full load versus one fully loaded dump truck. It bears repeating - no storage device can dictate to Windows how to best write and read files for the healthiest I/O profile that delivers optimum performance because only Windows controls how files are written to the logical disk. And that singular action is what determines the I/O density (or lack of) from server to storage.

The reason this is occurring is because there are no APIs that exist between the Windows OS and underlying storage system whereby free space at the logical layer can be intelligently synced and consolidated with the physical layer without change block movement that would otherwise wear out SSDs and trigger copy-on-write activity that would blow up storage services like replication, thin provisioning, and more.

This means Windows has no choice but to choose the next available allocation at the logical disk layer within the file systems itself instead of choosing the BEST allocation to write and subsequently read a file.

The problem is that the next available allocation is only ever the right size on day 1 on a freshly formatted NTFS volume. But as time goes on and files are written and erased and re-written and extended and many temporary files are quickly created and erased, that means the next available space is never the right size. So, when Windows is trying to write a 1MB file but the next available allocation at the logical disk layer is 4K, it will fill that 4K, split the file, generate another I/O operation, look for the next available allocation, fill, split, and rinse and repeat until the file is fully written, and your I/O profile is cluttered with split I/Os. The result is an I/O degradation of excessively small writes and reads that penalizes performance with a “death by a thousand cuts” scenario.

It’s for this reason, over 2,500 small, midsized, and large enterprises have deployed our I/O reduction software to eliminate all that noisy I/O robbing performance by addressing the root cause problem. Since Condusiv software sits at the storage driver level, our purview is able to supply patented intelligence to the Windows OS, enabling it to choose the BEST allocation for any file instead of the next available, which is never the right size. This ensures the healthiest I/O profile possible for maximum storage performance on every write and read. Above and beyond that benefit, our DRAM read caching engine (the same engine OEM’d by 9 of the top 10 PC manufacturers), eliminates hot reads from traversing the full stack from storage by serving it straight from idle, available DRAM. Customers who add anywhere to 4GB-16GB of memory to key systems with a read bias to get more from that engine, will offload 50-80% of all reads from storage, saving even more precious storage IOPS while serving from DRAM which is 15X faster than SSD. Those who need the most performance possible or simply need to free up more storage IOPS will max our 128GB threshold and offload 90-99% of reads from storage.

Let’s look at some real-world examples from customers.

Here is VDI in AWS shared by Curt Hapner (CIO, Altenloh Brinck & Co.). 63% of read traffic is being offloaded from underlying storage and 33% of write I/O operations. He was getting sluggish VDI performance, so he bumped up memory slightly on all instances to get more power from our software and the sluggishness disappeared.

Here is an Epicor ERP with SQL backend in AWS from Altenloh Brinck & Co. 39% of reads are being eliminated along with 44% of writes to boost the performance and efficiency of their most mission critical system.

 

Here’s from one of the largest federal branches in Washington running Windows servers on an all-flash Nutanix. 45% of reads are being offloaded and 38% of write traffic.

 

Here is a spreadsheet compilation of different systems from one of the largest hospitality and event companies in Europe who run their workloads in Azure. The extraction of the dashboard data into the CSV shows not just the percentage of read and write traffic offloaded from storage but how much I/O capacity our software is handing back to their Azure instances.

 

To illustrate we use the software here at Condusiv on our own systems, this dashboard screenshot is from our own Chief Architect (Rick Cadruvi), who uses Diskeeper on his SSD-powered PC. You can see him share his own production data in the recent “live demo” webinar on V-locity 7.0 - https://youtu.be/Zn2QGxBHUzs

As you can see, 50% of reads are offloaded from his local SSD while 42% of writes operations have been saved by displacing small, fractured files with large, clean contiguous files. Not only is that extending the life of his SSD by reducing write amplification, but he has saved over 6 days of I/O time in the last month.

 

Finally, regarding all-flash SAN storage systems, the full data is in this case study with the University of Illinois who used Condusiv I/O reduction software to more than double the performance of SQL and Oracle sitting on their all-flash arrays: http://learn.condusiv.com/rs/246-QKS-770/images/CS_University-Illinois.pdf?utm_campaign=CS_UnivIll_Case_Study

For a free trial, visit http://learn.condusiv.com/Try-V-locity.html. For best results, bump up memory on key systems if you can and make sure to install the software on all the VMs on the same host. If you have more than 10 VMs, you may want to Contact Us for SE assistance in spinning up our centralized management console to push everything at once – a 20-min exercise and no reboot required.

Please visit www.condusiv.com/v-locity for more than 20 case studies on how our I/O reduction software doubled the performance of mission critical applications like MS-SQL for customers of various environments.

New! Diskeeper 16 Guarantees “Faster than New” Performance for Physical Servers and PCs

by Brian Morin 26. September 2016 09:56

The world’s most popular defragmentation software for physical servers and PCs makes “defrag” a thing of the past and delivers “faster than new” performance by dynamically caching hot reads with idle DRAM.  As a result, Diskeeper® 16 guarantees to solve the toughest application performance issues on physical servers like MS-SQL and guarantees to fix sluggish PCs with faster than new performance or your money back for 90 days – no questions asked.

The market is still catching up to the fact that Diskeeper’s newest patented engine no longer “defrags” but rather proactively eliminates fragmentation with large, sequential writes from Windows to underlying HDDs, SSDs, and SAN storage systems. This eliminates the “death by a thousand cuts” scenario of small, tiny writes and reads that inflates I/Os per second, robs throughput, and shortens the lifespan of HDDs and SSDs alike. However, the biggest new announcement has to do with the addition of DRAM caching – putting idle DRAM to good use by serving hot reads without memory contention or resource starvation.

“Diskeeper 16 with DRAM caching served over 50% of my reads from DRAM and eliminated over 30% of write traffic by preventing fragmentation. Now everything is more responsive!” - David Bruce, Managing Partner, David Bruce & Associates

“Diskeeper 16 with DRAM caching doubled our throughput, so we could backup in half the time.  Our Dell Rapid Recovery backup server is running smoother than ever.” - Curtis Jackson, Network Admin, School City of Hammond

“WOW! Watch it go! I have 44GB of memory in the physical server and Diskeeper is using around 20GB of it to cache!! I can’t imagine having a server without it! Diskeeper 16 is a vastly improved version of Diskeeper!” - Andy Vabulas, Vabulas Enterprises

“Our Symantec app running on a physical server has been notoriously slow for as long as I can remember, but since adding Diskeeper 16 it has improved significantly.” Josh Currier, Network Infrastructure Manager, Munters Corporation

 “With Diskeeper 16 I can tell my workstation is more responsive with no lag or any type of hesitation. Truly SMART Technology.” - William Krasulak, Systems/Network Admin, Nacci Printing, Inc.

“Our most I/O intensive applications on physical servers needed some help, so we installed Diskeeper 16 with DRAM caching and were amazed by the performance boost!” - Victor Grandmaiter, IT Director, Fort Bend Central Appraisal District

“Diskeeper eliminated 32% of my write traffic by preventing fragmentation and cached 64% of my read traffic within idle memory. This saved my workstation over 20 hours in I/O time after 24 days of testing!” - Lou Goodreau, IT Manager, New England Fishery

“Installed Diskeeper 16 on our worst performing physical servers running ERP with a SQL database and saw an immediate 50% boost!" - Hamid Bouhassoune, Systems Engineer, Global Skincare Company

A top New York clothing brand tried Diskeeper 16 with DRAM caching on their physical servers and saw backup times with Veeam and Backup Exec drop by more than half!

Before Diskeeper Install:

8/7, 10GB, 14MB/s, 1:38

8/8, 11 GB, 13MB/s, 1:54

After Diskeeper Install:

          8/12, 13GB, 21MB/s, 1:30

        8/13, 14GB, 30MB/s, 0:58

        8/14, 13GB, 33MB/s, 0:55

        8/15, 11GB, 36MB/s, 0:44

        8/19, 17GB, 30MB/s, 1:06

 

A Large Illinois Non-Profit tested Diskeeper 16 with DRAM caching on Windows 2012R2 physical servers running CRM and accounting software with a MS-SQL backend. Note – these improvements were almost exclusively from Diskeeper 16’s write optimization engine since idle memory was not available to initiate the new caching engine.

 

See a screenshot of the new dashboard reporting that shows “time saved” from using Diskeeper 16 to eliminate fragmentation and cache reads with idle DRAM.

 

Try Diskeeper 16 with DRAM caching for 30-days -> 

 

 

 

Teaser: Coming Soon! Intelligent Caching and Fragmentation Prevention = IO Heaven

by Brian Morin 19. September 2016 04:53

Sometimes the performance of physical servers, PCs and laptops slows to a crawl. No matter what you do, it takes half an eternity to open some files. It’s tied into the architecture of the Windows operating system. The OS becomes progressively slower the longer it is used and the more it is burdened with added software and large volumes of data.

In the old days, the solution was easy – defragment the hard drive. However, many production servers can’t be taken offline to defragment, and many laptops only have solid state drives (SSDs) that don’t submit to defragmentation. So is there any hope?

Condusiv has solved these dilemmas in the soon to be released version of Diskeeper®. With over 100 million licenses sold, Diskeeper has been the undisputed leader for decades when it comes to keeping Windows systems fragment free and performing well. And with Diskeeper 16 coming out soon, feedback from Beta testers is that it goes way beyond a mere incremental release with a few added frills, bells and whistles. Instead, the consensus among them is that it is a “next generation” release that goes well beyond just keeping Windows systems running like new but actually boosts performance faster than new.

How is this being achieved? The company had been perfecting two technologies within its portfolio and is now bringing them together – fragmentation prevention and DRAM caching.

On the one side, the idea is that you prevent fragmentation before data is written to a production server. This is a lifesaver for IT administrators who need to immediately boost the performance of critical applications like MS-SQL running on physical servers. Diskeeper keeps systems running optimally with its patented fragmentation prevention engine that ensures large, clean, contiguous writes from Windows, eliminating the small, tiny writes that rob performance with “death by a thousand cuts” by inflating IOPS and stealing throughput.

But that’s only the half of it.  A little known fact about Condusiv is that it is also a world leader in caching. In addition to their incredible work on Diskeeper, the Condusiv development team has evolved a unique DRAM caching approach that has been implemented via OEM partners for several years. So popular has this technology become that the company has sold over 5 million caching licenses that have been tied to ultrabooks but now is being made available commercially.

Soon to be released Diskeeper 16’s DRAM caching electrifies performance:

·         Benchmark tests show MS-SQL workload performance boosts of up to 6X

·         An average of 40% latency reduction across hundreds of servers

·         No hint of memory contention or resource starvation

·         Fleets of laptops suddenly running like a dream

·         PCMark MS Office productivity tests show an increase of 73% on Windows 10 machines

·         Huge leaps in SSD write speed and extended SSD lifespan

·         Solves even the worst performing physical servers or Windows PCs backed by a money-back guarantee.

Could it be, then, that there really is hope to get PCs and physicals servers to be running faster than new?

 

You’ll have to wait until Diskeeper 16 is unveiled to hear the full story. 

The Gartner Cool Vendor Report in Storage Technologies: Vanity or Value

by Robert Woolery 22. April 2014 08:58

We all like lists that rank who is cool, best in class or top score in a buyer’s guide. Every year, Gartner releases their prized "Cool Vendor" selection. But is it just vanity for the vendor selected or is there actual, tangible value to the prospective customer that makes you care?

We believe one significant difference about the Cool Vendor Report compared to other reports is Gartner does a deep-dive examination of compelling vendors across the technology landscape, then upon selecting their "cool vendors" for the year, they reveal their analysis, why the vendor is cool, challenges the vendor faces and who should care.

Of all the technology companies on the landscape, Gartner chose to highlight four this year in the area of storage technologies, providing research into their innovative products and/or services.

When we were brainstorming our flagship product V-locity, we spoke to hundreds customers and we heard a common theme – performance problems in virtual environments whereby users were buying lots of hardware to solve an inherent software problem per the "I/O blender" effect.

As we dug in, a clearer picture emerged. We've become conditioned to medicating performance problems with hardware. And why not? In the past, performance gains were growing by 4X to 8X every ten years. Hardware was cheap. The price performance continued to improve every two years. And inertia, doing business as usual was low risk – buy more hardware because we’ve always done it that way and the financial folks understand the approach.

When we evangelize the problem of I/O growing faster than hardware could cost-effectively keep up and the need for a software only approach to easily cure it, we found the problem and solution resonated with many customers – webinar attendance ranged from 400 to 2,000 attendees. And while we are fast approaching 2,000 corporate installations, there are still customers wondering why they have not heard of the I/O problem we solve and our innovative way to solve it. They want some proof.

This is where the Gartner Cool Vendor report is helpful to IT users and their organizations. The reports help focus and reduce the learning curve on the relevant problems in IT, the innovative companies that warrant further investigation and highlight interesting new products and services that address issues in emerging trends.

The Cool Vendor Report can be read in the time it takes to have a cup of coffee. Not surprisingly, the Cool Vendor Reports are one of two top reports Gartner clients download.

Now for our vanity plug, Condusiv is listed in the Cool Vendor Report titled "Cool Vendors in Storage Technologies, 2014." This is usually only available to Gartner clients, but we paid for distribution rights so you could read it for free. Download Gartner's Cool Vendors in Storage Technologies Report

RecentComments

Comment RSS

Month List

Calendar

<<  August 2018  >>
MoTuWeThFrSaSu
303112345
6789101112
13141516171819
20212223242526
272829303112
3456789

View posts in large calendar