Condusiv Technologies Blog

Condusiv Technologies Blog

Blogging @Condusiv

The Condusiv blog shares insight into the issues surrounding system and application performance—and how I/O optimization software is breaking new ground in solving those issues.

Everything You Need to Know about SSDs and Fragmentation in 5 Minutes

by Howard Butler 17. November 2016 05:42

When reading articles, blogs, and forums posted by well-respected (or at least well intentioned people) on the subject of fragmentation and SSDs, many make statements about how (1) SSDs don’t fragment, or (2) there’s no moving parts, so no problem, or (3) an SSD is so fast, why bother? We all know and agree SSDs shouldn’t be “defragmented” since that shortens lifespan, so is there a problem after all?

The truth of the matter is that applications running on Windows do not talk directly to the storage device.  Data is referenced as an abstracted layer of logical clusters rather than physical track/sectors or specific NAND-flash memory cells.  Before a storage unit (HDD or SSD) can be recognized by Windows, a file system must be prepared for the volume.  This takes place when the volume is formatted and in most cases is set with a 4KB cluster size.  The cluster size is the smallest unit of space that can be allocated.  Too large of a cluster size results in wasted space due to over allocation for the actual data needed.  Too small of a cluster size causes many file extents or fragments.  After formatting is complete and when a volume is first written to, most all of the free space is in just one or two very large sections.  Over the course of time as files of various sizes are written, modified, re-written, copied, and deleted, the size of individual sections of free space as seen from the NTFS logical file system point of view becomes smaller and smaller.  I have seen both HDD and SSD storage devices with over 3 million free space extents.  Since Windows lacks file size intelligence when writing a file, it never chooses the best allocation at the logical layer, only the next available – even if the next available is 4KB. That means 128K worth of data could wind up with 32 extents or fragments, each being 4KB in size. Therefore SSDs do fragment at the logical Windows NTFS file system level.  This happens not as a function of the storage media, but of the design of the file system.

Let’s examine how this impacts performance.  Each extent of a file requires its own separate I/O request. In the example above, that means 32 I/O operations for a file that could have taken a single I/O if Windows was smarter about managing free space and finding the best logical clusters instead of the next available. Since I/O takes a measurable amount of time to complete, the issue we’re talking about here related to SSDs has to do with an I/O overhead issue.

Even with no moving parts and multi-channel I/O capability, the more I/O requests needed to complete a given workload, the longer it is going to take your SSD to access the data.  This performance loss occurs on initial file creation and carries forward with each subsequent read of the same data.  But wait… the performance loss doesn’t stop there.  Once data is written to a memory cell on an SSD and later the file space is marked for deletion, it must first be erased before new data can be written to that memory cell.  This is a rather time consuming process and individual memory cells cannot be individually erased, but instead a group of adjacent memory cells (referred to as a page) are processed together.  Unfortunately, some of those memory cells may still contain valuable data and this information must first be copied to a different set of memory cells before the memory cell page (group of memory cells) can be erased and made ready to accept the new data.  This is known as Write Amplification.  This is one of the reasons why writes are so much slower than reads on an SSD.  Another unique problem associated with SSDs is that each memory cell has a limited number of times that a memory cell can be written to before that memory cell is no longer usable.  When too many memory cells are considered invalid the whole unit becomes unusable.  While TRIM, wear leveling technologies, and garbage collection routines have been developed to help with this behavior, they are not able to run in real-time and therefore are only playing catch-up instead of being focused on the kind of preventative measures that are needed the most.  In fact, these advanced technologies offered by SSD manufacturers (and within Windows) do not prevent or reverse the effects of file and free space fragmentation at the NTFS file system level.

The only way to eliminate this surplus of small, tiny writes and reads that (1) chew up performance and (2) shorten lifespan from all the wear and tear is by taking a preventative approach that makes Windows “smarter” about how it writes files and manages free space, so more payload is delivered with every I/O operation. That’s exactly why more users run Condusiv’s Diskeeper® (for physical servers and workstations) or V-locity® (for virtual servers) on systems with SSD storage. For anyone who questions how much value this approach adds to their systems, the easiest way to find out is by downloading a free 30-day trial and watch the “time saved” dashboard for yourself. Since the fastest I/O is the one you don’t have to write, Condusiv software understands exactly how much time is saved by eliminating multiple, fractured writes with fewer, larger contiguous writes. It even has an additional feature to cache reads from idle, available DRAM (15X faster than SSD), which further offloads I/O bandwidth to SSD storage. Especially for businesses with many users accessing a multitude of applications across hundreds or thousands of servers, the time savings are enormous.

 

ATTO Benchmark Results with and without Diskeeper 16 running on a 120GB Samsung SSD Pro 840. The read data caching shows a 10X improvement in read performance.

First-ever “Time Saved” Dashboard = Holy Grail for ROI

by Brian Morin 2. November 2016 10:03

If you’ve ever wondered about the exact business value that Condusiv® I/O reduction software provides to your systems, the latest “time saved” reporting does exactly that.

Prior to V-locity® v6.2 for virtual servers and Diskeeper® 16 for physical servers and endpoints, customers would conduct expansive before/after tests to extract the intrinsic performance value, but struggled to extract the ongoing business benefit over time. This has been especially true during annual maintenance renewal cycles when key stakeholders need to be “re-sold” to allocate budget for ongoing maintenance, or push new licenses to new servers.

The number one request from customers has been to better understand the ongoing business benefit of I/O reduction in terms that are easily relatable to senior management and makes justifying the ROI painless. This “holy grail” search on part of our engineering team has led to the industry’s first-ever “time saved” dashboard for an I/O optimization software platform.

When Condusiv software proactively eliminates the surplus of small, fractured writes and reads and ensures more “payload” with every I/O operation, the net effect is fewer write and read operations for any given workload, which saves time. When Condusiv software caches hot reads within idle, available DRAM, the net effect is fewer reads traversing the full stack down to storage and back, which saves time.

In terms of benefits, the new dashboard shows:

    1. How many write I/Os are eliminated by ensuring large, clean, contiguous writes from Windows

    2. How many read I/Os are cached from idle DRAM

    3. What percentage of write and read traffic is offloaded from underlying SSD or HDD storage

    4. Most importantly – the dashboard relates I/O reduction to the business benefit of … “time saved”

This reporting approach makes the software fully transparent on the type of benefit being delivered to any individual system or groups of systems. Since the software itself sits within the Windows operating system, it is aware of latency to storage and understands just how much time is saved by serving an I/O from DRAM instead of the underlying SSD or HDD. And, most importantly, since the fastest I/O is the one you don’t have to write, Condusiv software understands how much time is saved by eliminating multiple small, fractured writes with fewer, larger contiguous writes.  

Have you ever wondered how much time V-locity will save a VDI deployment? Or an application supported by all-flash? Or a Hyperconverged environment? Rather than wonder, just install a 30-day version of the software and monitor the “time saved” dashboard to find out. Benefits are fully transparent and easily quantified.

Have you ever needed to justify Diskeeper’s endpoint solution across a fleet of corporate laptops with SSDs? Now you can see the “time saved” on individual systems or all systems and quantify the cost of labor against the number of hours that Diskeeper saved in I/O time across any time period. The “no brainer” benefit will be immediately obvious.

Customers will be pleasantly surprised to find out the latest dashboard doesn’t just show granular benefits but also granular performance metrics and other important information to assist with memory tuning. See the avg., min, and max of idle memory used for cache over any time period (even by the hour) to make quick assessments on which systems could use more memory to take better advantage of the caching engine for greater application performance. Customers have found if they can maintain at least 2GB used for cache, that's where they begin to get into the sweet spot of what the product can do. If even more can be maintained to establish a tier-0 cache strategy, performance rises even further. Systems with at least 4GB idle for cache will invariably serve 60% of reads or more. 

 

 

       Lou Goodreau, IT Manager, New England Fishery

      “32% of my write traffic has been eliminated and 64% of my read traffic has been cached within idle memory. This saved over 20 hours in I/O time after 24 days of testing!”

       David Bruce, Managing Partner, David Bruce & Associates

                                    “Over 50% of my reads are now served from DRAM and over 30% of write traffic has

                                   been eliminated by ensuring large, contiguous writes. Now everything is more

                                   responsive!"

 

New! Diskeeper 16 Guarantees “Faster than New” Performance for Physical Servers and PCs

by Brian Morin 26. September 2016 09:56

The world’s most popular defragmentation software for physical servers and PCs makes “defrag” a thing of the past and delivers “faster than new” performance by dynamically caching hot reads with idle DRAM.  As a result, Diskeeper® 16 guarantees to solve the toughest application performance issues on physical servers like MS-SQL and guarantees to fix sluggish PCs with faster than new performance or your money back for 90 days – no questions asked.

The market is still catching up to the fact that Diskeeper’s newest patented engine no longer “defrags” but rather proactively eliminates fragmentation with large, sequential writes from Windows to underlying HDDs, SSDs, and SAN storage systems. This eliminates the “death by a thousand cuts” scenario of small, tiny writes and reads that inflates I/Os per second, robs throughput, and shortens the lifespan of HDDs and SSDs alike. However, the biggest new announcement has to do with the addition of DRAM caching – putting idle DRAM to good use by serving hot reads without memory contention or resource starvation.

“Diskeeper 16 with DRAM caching served over 50% of my reads from DRAM and eliminated over 30% of write traffic by preventing fragmentation. Now everything is more responsive!” - David Bruce, Managing Partner, David Bruce & Associates

“Diskeeper 16 with DRAM caching doubled our throughput, so we could backup in half the time.  Our Dell Rapid Recovery backup server is running smoother than ever.” - Curtis Jackson, Network Admin, School City of Hammond

“WOW! Watch it go! I have 44GB of memory in the physical server and Diskeeper is using around 20GB of it to cache!! I can’t imagine having a server without it! Diskeeper 16 is a vastly improved version of Diskeeper!” - Andy Vabulas, Vabulas Enterprises

“Our Symantec app running on a physical server has been notoriously slow for as long as I can remember, but since adding Diskeeper 16 it has improved significantly.” Josh Currier, Network Infrastructure Manager, Munters Corporation

 “With Diskeeper 16 I can tell my workstation is more responsive with no lag or any type of hesitation. Truly SMART Technology.” - William Krasulak, Systems/Network Admin, Nacci Printing, Inc.

“Our most I/O intensive applications on physical servers needed some help, so we installed Diskeeper 16 with DRAM caching and were amazed by the performance boost!” - Victor Grandmaiter, IT Director, Fort Bend Central Appraisal District

“Diskeeper eliminated 32% of my write traffic by preventing fragmentation and cached 64% of my read traffic within idle memory. This saved my workstation over 20 hours in I/O time after 24 days of testing!” - Lou Goodreau, IT Manager, New England Fishery

“Installed Diskeeper 16 on our worst performing physical servers running ERP with a SQL database and saw an immediate 50% boost!" - Hamid Bouhassoune, Systems Engineer, Global Skincare Company

A top New York clothing brand tried Diskeeper 16 with DRAM caching on their physical servers and saw backup times with Veeam and Backup Exec drop by more than half!

Before Diskeeper Install:

8/7, 10GB, 14MB/s, 1:38

8/8, 11 GB, 13MB/s, 1:54

After Diskeeper Install:

          8/12, 13GB, 21MB/s, 1:30

        8/13, 14GB, 30MB/s, 0:58

        8/14, 13GB, 33MB/s, 0:55

        8/15, 11GB, 36MB/s, 0:44

        8/19, 17GB, 30MB/s, 1:06

 

A Large Illinois Non-Profit tested Diskeeper 16 with DRAM caching on Windows 2012R2 physical servers running CRM and accounting software with a MS-SQL backend. Note – these improvements were almost exclusively from Diskeeper 16’s write optimization engine since idle memory was not available to initiate the new caching engine.

 

See a screenshot of the new dashboard reporting that shows “time saved” from using Diskeeper 16 to eliminate fragmentation and cache reads with idle DRAM.

 

Try Diskeeper 16 with DRAM caching for 30-days -> 

 

 

 

Is Fragmentation Robbing SAN Performance?

by Brian Morin 16. March 2015 09:39

This month Condusiv® announced the most significant development in the Diskeeper® product line to date – expanding our patented fragmentation prevention capabilities beyond server local storage or direct-attached storage (DAS) to now include Storage Area Networks, making it the industry's first real-time fragmentation solution for SAN storage.

Typically, as soon as we mention "fragmentation" and "SAN" in the same sentence, an 800 pound gorilla walks into the room and we’re met with some resistance as there is an assumption that RAID controllers and technologies within the SAN mitigate the problem of fragmentation at the physical layer.

As much as SAN technologies do a good job of managing blocks at the physical layer, the real problem why SAN performance degrades over time has nothing to do with the physical disk layer but rather fragmentation that is inherent to the Windows file system at the logical disk software layer.

In a SAN environment, the physical layer is abstracted from the Windows OS, so Windows doesn't even see the physical layer at all – that’s the SAN's job. Windows references the logical disk layer at the file system level.

Fragmentation is inherent to the fabric of Windows. When Windows writes a file, it is not aware of the size of the file or file extension, so it will break that file apart into multiple pieces with each piece allocated to its own address at the logical disk layer. Therefore, the logical disk becomes fragmented BEFORE the SAN even receives the data.

How does a fragmented logical disk create performance problems? Unnecessary IOPS (input/output operations per sec). If Windows sees a file existing as 20 separate pieces at the logical disk level, it will execute 20 separate I/O commands to process the whole file. That’s a lot of unnecessary I/O overhead to the server and, particularly, a lot of unnecessary IOPS to the underlying SAN for every write and subsequent read.

Diskeeper 15 Server prevents fragmentation from occurring in the first place at the file system layer. That means Windows will write files in a more contiguous or sequential fashion to the logical disk. Instead of breaking a file into 20 pieces that needs 20 separate I/O operations for every write and subsequent read, it will write that file in a more contiguous fashion so only minimal I/O is required.

Perhaps the best way to illustrate this is with a traffic analogy. Bottlenecks occur where freeways intersect. You could say the problem is not enough lanes (throughput) or the cars are too slow (IOPS), but we’re saying the easiest problem to solve is the fact of only one person per car!

By eliminating the Windows I/O "tax" at the source, organizations achieve greater I/O density, improved throughput, and less I/O required for any given workload – by simply filling the “car” with more people. Fragmentation prevention at the top of the technology stack ultimately means systems can process more data in less time.

When openBench Labs tested Diskeeper Server, they found throughput increased 1.3X. That is, from 75.1 MB/sec to 100 MB/sec. A manufacturing company saw their I/O density increase from 24KB to 45KB. This eliminated 400,000 I/Os per server per day, and the IT Director said it "eliminated any lag during peak operation."

Many administrators are led to believe they need to buy more IOPS to improve storage performance when in fact, the Windows I/O tax has made them more IOP dependent than they need to be because much of their workload is fractured I/O. By writing files in a more sequential fashion, the number of I/Os required to process a GB of data drops significantly so more data can be processed in less time.

Keep in mind, this is not just true for SANs with HDDs but SSDs as well. In a SAN environment, the Windows OS isn’t aware of the physical layer or storage media being used. The I/O overhead from splitting files apart at the logical disk means just as many unnecessary IOPS to SSD as HDD. SSD is only processing that inefficient I/O more quickly than a hard disk drive.

Diskeeper 15 Server is not a "defrag" utility. It doesn’t compete with the SAN for management of the physical layer by instructing the RAID controllers on the how to manage the data. Diskeeper’s patented proactive approach is the perfect complement to a SAN by ensuring only productive I/O is processed from server to storage to keep physical servers and SAN storage running like new.

With organizations spending tens of thousands of dollars on server and storage hardware and even hundreds of thousands of dollars on large SSD deployments, why give 25% or more performance over to fragmentation when it can be prevented altogether for a mere $400 per physical server at our lowest volume tier?

Try Diskeeper 15 Server for 30 Days ->

The Biggest Missed Culprit in SQL Performance Troubleshooting

by Brian Morin 18. February 2015 09:53

"We didn't know how much of our SQL performance was being dampened by the nasty 'I/O blender' effect….."

As it turned out, it was HALF. 

That's right. Their systems were processing HALF as many MB/sec than they should due to the noise of all their VM workloads meeting and mixing at the point of the hypervisor. The first thing the "I/O blender" effect does is tax throughput, so your application performance becomes far more dependent on storage IOPS than it needs to be.

Read the full story how I.B.I.S., Inc. doubled performance of their CRM and ERP by eliminating the I/O
blender effect ->
 

So what is the "I/O blender" effect and how is it taxing application performance? 

The "I/O blender" effect is a phenomena specific to a virtual server environment where the I/O streams from disparate VMs are "funneled" together at the point of the hypervisor before sending out to storage a very random I/O stream that penalizes overall application performance.

Every organization that has virtualized has experienced this pain. They virtualized their applications only to discover mounting I/O pressure on the backend storage infrastructure. This was the unintended consequence of virtualization. Organizations save costs on the compute layer via virtualization only to trade those savings to backend storage where a forklift upgrade is necessary to handle the new random I/O demand.

In the case of I.B.I.S., Inc., their IT Director wanted to look into this problem a little further to see what could be done before reactively buying more storage hardware for improved performance.

"We wanted to try V-locity® I/O reduction software first to see if it could tackle the root cause problem as advertised at the VM level where I/O originates," said Kevin Schmidt, IT Director.

As much as IT departments lack monitoring tools that show exactly how much performance is dampened by the "I/O blender" effect, V-locity comes with an embedded benchmark to give a before/after picture of I/O reduction and demonstrate how much performance is improved by combatting this problem at the Windows operating system layer.

As it turned out, I.B.I.S., Inc.'s heaviest SQL workloads saw a 120% improvement in data throughput. Before V-locity, it took 82,000 I/Os to process 1GB of data. After V-locity, that number was cut to 29,000 I/Os per GB. Due to the increase in I/O density, instead of taking .78 minutes to process 1GB, it now only takes .36 minutes.

"Since we're no longer dealing with so many small split I/Os and random I/O streams, V-locity has enabled our CRM and ERP systems to process twice the amount of data in the same amount of time. The best part is that we didn't have to spend a single dime on expensive new hardware to get that performance," said Schmidt.

Read the full case study ->

Tags: , , , ,

Disruption, Application Performance, IOPS | virtualization | V-Locity

RecentComments

Comment RSS

Month List

Calendar

<<  December 2018  >>
MoTuWeThFrSaSu
262728293012
3456789
10111213141516
17181920212223
24252627282930
31123456

View posts in large calendar