Condusiv Technologies Blog

Condusiv Technologies Blog

Blogging @Condusiv

The Condusiv blog shares insight into the issues surrounding system and application performance—and how I/O optimization software is breaking new ground in solving those issues.

University of Illinois Doubles SQL and Oracle Performance on All-Flash Arrays with V-locity® I/O Reduction Software

by Brian Morin 19. May 2017 11:00

The University of Illinois, had already deployed an all-flash Dell Compellent storage array to support their hardest hitting application, AssetWorks AiM, that runs on Oracle. After a year in service, performance began to erode due to growth in users and overall workload. Other hard hitting MS-SQL applications supported by hybrid arrays were suffering performance degradation as well.

The one common denominator is that all these applications ran on Windows servers – Windows Server 2012R2.

“As we learned through this exercise with Condusiv’s V-locity I/O reduction software, we were getting hit really hard by thousands of excessively small, tiny writes and reads that dampened performance significantly,” said Greg Landes, Manager of Systems Services. “Everything was just slower due to Windows Server write inefficiencies that break writes down to be much smaller than they need to be, and forces the all-flash SAN to process far more I/O operations than necessary for any given workload.”

Landes continued, “When you have a dump truck but are only filling it a shovelful at a time before sending it on, you’re not getting near the payload you should get with each trip. That’s the exact effect we were getting with a surplus of unnecessarily small, fractured writes and subsequent reads, and it was really hurting our storage performance, even though we had a really fast ‘dump truck.’ We had no idea how much this was hurting us until we tried V-locity to address the root-cause problem to get more payload with every write. When you no longer have to process three small, fractured writes for something that only needs one write and a single I/O operation, everything is just faster.”

When testing the before-and-after effect on their production system, Greg and his team first measured without V-locity. It took 4 hours and 3 minutes to process 1.57TB of data, requiring 13,910,568 I/O operations from storage. After V-locity, the same system processed 2.1TB of data in 1 hour and 6 minutes, while only needing to process 2,702,479 I/O operations from underlying storage. “We processed half a terabyte more in a quarter of the time,” said Landes.

“Not only did V-locity dramatically help our write-heavy MS-SQL and Oracle Servers by increasing performance 50–100% on several workloads, we saw even bigger gains on our read heavy applications that could take advantage of V-locity’s patented DRAM caching engine, that put our idle, unused memory to good use. Since we had provisioned adequate memory for these I/O-intensive systems, we were well positioned to get the most from V-locity.”

“We thought we were getting the most performance possible from our systems, but it wasn’t until we used V-locity that we realized how inefficient these systems really are if you’re not addressing the root cause performance issues related to the I/O profile from Windows servers. By solving the issue of small, fractured, random I/O, we’ve been able to increase the efficiency of our infrastructure and, ultimately, our people,” said Landes.

Tags:

General | SAN | Success Stories | virtualization | V-Locity | Windows Server 2012

I Have Backups and Snapshots, So Why Do Condusiv Customers Use Undelete®?

by James Fields, Director of Customer Support 10. May 2017 09:57

Backups and snapshots are used by enterprises to recover data sets in the event of system failure. But how about individual files on file servers? Often times they are accidentally deleted by users or overwritten.

Backups and snapshots can still be used to retrieve those files but that can be akin to finding a needle in a haystack and laborious to recover. Which backup contains the most recent version? Was the file created or modified before the last backup or snapshot took place? In that case, the backup or snapshot isn’t of any help.

Condusiv customers use Undelete as a first line of defense for data protection on file servers to keep administrators from digging through backups and snapshots for individual files or folders. If any user accidentally deletes a file over a network share, it goes into Undelete’s recycle bin. This ensures real-time protection of all files on a file server that can be quickly and immediately recovered. Many organizations use Undelete for their HelpDesk team to recover individual files instead of tasking IT staff with the tedious task of accessing backups for a single file.

Moreover, Undelete keeps prior versions of MS Office documents, so if your CEO accidentally overwrites his PowerPoint presentation, you can always recover prior versions of saved files that share the same file name. One of the features admins like most about Undelete is the ability to see who deleted a file, when it was deleted, and who created the file.  This is especially useful if you are concerned with the possible nefarious activities of staff. 

Last month, 53 Undelete customers participated in an Undelete product survey and told us what they like the most, and here are some of their answers:

“Ease of restoring files deleted from network locations without having to use backups.”

“We have backups on daily basis, but nothing that keeps the deleted network files available and reported on who deleted. Undelete covers this issue.”

“We needed a product that provided "Recycle Bin" functionality for network shares.  Undelete Server goes one step further and even tracks revisions. Great product.”

“Sometimes, human errors occur and files get overwritten or deleted, which means losing several hours of work since the last regular backup. We need to be able to instantly recover deleted files - this is not possible with scheduled backups or VSS.”

“Versioning control on modified or overwritten files.”

“Much quicker and simpler than reverting to any BACKUP/RESTORE software. Plus, our HelpDesk can use it.”

“On a number of occasions, users will "lose" or delete files. It is simpler to use Undelete than scour through backups.”

“Close the time gap between an incident and the last regular backup.”

“HelpDesk needed this tool to offload menial requests from IT staff to dig through backups for one file”

 

James Fields | DIRECTOR OF CUSTOMER SUPPORT

Tags:

Data Protection | File Recovery | General | Windows 7 | Windows 8 | Windows Server 2012

How Can I/O Reduction Software Guarantee to Solve the Toughest Performance Problems?

by Brian Morin 14. January 2017 01:00

The #1 request I’ve been getting from customers is a white board video that succinctly explains the two silent killers of VM performance and how our I/O reduction guarantees to solve performance problems, so applications run perfectly on every Windows server.

Expensive backend storage upgrades should ONLY take place when needing more capacity – not more performance. Anytime I tell someone our I/O reduction software guarantees to solve their toughest performance problems…the very first response is invariably the same…HOW? Not only have I answered this question hundreds of times, our own customers find themselves answering this question repeatedly to other team members or new hires.

To make this easier, I’ve answered it all here in this 10-min White Board Video ->, or you can continue reading.

 Most of us have been upgrading hardware to get more performance ever since we can remember. It’s become so engrained, it’s often times the ONLY approach we think of when needing a performance upgrade.

For many organizations, they don’t necessarily need a performance boost on EVERY application, but they need it on one or two I/O intensive applications. To throw a new all-flash array or new hybrid array at a performance problem ends up being the most expensive and disruptive way to solve a performance problem when all you have to do is the same thing thousands of our customers have done: simply try our I/O reduction software on any Windows server and watch the application run at least 50% faster and in many cases 2X-10X faster.

Most IT professionals are unaware of the fact that as great as virtualization has been for server efficiency, the one downside is how it adds complexity to the data path. On top of that, Windows doesn’t play well in a virtual environment (or any environment where it is abstracted from the physical layer). This means I/O characteristics that are a lot smaller, more fractured and more random than they need to be – the perfect trifecta for bad storage performance.

This “death by a thousand cuts” scenario means systems are processing workloads about 50% slower than they should. Condusiv’s I/O reduction software solves this problem by displacing many small tiny writes and reads with large, clean contiguous writes and reads. As huge as that patented engine is for our customers, it’s not the only thing we’re doing to make applications run smoothly. Performance is further electrified by establishing a tier-0 caching strategy - automatically using idle, available memory to serve hot reads. This is the same battle-tested technology that has been OEM’d by some of the largest out there – Dell, Lenovo, HP, SanDisk, Western Digital, just to name a few.

Although we might be most known for our first patented engine that solves Windows write inefficiencies to HDDs or SSDs, more and more customers are discovering just how important our patented DRAM caching engine is. If any customer can maintain even just 4GB of available memory to be used for cache, they most often see cache hit rates in the range of 50%. That means serving data out of DRAM, which is 15X faster than SSD and opens up even more precious bandwidth to and from storage for everything else. Other customers who really need to crank up performance are simply provisioning more memory on those systems and seeing >90% cache hit rates.

See all this and more described in the latest Condusiv I/O Reduction White Board video that explains eeevvvveeerything you need to know about the problem, how we solve it, and the typical results that should be expected in the time it takes you to drink a cup of coffee. So go get a cup of coffee, sit back, relax, and see how we can solve your toughest performance problems – guaranteed.

 

Overview of How We Derive Storage I/O Time Saved

by Rick Cadruvi, VP Engineering 11. January 2017 01:00

The latest versions of V-locity® (for virtual servers) and Diskeeper® (for physical servers and PCs) both contain built-in dashboards that show the exact benefit of the product to any one system or group of systems by showing how much and what percentage of read/write traffic is offloaded from storage and how much “I/O Time” that saves.

To understand the computation on “I/O Time Saved,” in its simplest form, the formula is essentially:

       Storage I/O Time Saved = Total I/Os Eliminated * Average I/O Response Time

In essence, if you take Total I/Os Eliminated from the dashboard Benefits screen and multiply it times the average latency from the I/O Performance dashboard screen, you will generally end up in the ballpark of the “I/O Time Saved.”

I/O counts and I/O times are accumulated on a per I/O basis. Every I/O that goes to storage is timed using Windows High Performance Counters for accuracy.  That timing is from when the I/O is sent down the stack until it comes back up. In essence we time I/O response time (IORT) or latency that the application sees, not the storage device.  We also track reads and writes separately as they impact the storage “I/O Time Saved” differently.

The data is accumulated and calculated during periods of time rather than across the entire reporting period. In the long term, that period of time ends up being hourly. Very active I/O periods will have longer IORTs and therefore the amount of I/O storage time saved per I/O eliminated will likely be greater than during relatively light periods. 

If there is a high queue depth, the IORT we time will be larger than the per I/O storage IORT.  We look at the effective IORT the application would see rather than the time the underlying storage takes to process any single I/O.  After all, the user only cares about how long the application took to process an I/O he/she requested, not how long a HDD or SSD took for any single I/O when it got around to processing it.

Let’s talk for a moment about storage “I/O Time Saved” versus clock time because they are not the same and our technologies can, in some cases, save far more storage I/O time than clock time.

If all storage I/O was sequential for the entire instance of the operating system, then the maximum amount of storage “I/O Time Saved” would be the amount of time since installation, and you would expect it to be considerably less as we are unlikely to eliminate ALL I/Os. And you might expect some idle time. Of course, applications do not do pure sequential I/O.  Modern applications are almost always multi-threaded and most computer systems are running multiple applications or instances of them at the same time.  Also, other operations are happening on the system outside of the primary application.  Think of Outlook running in the background while you do some other work on your system. Outlook is constantly receiving updated data.  Windows is also processing lots of I/Os in the background just for it to be able to continue operations.  These I/Os happen in parallel to any I/Os that users may be doing with an application.

In general, there are lots of I/Os that are being processed at the same time.  You would not want to work on a computer system where only a single I/O was being processed at any one point in time as it would be VERY slow.  If the average queue depth would have been 5 without us but 2 with us, that means every time 2 I/Os go through to storage, we would have eliminated 3 I/Os.  The end result would be a storage “I/O Time Saved” of somewhere between 1.5-3x clock time, depending on how the underlying storage processed the I/Os. 

Another factor that contributes to the possibility of storage “I/O Time Saved” exceeding of clock time is the reduction of split I/Os.  Let’s say that without our product all I/Os actually end up being split into 3 I/Os due to Windows writing files in an excessively small, fragmented manner.  After installing our product, by displacing small, tiny writes with large, contiguous writes, each of those I/Os that had to be split into 3 are now being completed as a single I/O.  If that was the normal case, the storage “I/O Time Saved” for each I/O would be roughly 2x the actual storage I/O time due to prevention of fragmentation.

FAL Remediation and Improved Performance for MEDITECH

by Gary Quan 28. November 2016 12:11

When someone mentions heavy fragmentation on a Windows NTFS Volume, the first thing that usually comes to mind is performance degradation. While performance degradation is certainly bad, what’s worse is application failure when the application gets this error.

 

Windows Error - "The requested operation could not be completed due to a file system limitation“

 

That is exactly what happens in severely fragmented environments. These are show-stoppers that can stop a business in its tracks until the problem is remediated. We have had users report this issue to us on SQL databases, Exchange server databases, and cases involving MEDITECH EMR systems.

Some refer to this problem as the “FAL Size Issue” and here is why. In the Windows NTFS file system, as files grow in size and complexity (i.e., more and more fragmented data), they can be assigned additional metadata structures. One of these metadata structures is called the File Attribute List (FAL). The FAL structure can point to different types of file attributes, such as security attributes or standard information such as creation and modification dates and, most importantly, the actual data contained within the file. In the extremely fragmented file case, the FAL will keep track of where all the fragmented data is for the file. The FAL actually contains pointers indicating the location of the file data (fragments) on the volume. As more fragments accumulate in a file, more pointers to the fragmented data are required, which in turn increases the size of the FAL. Herein lies the problem: the FAL size has an upper limitation size of 256KB. When that limit is reached, no more pointers can be added, which means NO more data can be added to the data file. And, if it is a folder file, NO more files can be added under that folder file.

If a FAL reaches the size limitation, the only resolution was to bring the volume offline, which can mean bringing the system down, then copying the file to a different location (a different volume is recommended), deleting or renaming the original file, making sure there is sufficient contiguous free space on the original volume, rebooting the system to reset the free space cache, then copying the file back. This is not a quick cycle, and if that file is large in size, this process can take hours to complete, which means the system will remain offline for hours while attempting to resolve.

You would think that the logical solution would be – why not just defragment those files? The problem is that traditional defragmentation utilities can cause the FAL size to grow. While it can decrease the number of pointers, it will not decrease the FAL size. In fact, due to limitations within the file system, traditional methods of defragmenting files cause the FAL size to grow even larger, making the problem worse even though you are attempting to remediate it. This is true with all other defragmenters, including the built-in defragmenter that comes Windows. So what can be done about it?

 

The Solution

Condusiv Technologies has introduced a new technology to address this FAL size issue that is unique only to the latest version of V-locity®  for virtual servers and Diskeeper® for physical servers. This new technology called MediWrite™ contains features to help suppress this issue from occurring in the first place, give sufficient warning if it is or has occurred, plus tools to quickly and efficiently reduce the FAL size. It includes the following:

Unique FAL handling: As indicated above, traditional methods of defragmentation can cause the FAL size to grow even further. MediWrite will detect when files are having FAL size issues and will use a proprietary method of defragmentation that keeps the FAL from growing in size. An industry first!

Unique FAL growth prevention: Along with MediWrite, V-locity and Diskeeper contain another very important, patented technology called IntelliWrite® which automatically prevents new fragmentation from occurring. By preventing fragmentation from occurring, IntelliWrite minimizes any further FAL size growth issues.

Unique Offline FAL Consolidation tools: The above technologies help stop the FAL size from growing any larger, but due to File System restrictions, it cannot shrink or reduce the FAL size online. To do this, Condusiv developed proprietary offline tools that will reduce the FAL-IN-USE size in minutes.  This is extremely helpful for companies that already have a file FAL size issue before installing our software. With these tools, the user can reduce the FAL-IN-USE size back down to 100kb, 50kb, or smaller and feel completely safe from the maximum FAL size limits. The reduction process itself takes less than 5 minutes. This means that the system will only need to be taken offline for minutes which is much better than all the hours needed with the current Windows copy method.

FAL size Alerts: MediWrite will dynamically scan the volumes for any FAL sizes that have reached a certain limit (the default is a conservative 50% of the maximum size) and will create an Alert indicating this has occurred. The Alert will also be recorded in the Windows Event log, plus the user has the option to get notified by email when this occurrence happens.

 

For more information, see our MEDITECH Solution Brief

 

 

 

Tags:

Diskeeper | Disruption, Application Performance, IOPS | General | IntelliMemory | IntelliWrite | MEDITECH | V-Locity

Month List

Calendar

<<  May 2017  >>
MoTuWeThFrSaSu
24252627282930
1234567
891011121314
15161718192021
22232425262728
2930311234

View posts in large calendar