Condusiv Technologies Blog

Condusiv Technologies Blog

Blogging @Condusiv

The Condusiv blog shares insight into the issues surrounding system and application performance—and how I/O optimization software is breaking new ground in solving those issues.

Fragmentation and Data Corruption

by Michael 31. March 2011 04:54

Diskeeper (data performance for physical systems) and V-locity (optimization for virtual systems) are designed to deliver performance, reliability, longer life and energy savings. Increased performance and saved energy from our software are relatively easy to empirically test and validate. Longer life is a matter of minimizing wear and tear on hard drives (MTTF) and providing an all around better experience for users so they can continue to be productive with aging equipment (rather than frequent hardware refreshes).

Reliability is far more difficult to pinpoint as the variables involved are difficult, if not impossible, to isolate in test cases. We have overwhelming anecdotal evidence from customers in surveys, studies, and success stories that application hangs, freezes, crashes, and the sort are all remedied or reduced with Diskeeper and/or V-locity.

However, there is a reliability "hard ceiling" in the NTFS file system; a point in which fragmentation/file attributes become so numerous reliability is jeopardized. In NTFS, files that hit the proverbial "fan", and spray out into hundreds of thousands and millions of fragments, result in a mess that is well... stinky.

In short, fragmentation can become so severe that it ultimately ends up in data loss/corruption. A Microsoft Knowledge Base article describes this phenomenon. I've posted it below for reference:

A heavily fragmented file in an NTFS file system volume may not grow beyond a certain size caused by an implementation limit in structures that are used to describe the allocations.

In this scenario, you may experience one of the following issues:

When you try to copy a file to a new location, you receive the following error message:
In Windows Vista or in later versions of Windows
The requested operation could not be completed due to a file system limitation
In versions of Windows that are earlier than Windows Vista
insufficient system resources exist to complete the requested service
When you try to write to a sparse file from the Application log, Microsoft SQL Server may log an event that resembles the following:
In Windows Vista or in later versions of Windows
Event Type: Information
Event Source: MSSQLSERVER

Description: ...
665(The requested operation could not be completed due to a file system limitation.) to SQL Server during write at 0x000024c8190000, in filename...
In versions of Windows that are earlier than Windows Vista
Event Type: Information
Event Source: MSSQLSERVER

Description: ...
1450(Insufficient system resources exist to complete the requested service.) to SQL Server during write at 0x000024c8190000, in file with handle 0000000000000FE8 ...
When a file is very fragmented, NTFS uses more space to save the description of the allocations that is associated with the fragments. The allocation information is stored in one or more file records. When the allocation information is stored in multiple file records, another structure, known as the ATTRIBUTE_LIST, stores information about those file records. The number of ATTRIBUTE_LIST_ENTRY structures that the file can have is limited.

We cannot give an exact file size limit for a compressed or a highly fragmented file. An estimate would depend on using certain average sizes to describe the structures. These, in turn, determine how many structures fit in other structures. If the level of fragmentation is high, the limit is reached earlier. When this limit is reached, you receive the following error message:

Windows Vista or later versions of Windows:
STATUS_FILE_SYSTEM_LIMITATION The requested operation could not be completed due to a file system limitation

Versions of Windows that are earlier than Windows Vista:
STATUS_INSUFFICIENT_RESOURCES insufficient system resources exist to complete the requested service

Compressed files are more likely to reach the limit because of the way the files are stored on disk. Compressed files require more extents to describe their layout. Also, decompressing and compressing a file increases fragmentation significantly. The limit can be reached when write operations occur to an already compressed chunk location. The limit can also be reached by a sparse file. This size limit is usually between 40 gigabytes (GB) and 90 GB for a very fragmented file.  

WORKAROUND
For files that are not compressed or sparse, the problem can be lessened by running Disk Defragmenter. Running Disk Defragmenter will not resolve this problem for compressed or sparse files.

Tags:

Defrag | Diskeeper | Success Stories | V-Locity

All Around the World (part deux)

by Colleen Toumayan 18. February 2011 07:07

 

“I am the person who proposed Diskeeper a few years ago in our company because we had some people who were complaining about slow machines. Most of the times the problem was related to the hard disks that were non-stop reading/writing. I tried a few times the internal defragmenter; it helped in reducing the slowness of the machine but it was always for a short time. So I looked for a better product and found Diskeeper.  

I made contact with Diskeeper UK and we had the pleasure to deal with an employee who arranged for an evaluation version of Diskeeper and Diskeeper Administrator to test in our company. 

We have a high number of computers, 16000, which are now running smoothly. The IntelliWrite does a good job preventing fragmentation. The number of calls for slow machines have dropped but we never had real measurements about the performance of Diskeeper. I am very curious about Diskeeper 2011 and what it can bring more so than this version. Diskeeper works and it is a good product. The price is also very good." 

Marc Vanderhaegen, SNCB (Société nationale des Chemins de fer belges)

Desktop Management

Brussels, Belgium

Tags:

IntelliWrite | Success Stories

Helping Customers All Across the Globe

by Colleen Toumayan 17. February 2011 08:54

“The initial feedback we're getting from our end users is that the performance improved after we implemented Diskeeper. End users had been suffering from slow access to the databases and the huge systems images. This is much better lately due to using the Diskeeper EnterpriseServer. Our main goal for using the Diskeeper has been achieved and people are feeling the difference."

“We are mainly a Health Sector solutions provider for the whole Middle East and soon North Africa, for variety vendors with high focus on Philips solutions. We are vendor free when it comes to IT products, so we use different brands like IBM, HP with Microsoft platforms only, and on top of all that the medical and health solutions like Cardiac Pictures and Archiving solutions, CPACK, and patients monitoring.

 

 

“Please feel free posting my comments. This is the least thing we can afford in paying back your reputable company for availing this fine product.” 

Ayman A. Nimer

Technology Services Segment Manager

Al Faisaliah Medical Systems

Tags:

Defrag | Success Stories

Defragged Files = Greater DR success

by Colleen Toumayan 14. July 2010 06:15

Here's a cool succes about something many IT pro's wouldn't normally associate with defrag:

 

When we began to invest in Diskeeper we had 8 servers and over 100+ clients that ran for years and would run for many more years. Every 11 weeks we start a new quarter and cycled on average 400 users across all of them each quarter. So you can image the amount of data our students move across these machines. Not to mention the amount of data that is moved across our servers with 600+ users. 

Diskeeper drastically improved our disk performance and kept it constant with the automatic defragmentation during idle time. Diskeeper not only helped with maintaining the disk performance, but also made it possible for quick disaster recovery.

We had an old file server that crashed and file backups were being neglected, so we had to run file recovery on the RAID 5 array. Without Diskeeper automatically defragging the drive and keeping all the pieces of the files together, we would have lost 50% of the data; instead we were able to recover 95% of the 300 Gb of information. 

Aaron Cowell,

Network Administrator,

Davis College, Toledo, OH

Proud supporter of:

ShoreTel VOIP, Moodle, Microsoft, Diskeeper

Tags: , , ,

Diskeeper | Success Stories

Defrag on HP EVA SANs - 45 million fragments handled

by Colleen Toumayan 12. May 2010 11:49

We have been running Diskeeper 2010 EnterpriseServer for two months on an HP EVA SAN 4000 and 4400, with 4 1TB volumes each.  

Diskeeper removed over 45 million fragments in the last two months on a specific volume that had only 15% free space, and IntelliWrite prevented 24,000 fragments. I believe that will be even better as soon as we can extend this volume to two TB. 

We see a big improvement on the backup time which came down from 48 hours to 32 now, and it’s still going down. 

I believe Diskeeper worth the price and I never had any trouble with software from Diskeeper Corporation, so that alone narrowed the field of choices. 

Jean-François Poirier
Technicien Telecommunication
Spectra Premium Industries Inc.

Tags:

Diskeeper | IntelliWrite | SAN | Success Stories

RecentComments

Comment RSS

Month List

Calendar

<<  October 2018  >>
MoTuWeThFrSaSu
24252627282930
1234567
891011121314
15161718192021
22232425262728
2930311234

View posts in large calendar