RAID Disk Failed or Array Offline? What to Do

The short answer: You can safely perform diagnostics yourself, but not repairs. Determine whether the array is merely degraded (still running but without redundancy) or has failed completely (offline and unable to assemble), read the SMART data from every drive, and record the slot order. Before running anything, create a sector-level image of every drive — only then will your attempts become reversible. Start a rebuild only if all remaining drives have no reallocated or pending sectors and you have a backup. A second failed drive, clicking noises, or a failed rebuild means you must stop trying yourself.

Most failed arrays that arrive at our laboratory did not fail on their own — they failed during the second or third rebuild attempt. We will therefore begin with what is reasonable to try and, more importantly, where to draw the line. If you have a specific device such as a Synology, QNAP, or a server with a hardware controller, additional vendor-specific considerations apply — see the RAID and NAS recovery page for details. You can bring or send the array drives to our laboratory in Brno-Židenice — transport to us is free, a straightforward initial assessment may be free, and we will quote more demanding analysis in advance. Feel free to call us first, 24/7, on +420 775 556 063, or submit a RAID data recovery enquiry.

RAID array data recovery in the ITHOPE laboratory in Brno — drives are labelled by slot and prepared for sector-by-sector cloning
With a failed RAID array, drive order, the condition of each medium, and working exclusively from clones are critical. Once received, the original drives are subjected to no further load.

Service: our laboratory reconstructs arrays and recovers their data — RAID array data recovery (process, price after diagnostics, 24/7 intake). This article explains what to determine and what not to run before handing over the array.

What you can try yourself

You can perform comprehensive diagnostics and preparation yourself — and with a failed array, that accounts for most of the value. The dangerous part is making changes that cannot be undone.

What not to do

  • Do not start a rebuild with a suspect drive. A rebuild forces the controller to read every sector on all remaining drives. A drive that operated for years because normal workloads never touched its weak areas may fail under this load. This is explained in detail in Why rebuilds kill RAID arrays.
  • Do not initialise the array or create a new one. „Create„, „Initialize", „Format„, or „Clear foreign configuration" on a hardware controller — all of these overwrite metadata (mdadm superblocks, LVM headers, or the controller’s array definition). Without that metadata, determining the drive order, stripe size, and parity rotation becomes much harder. „Import foreign config" is less destructive, but confirming it blindly without recording the original state is still a gamble.
  • Do not run chkdsk or fsck on array members. An individual drive removed from a RAID array does not contain the complete file system, so a disk check will either find nothing or write „repairs" into structures that belong elsewhere. Likewise, do not run chkdsk on a volume that has assembled only partially — anything it identifies as an inconsistency may be deleted (or, at best, moved to found.000).
  • Do not write to the array. If you need to inspect it, mount it read-only (mount -o ro). This rule applies without exception, even if you intend to „copy just one file".
  • Do not swap drives around by trial and error. Before removing them, label the slots with a marker or labels and photograph the rear of the enclosure.

What to check

  1. SMART data for every drive. Ideally, examine each drive outside the array and connect it directly through SATA or an HBA: smartctl -a /dev/sda (from the smartmontools package). Behind a hardware controller, use smartctl -a -d megaraid,0 /dev/sda. Pay attention to attributes 5 Reallocated_Sector_Ct, 197 Current_Pending_Sector, 198 Offline_Uncorrectable, 187 Reported_Uncorrect, and 188 Command_Timeout. Any non-zero value for 197 or 198 means that the drive contains unreadable areas and a rebuild is likely to fail on it. Attribute 199 UDMA_CRC_Error_Count, by contrast, often indicates only a cable or backplane problem. A short self-test (smartctl -t short) is acceptable on a drive with no pending sectors; do not run a long test or any self-test on a clicking drive.
  2. Degraded or failed. A degraded array continues running and its data remains accessible, but redundancy has been lost — mdadm --detail /dev/md2 reports State : clean, degraded, while a NAS displays „Degraded„ or „Reduced protection". A failed array will not assemble, the volume is missing, cat /proc/mdstat shows inactive, and the controller reports Offline/Failed. This determines everything that follows: with a degraded array, the best first step is to copy the data elsewhere; with a failed array, do nothing without first making copies of the drives.
  3. Drive order and which data is most recent. Running mdadm --examine /dev/sda3 on each member displays the Array UUID, Raid Level, Device Role, Data Offset, Chunk Size, and, most importantly, Events. A drive with a significantly lower event count dropped out of the array first and contains older data — returning it as a valid member would mix data from two different points in time. Save the command output rather than copying it down manually.

Why you should image every drive first

Every attempt to assemble the array, every rebuild, and every decision to „let it boot once more" involves writes, and writes are irreversible. A sector-level image turns an irreversible attempt into a reversible one: you can test five different drive orders and stripe sizes and always return to the starting point.

Use ddrescue with a mapfile, not dd — dd may stall at the first unreadable sector or fill the error with zeros without telling you where the problem areas are. Begin with a fast pass without retries:

ddrescue -n /dev/sda /mnt/kopie/disk1.img /mnt/kopie/disk1.map

Only then run a retry pass (-r1) using the same mapfile. Connect the drives directly through SATA or an HBA; USB adapters hide SMART data and reset when an error occurs.

The practical limit is storage capacity: you need roughly the combined raw capacity of every array member. Four 4 TB drives are manageable; twenty 8 TB drives are not — and this is where most DIY attempts end.

When a DIY attempt is reasonable

It makes sense only if all of the following are true:

  • the array is degraded, not offline, and the data is accessible;
  • all remaining drives have clean SMART data and can be read at full speed;
  • you have a backup that you have verified by restoring at least a sample of the files;
  • the case is straightforward — RAID 1 with one healthy drive that can be read separately in read-only mode, or RAID 5 with only one failed drive.

It does not make sense with RAID 5 that went down after a second drive failed, RAID 6 after a third failure, any array containing a clicking drive, proprietary layouts whose parameters are unknown, or any situation where you cannot afford to lose the data and have nowhere to store copies.

STOP point: when to stop trying

Three situations mean the end of any DIY attempt. Do not try „just once more, carefully"; power the system down and leave it off:

  • A second drive has failed in RAID 5 (or a third in RAID 6). There is no longer enough data to reconstruct what is missing — parity cannot compensate for it. The only route forward is to extract data physically from the failed drive.
  • A drive clicks, squeals, repeatedly spins up, or is not detected at all. This is a mechanical fault. Every additional startup means more head movement across the platter and irreversible loss of the very data you are trying to recover.
  • The rebuild has failed or stalled. Repeating it will produce the same result, only with the drives in a worse condition.

Other warning signs include a growing number of errors during copying, a drive disappearing from dmesg, a sharp drop in read speed, or the smell of burnt electronics. In every case, power down the array, leave the labelled drives in their original slots, and make no further attempts.

Quick guide: RAID repair by symptom

The phrase „RAID repair" suggests that the priority is to get the array running again as quickly as possible. But when an array fails — reporting degraded or offline status, or crashing during a rebuild — the first objective is to extract the data from the failed RAID array. Only then should you address replacement storage. Follow these steps according to the symptom:

  • RAID reports Degraded: do not begin another rebuild unless you are certain that every other drive is readable. First verify the backup, stop all writes, and consult a specialist.
  • The rebuild has stalled or failed: power down the array. Repeating the rebuild will typically make bad sectors on the remaining drives worse.
  • RAID 5 is Offline after the second drive failed: do not attempt to „bring up" the original configuration. With RAID 5, the precise drive order, stripe size, and parity layout are critical.
  • RAID 6 failed after a third problem: leave the drives in the same slots, label their positions, and arrange an offline reconstruction from clones.
  • RAID 0 after a drive failure: the data is not redundant, but if the failed drive can be physically read or repaired, the array structure can be reconstructed.

This is the distinction between servicing a device to „repair RAID" and laboratory RAID array data recovery. The first seeks to make the device operational again; the second protects the content that remains on the drives.

1. The myth that „RAID is a backup"

Before examining the technical details, we must dispel the most dangerous myth in the industry. A RAID array (Redundant Array of Independent Disks) was never designed as a backup. It is a tool for increasing availability and protecting against the hardware failure of one or more drives.

Why is RAID not a backup?

  • It does not protect against files deleted through human error.
  • It will not save you from ransomware that encrypts the logical volume.
  • It cannot help when a failed controller writes invalid metadata to the array.
  • It fails when the array is accidentally reinitialised or recreated.

If the entire array fails because of a firmware bug, a power surge, or multiple drive failures, you are just as completely without your data as you would be if it had been stored on a single external drive. The only real backup is a physically separate copy of the data, ideally off-site or on another medium. Once an array reaches the „offline" state, recovering it requires specialist intervention, regardless of how advanced the RAID level may be.

2. RAID levels and their real-world fault tolerance

Different RAID levels provide fault tolerance in different ways. To recover data from RAID, you need to understand what happens inside the array when a drive fails.

RAID 0 — striping without fault tolerance

Data is distributed in stripes across all drives to maximise performance.

  • Fault tolerance: none. A single drive failure means the loss of all data because every file is split into fragments across all drives.
  • RAID 0 recovery: difficult, but not impossible. We must physically repair the failed drive (for example, by replacing its read/write heads in a clean laminar-flow enclosure), clone it, and then reconstruct the data blocks from every drive offline in the correct order. The absence of parity simplifies the mathematics, but the lack of redundancy places greater pressure on the hardware repair to be successful.

RAID 1 — mirroring

Data is written simultaneously to both drives, or to each mirrored pair. The failure of one drive does not interrupt operation. The risk arises if the controller fails or if the remaining drive in the mirror is also defective. Recovery often consists simply of reading the data from at least one working medium.

RAID 5 — block-level parity

Distributed parity allows the array to tolerate the failure of one drive.

  • Degraded mode: after a drive fails, the array continues operating by reconstructing the missing data from parity. This places an enormous load on the remaining drives.
  • Rebuild failure: if recovery fails while the array is degraded — for example, because a bad sector appears on a second drive — the entire array goes offline. The challenge is to determine exactly where each stripe ends and how parity rotates, which requires analytical reconstruction in a laboratory.

RAID 6 — dual parity

RAID 6 tolerates the simultaneous failure of two drives. It is a standard choice for large enterprise NAS systems.

  • A real-world scenario: although the array survives the loss of two drives, disaster often occurs during a rebuild when a third drive fails (see our real case involving a 20-drive array below). The two parity planes make offline reconstruction extremely complex.

RAID 10 — mirrored stripes

RAID 10 combines the performance of RAID 0 with the redundancy of RAID 1. It can survive several drive failures provided they do not occur in the same mirrored pair. If they do, the entire stripe fails.

RAID 50/60

Briefly, these levels combine striping with parity. Their nested structure multiplies the complexity of recovery and requires advanced analysis.

3. Typical failure scenarios we see every day

At our Brno laboratory, we repeatedly encounter patterns of behaviour that lead to disaster. Once you understand what happened, you will also understand why RAID cannot simply be „repaired" with a click in the BIOS.

(a) A second drive fails during the rebuild

The administrator replaces the failed drive. To reconstruct the data onto the new drive, the controller must read every sector on all remaining platters. Old drives that have run without errors for years suddenly reveal weak areas under this severe load and begin producing bad sectors. The entire array then fails.

(b) A stalled rebuild

The rebuild either fails to start or stalls at a few percent after running for dozens of hours. The controller has encountered a bad sector and does not know how to work around it. Forcing it to continue risks destroying the read/write head mechanism.

Open hard drive with a damaged platter after failure during a RAID array rebuild at the ITHOPE laboratory in Brno
A rebuild forces every remaining drive to read every sector. Hidden bad sectors or damaged heads often become apparent at this stage — suddenly turning a simple drive replacement into a laboratory recovery.

(c) Controller failure and a proprietary layout

A dead controller can corrupt the metadata. A replacement controller may recognise the drives, but if it does not know the original stripe offset and parity rotation, it may offer only „Foreign configuration" and demand initialisation.

(d) Human error

Someone accidentally disconnects the wrong drive, swaps the positions of drives in their bays, and starts the array. Or they reinitialise it after seeing an error message.

(e) Multiple drives fail at once

A power supply or NAS backplane failure can kill three or four drives simultaneously. The array may then report that the logical unit is empty or inaccessible.

4. What you must not do if the data is worth recovering

This is the most important section of the article. Most data that becomes permanently unrecoverable is lost only after panicked DIY RAID repair attempts.

  • Never repeat a rebuild on a degraded array. If the rebuild failed once, it will fail again — and the load imposed by every further attempt kills the supposedly „healthy" drives. This exact scenario brought down the 20-drive array in the case below.
  • Do not swap or replace drives by trial and error. With an offline array, you must know the exact drive order. Randomly rearranging drives in the bays destroys the chance of logical reconstruction. Always label the positions before removing the drives (Slot 1, Slot 2, and so on).
  • Do not install „RAID recovery" software on a live array. Never run recovery tools such as R-Studio while the operating system is also writing to the array. Every additional write — logs, temporary files, and other activity — overwrites remnants of the original structures.
  • Do not leave the array running in degraded mode. If one drive has failed, do not plan to shut down the server „tomorrow". You would be gambling on whether a second drive survives another hour. Stop all writes immediately and consult us.
  • Do not use „Repair„ or „Factory Reset". Do not upgrade the firmware (DSM/QTS) or initialise a new logical volume. Doing so erases the remaining file-system metadata.

If you have already done any of the above, stop now. The situation may still be recoverable, but the array must be powered down immediately.

5. How a professional laboratory recovers a RAID array

Our work is based not on guesswork, but on technical precision. Before beginning logical recovery, we must physically protect the data.

Sector-level cloning

We never work on the original drives. Every array drive — healthy, defective, or clicking — is connected to dedicated hardware imagers (DeepSpar Disk Imager). These devices read unstable surfaces with extreme sensitivity, skip bad sectors, and avoid subjecting the drives to operating-system timeouts. Where an ordinary computer freezes, DeepSpar retrieves as much data as possible.

Working in a clean environment

If a drive has suffered a physical mechanical failure, such as clicking heads or burnt electronics, we replace the read/write heads in a laminar-flow enclosure (clean room). Without it, even a microscopic speck of dust could immediately damage the platter.

Offline analysis and reconstruction of proprietary layouts

Once bit-for-bit copies (clones) have been created, the intellectual core of the recovery begins:

  1. Determining drive order: the physical slots often do not correspond to the controller’s parity order.
  2. Calculating stripe size and offset: we determine where the first data block begins. A shift of even one sector produces a completely incoherent mass of bytes.
  3. Parity rotation: with RAID 5/6, we must identify which block contains parity and determine the layout, such as left-symmetric or right-asymmetric.
  4. Virtual reconstruction: using PC-3000 equipment and our own scripts, we assemble the array from bit-for-bit copies and build a virtual volume, which we inspect on our staging storage system (120 TB SATA RAID 10).

Our laboratory is in Brno, and drive intake and consultations are available 24/7 on +420 775 556 063. If you have a Synology, QNAP, or a server with a hardware controller, additional considerations apply — see Synology NAS data recovery, QNAP NAS, and server data recovery.

6. A real 20-drive RAID 6 failure — and why it happened to you too

The best illustration is a 20-drive RAID 6 sent to us by an engineering company in Brno.

Parallel cloning of 20 drives from an enterprise RAID 6 array on imaging stations at the ITHOPE laboratory in Brno
All 20 drives from the failed RAID 6 array are cloned in parallel, sector by sector. The array is then reconstructed exclusively from these copies — the originals are never written to again.

Situation: an enterprise NAS with 20 drives, 160 TB, and RAID 6 (dual parity). Drive #7 failed on Monday. The administrator ordered a replacement and started a rebuild overnight. On Tuesday, a second drive, #12, failed — leaving the array in a „degraded-critical„ state where any further problem would be fatal. Such an array should have been left untouched for two days, but the rebuild continued. After 30 hours, a read operation encountered a bad sector on the apparently „healthy" drive #5. The controller treated this as a third drive failure and took the array offline. It could not be recovered by conventional means.

Our solution: we received all 20 drives. Using DeepSpar, we cloned them one by one, dealing with extremely slow reads from drive #5. Offline analysis revealed a proprietary array layout (mdadm + LVM). We determined the true drive order, stripe offset, and parity rotation, and successfully assembled a virtual array from the cloned sectors. The process took 11 days, and the customer recovered 100% of their files.

If the administrator had tried another rebuild, drive #5 would have failed completely and the RAID data recovery would have become a dramatically more expensive physical operation.

7. How much does it cost, and how long does it take?

The cost of RAID array data recovery always depends on the specific situation — approximate ranges by fault type and array size are available in our data recovery price list. At ITHOPE, we take a fair approach: an initial assessment determines the extent of the damage. A straightforward assessment may be free; analysis involving multiple drives, array reconstruction, or metadata reconstruction may be chargeable, and we will quote the price in advance. We determine whether the issue is purely logical, such as deleted metadata, or whether physical repairs and cloning are required. We provide the price before beginning the work, never afterwards. If the data cannot be recovered — which can occasionally happen when metadata has failed and no backups exist — you do not pay anything for the recovery.

Recovery time depends on the capacity and condition of the platters. Work involving physically defective mechanisms is the slowest. As a general rule, the fewer amateur rebuild attempts an array has undergone before it reaches us, the faster and less expensive the recovery will be.

Frequently asked questions

Can I start the rebuild myself?

If the array is merely degraded, all remaining drives have clean SMART data (zero reallocated or pending sectors and no timeouts), and you have a backup that you have verified by restoring a sample, replacing the drive and rebuilding is a routine maintenance task. If any remaining drive has pending sectors, a growing number of reallocations, or slow reads, the rebuild will probably finish it off while reading every sector. In that case, create sector-level copies of the drives first or take the array straight to a laboratory.

How can I tell a degraded array from a failed one?

A degraded array continues to run and its data remains accessible, but redundancy has been lost — mdadm --detail reports clean, degraded, while a NAS displays „Degraded„ or „Reduced protection". A failed (offline) array will not assemble, the volume is missing, /proc/mdstat shows the array as inactive, and a hardware controller reports Offline or Failed. The distinction is crucial: with a degraded array, you have time to copy the data elsewhere carefully; with a failed array, every additional startup is a risk.

What should I use to image the drives, and how much space do I need?

Use ddrescue with a mapfile, not dd — dd may stall at the first unreadable sector or fill the error with zeros without telling you where the problem areas are. Begin with a fast pass using -n and no retries, then run a retry pass. Connect the drives directly through SATA or an HBA, not through a USB adapter that hides SMART data and resets when an error occurs. You need roughly the combined raw capacity of every array member — this is where most DIY attempts come to an end.

Can I run chkdsk or „RAID recovery" software on the array members?

Not on individual drives removed from the array — a single member does not contain the complete file system, and a disk check may write structural „repairs" that do not belong there. Do not run it on an assembled but inconsistent volume either: chkdsk removes what it considers inconsistencies, which may include your files. Run reconstruction software only on drive copies, never on the originals.

When should I stop trying to recover the array myself?

Stop as soon as a second drive fails in RAID 5 or a third drive fails in RAID 6, if any drive clicks, squeals, or repeatedly spins up, or if a rebuild fails or stalls at a few percent. In all three situations, another startup only reduces the chance of recovery. Power the array down, leave the drives in their original slots, and arrange a diagnostic assessment.

What to do now

  1. Stop all writes and power down the array. No more rebuilds and no „one last attempt". Every minute a degraded array remains in operation reduces the chance of recovery.
  2. Do not change the drive order. If you must remove the drives, first label their positions with a marker or labels (Slot 1, Slot 2, and so on).
  3. Do not run recovery software on a live array and do not initialise a new volume or „foreign config".
  4. Contact the laboratory. Call us 24/7 on +420 775 556 063 or request a no-obligation consultation. We will arrange collection or safe transport to Brno.
  5. Expect a diagnostic assessment. A straightforward initial assessment may be free; we will quote more demanding array analysis in advance. We will tell you the extent of the damage and the recovery price beforehand — you pay for the recovery only when it produces a result.

Key points

  • You can perform diagnostics yourself, but not repairs — check the SMART data for every drive, distinguish a degraded array from a failed one, record the slot order, and create sector-level images before attempting to assemble anything.
  • RAID is not a backup — it does not protect against deletion, ransomware, controller failure, or multiple drive failures.
  • The most common cause of total data loss is repeatedly rebuilding a degraded array until old, supposedly „healthy" drives fail in a cascade.
  • Every RAID level fails differently — RAID 0 has no tolerance, RAID 5 tolerates one failed drive, and RAID 6 tolerates two; recovery always depends on drive order, stripe offset, and parity rotation.
  • Recovery means a sector-level clone of every drive followed by offline reconstruction, never working on the originals.
  • The STOP point is clear — a second failed drive, clicking noises, or a failed rebuild means the end of any DIY attempt.
  • For specific devices, see Synology, QNAP, and servers with hardware controllers.

Have you reached the STOP point — a second failed drive, clicking noises, or a failed rebuild? Power down the array and do not start it again. No-obligation consultation · Initial assessment · Price list · Contact · +420 775 556 063 (24/7)

See also: Why rebuilds kill RAID arrays · Synology NAS data recovery · QNAP NAS · Server data recovery · RAID and NAS recovery (service) · Case study: 20-drive RAID 6 · Bad sectors on an HDD


About the author

Ing. Miroslav Jaroš is the owner and senior technician of ITHOPE s.r.o. in Brno. He has specialised in data recovery since 2008 — over 18 years, the laboratory has handled more than 2,500 cases, ranging from individual drives to enterprise RAID arrays and NAS systems. The article underwent expert fact-checking by Tomáš Kopřiva against real-world practices at the ITHOPE laboratory. The 20-drive RAID 6 case described here is based on a real anonymised assignment.

Where to find us

ITHOPE is a company based in Brno, Czechia: we do data recovery from disks, RAID and NAS, PC and laptop service and IT for business, all in our own lab at Viniční 3587/216, 615 00 Brno.

Phone +420 775 556 063 (24/7 line). You can bring the device in person, or send it by courier (Zásilkovna, PPL).