Skip to content

Bringing back realistic RAID recovery to inexpensive disks

License

Notifications You must be signed in to change notification settings

Fullaxx/microraids

Folders and files

NameName
Last commit message
Last commit date

Latest commit

fa3c1de · Aug 23, 2023
Sep 30, 2020
Apr 24, 2020
Nov 30, 2020
Apr 24, 2020
May 12, 2020
Aug 13, 2020
Apr 12, 2020
Apr 25, 2020
May 12, 2020
Jun 7, 2022
May 12, 2020
May 12, 2020
Sep 2, 2020
May 12, 2020
Apr 23, 2020
Aug 23, 2023
Sep 19, 2020
Jan 31, 2023
Mar 26, 2021
Apr 23, 2020
Jan 31, 2023
Jul 11, 2021
May 4, 2020
Apr 23, 2020
May 4, 2020
Apr 25, 2020
Apr 30, 2020
Apr 30, 2020

Repository files navigation

microraids

Fighting BITROT and URE with every last breath

Overview

I have always been a huge proponent of typical RAID. Back in the day of 2TB drives, disks were quite reliable. Even when a disk failed, I never had a problem doing a RAID recovery/resync onto a new disk. The RAID would keep running for days or weeks no problem even with 1 disk missing, up until the point a new disk could be inserted and replace the bad disk. From that perspective, the RAID system was very reliable. Fast-Forward to today where 8TB drives are ubiquitous and the probability of doing a RAID5 recovery a 9x8TB array is between 0.3% and 56.2%. I had to learn about URE percentages the hard way, so here we are.
I propose that the fundamentals of RAID are still good (including the software), but doing a full recovery on a LARGE array is no longer a realistic option given the bit error rates of current drives (i.e. RAID5 on a 72TB array). I choose to make many small "microraids" to encapsulate my data. This will keep the recovery percentages very high for each array. Each microraid is backed by a set of disk images, placed anywhere on any disk. For each microraid you can choose a different level of redundancy, even though they are stored on the same set of physical disks. microraids gives the flexibility to have any number of RAID 0/1/4/5/6 arrays as long as available drive space will allow it. Also each microraid can be checked independently for integrity and consistency in multiple ways.

Advantages

  • Disk capacities do not have to match
  • Integrity issues can be identified before recovery
  • Recovery probabilities are significantly increased
  • You can put multiple raid types (i.e. 0/1/4/5/6) on the same disk

Required Software

  • calc
  • Standard Utilities: sgdisk / hdparm / dd / losetup / mdadm

HOWTO

Tested Operating Systems

Simple Mini-ITX Setup

Alternate Hardware

More Information