0 - Overview
This blog post is a walkthrough of the resiliency offered by Storage Spaces when using mirrored spaces.
For this exercise we have a single virtual machine with multiple data disks used to create the spaces.
The idea is to simulate a number of different failure and recovery scenarios, reviewing the behavior.
It includes the procedure to replace a disk and repair a degraded Space.
1 - Two-way mirror with 3 disks
Part 1 of this post focuses on testing the resiliency of a two-way mirrored virtual disk (storage space) on a storage pool with 3 physical disks.
For each section you have the steps to perform (useful if you want to cut/paste), the actual output of the cmdlets on my test system and finally some notes explaining what happened.
1.1 - Attach 3 disks to a VM
Steps to perform on the host:
1..3 | % { New-VHD –Path D:\demo\disk$_.VHDX –Dynamic –SizeBytes 20GB } | Out-Null
1..3 | % { Add-VMHardDiskDrive –VMName VM2 –Path D:\demo\disk$_.VHDX -ControllerType SCSI}
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> 1..3 | % { New-VHD –Path D:\demo\disk$_.VHDX –Dynamic –SizeBytes 20GB } | Out-Null
PS C:\> 1..3 | % { Add-VMHardDiskDrive –VMName VM2 –Path D:\demo\disk$_.VHDX -ControllerType SCSI}
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Nothing much. Just your regular VM with 1 IDE boot disk and 3 SCSI data disks.
1.2 - Create pool and mirrored space
Steps to perform on the guest:
$s = Get-StorageSubSystem -FriendlyName *Spaces*
$d = Get-PhysicalDisk -CanPool $true
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d
$p = Get-StoragePool Pool1
Set-ResiliencySetting -StoragePool $p -Name Mirror -NumberofColumnsDefault 1 -NumberOfDataCopiesDefault 2
$p | New-VirtualDisk -FriendlyName Space1 -ResiliencySettingName Mirror –UseMaximumSize
$L ="X”
$N = (Get-VirtualDisk -FriendlyName Space1 | Get-Disk).Number
Set-Disk -Number $N -IsReadOnly 0
Set-Disk -Number $N -IsOffline 0
Initialize-Disk -Number $N -PartitionStyle MBR
New-Partition -DiskNumber $N -DriveLetter $L -UseMaximumSize
Start-Sleep –Seconds 10
Initialize-Volume -DriveLetter $L -FileSystem NTFS -Confirm:$false
Output:
PS C:\> $s = Get-StorageSubSystem -FriendlyName *Spaces*
PS C:\> $d = Get-PhysicalDisk -CanPool $true
PS C:\> New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> $p = Get-StoragePool Pool1
PS C:\> Set-ResiliencySetting -StoragePool $p -Name Mirror -NumberofColumnsDefault 1 -NumberOfDataCopiesDefault 2
PS C:\> $p | New-VirtualDisk -FriendlyName Space1 -ResiliencySettingName Mirror –UseMaximumSize
FriendlyName ResiliencySettingNa OperationalStatus HealthStatus IsManualAttach Size
me
------------ ------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 27 GB
PS C:\> $L ="X”
PS C:\> $N = (Get-VirtualDisk -FriendlyName Space1 | Get-Disk).Number
PS C:\> Set-Disk -Number $N -IsReadOnly 0
PS C:\> Set-Disk -Number $N -IsOffline 0
PS C:\> Initialize-Disk -Number $N -PartitionStyle MBR
PS C:\> New-Partition -DiskNumber $N -DriveLetter $L -UseMaximumSize
Disk Number: 4
PartitionNumber DriveLetter Offset Size Type
--------------- ----------- ------ ---- ----
1 X 1048576 27 GB Logical
PS C:\> Start-Sleep –Seconds 10
PS C:\> Initialize-Volume -DriveLetter $L -FileSystem NTFS -Confirm:$false
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
Notes:
Created a Pool and Space. Resulting disk is initialized and mounted as volume X:.
1.3 - Verify health of the pool and space
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | FT -AutoSize
Disk Number: 4
PartitionNumber DriveLetter Offset Size Type
--------------- ----------- ------ ---- ----
1 X 1048576 27 GB IFS
PS C:\> Get-Volume X | Get-Partition | Get-Disk | FT -AutoSize
Number Friendly Name OperationalStatus Total Size Partition Style
------ ------------- ----------------- ---------- ---------------
4 Microsoft Storage Space Device Online 27 GB MBR
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk2 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
Notes:
All is healthy: Volume, Space (Virtual Disk), Pool and Physical Disks.
1.4 - Remove 1 disk
Steps to perform on the host:
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Removed one of the 3 SCSI disks for the VM.
That is the equivalent, in a physical machine, to pulling a drive out while the system is running (assuming you have a drive that supports hot removal).
1.5 - Verify effect on the pool and space
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-EventLog -LogName System -Source Disk | ? EntryType -ne "Information" | FL
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Level -lt 4 | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror Incomplete Warning False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Degraded Warning False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk2 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-EventLog -LogName System -Source Disk | ? EntryType -ne "Information" | FL
Index : 14802
EntryType : Warning
InstanceId : 2147745949
Message : Disk 1 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR11, 1}
Source : disk
TimeGenerated : 3/29/2014 5:04:16 PM
TimeWritten : 3/29/2014 5:04:16 PM
UserName :
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Level -lt 4 | FL
TimeCreated : 3/29/2014 5:04:16 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 4:54:00 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 103
Message : The capacity consumption of the storage pool {1874344D-B794-11E3-80B7-00155D010205} has exceeded the
threshold limit set on the pool. Return Code: STATUS_SUCCESS
Notes:
As expected, the mirror is now incomplete and pool is degraded, but the volume is still healthy.
Applications won’t see the event, but alarm bells are ringing in the event logs.
1.6 - Add back the removed disk
Steps to perform on the host:
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Added the VHDX file back.
That is the equivalent, in a physical machine, plugging the drive back in (assuming you have a drive that supports hot plugging).
1.7 - Verify that the pool and space recover
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Message -like "*repair*" | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk2 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Message -like "*repair*" | FL
TimeCreated : 3/29/2014 5:11:39 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874345F-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 4:54:00 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874345F-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
Notes:
Storage Spaces, seeing the drive coming back, attempts a repair and succeeds. Not sure why the log shows two repairs.
After that, all is well again: virtual disks, storage pools and physical disks all report as healthy.
Again, this is completely transparent to the applications.
1.8 - Remove 2 disks
Steps to perform on the host:
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Removing 2 of the 3 disks now. There is now only one surviving disk. This will be trouble...
1.9 - Verify effect on the pool and space
Steps to perform on the guest:
Get-Volume X | FT –AutoSize
Get-VirtualDisk Space1 | FT -AutoSize
Get-VirtualDisk Space1 | Get-StoragePool | FT -AutoSize
Get-VirtualDisk Space1 | Get-PhysicalDisk | FT -AutoSize
Get-EventLog -LogName System -Source Disk -After "3/29/2014 5:10 PM” | ? EntryType -ne "Information" | FL
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 5:10 PM") } | FL
Output:
PS C:\> Get-Volume X | FT –AutoSize
Get-Volume : No MSFT_Volume objects found with property 'DriveLetter' equal to 'X'. Verify the value of the property and retry.
At line:1 char:1
+ Get-Volume X | FT –AutoSize
+ ~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (X:Char) [Get-Volume], CimJobException
+ FullyQualifiedErrorId : CmdletizationQuery_NotFound_DriveLetter,Get-Volume
PS C:\> Get-VirtualDisk Space1 | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror Detached Unhealthy False 27 GB
PS C:\> Get-VirtualDisk Space1 | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Read-only Unhealthy False False
PS C:\> Get-VirtualDisk Space1 | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-EventLog -LogName System -Source Disk -After "3/29/2014 5:10 PM” | ? EntryType -ne "Information" | FL
Index : 14807
EntryType : Warning
InstanceId : 2147745949
Message : Disk 4 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR14, 4}
Source : disk
TimeGenerated : 3/29/2014 5:17:32 PM
TimeWritten : 3/29/2014 5:17:32 PM
UserName :
Index : 14806
EntryType : Warning
InstanceId : 2147745949
Message : Disk 2 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\...\DR12, 2}
Source : disk
TimeGenerated : 3/29/2014 5:17:32 PM
TimeWritten : 3/29/2014 5:17:32 PM
UserName :
Index : 14805
EntryType : Warning
InstanceId : 2147745949
Message : Disk 1 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR15, 1}
Source : disk
TimeGenerated : 3/29/2014 5:17:31 PM
TimeWritten : 3/29/2014 5:17:31 PM
UserName :
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 5:10 PM") } | FL
TimeCreated : 3/29/2014 5:17:32 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 5:17:32 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 203
Message : Physical drive {38133EAF-D68C-7229-E30B-64D68782E509} failed an IO operation. Return Code: This device
does not exist.. Additional related events may be found in the System event log for Disk 2.
This drive may need to be replaced. To view its reliability counters, run this command in PowerShell:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{38133EAF-D68C-7229-E30B-64D68782E509}" } |
Get-StorageReliabilityCounter
This drive may be located using the following information:
Drive Manufacturer: Msft
Drive Model Number: Virtual Disk
Drive Serial Number:
More information can be obtained using this PowerShell command:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{38133EAF-D68C-7229-E30B-64D68782E509}" }
If this drive is in an enclosure, it may be located using the following information:
Enclosure Manufacturer:
Enclosure Model Number:
Enclosure Serial Number:
Enclosure Slot: -1
It may also be located by running this command in PowerShell:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{38133EAF-D68C-7229-E30B-64D68782E509}" } |
Enable-PhysicalDiskIndication
TimeCreated : 3/29/2014 5:17:32 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 303
Message : Drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or are
missing. As a result, no copy of data is available. Return Code: This device does not exist.
TimeCreated : 3/29/2014 5:17:32 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 303
Message : Drives hosting data for storage space {18743460-B794-11E3-80B7-00155D010205} have failed or are
missing. As a result, no copy of data is available. Return Code: This device does not exist.
TimeCreated : 3/29/2014 5:17:31 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 5:11:38 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
Notes:
As expected, things get broken. The virtual disk is detached and the volume simply disappears completely.
The storage pool is showing as unhealthy and read-only. Two physical disks are correctly reported missing.
If there is an application using this specific volume, there will be failures and down time associated with this event.
1.10 - Add back the removed disks
Steps to perform on the host:
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1 –Path D:\demo\disk2.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1 –Path D:\demo\disk2.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Putting the two drives back in. This should fix things.
1.11 - Verify that the pool and space recover
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Message -like "*repair*") -and ($_.TimeCreated -gt "3/29/2014 5:20 PM") } | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Message -like "*repair*") -and ($_.TimeCreated -gt "3/29/2014 5:20 PM") } | FL
TimeCreated : 3/29/2014 6:12:50 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874345F-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 6:12:47 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874345F-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
Notes:
Without any need for repair commands, the physical disks are detected and go back to a healthy state.
The storage pool goes back to healthy and is no longer read-only. The virtual disk recovers and the volume shows up again.
You will have some work to do restart and recover your applications, which suffered the equivalent of a hard crash.
1.12- Remove 1 disk, add replacement disk
Steps to perform on the host:
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
New-VHD –Path D:\demo\disk1r.VHDX –Dynamic –SizeBytes 20GB
Add-VMHardDiskDrive –VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1r.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
PS C:\> New-VHD –Path D:\demo\disk1r.VHDX –Dynamic –SizeBytes 20GB | Out-Null
PS C:\> Add-VMHardDiskDrive –VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1r.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1r.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
Notes:
Now we’re simulating that you lost a disk and it’s really broken and never coming back.
You add a brand new disk that will be the replacement to the old disk.
1.13 - Verify effect on the pool and space
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-PhysicalDisk –CanPool $true | FT -AutoSize
Get-EventLog -LogName System -Source Disk -After "3/29/2014 6:20 PM” | ? EntryType -ne "Information" | FL
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 6:20 PM") } | FL
Output:
PS C:\> Get-Volume X | FT –AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror Incomplete Warning False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Degraded Warning False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-PhysicalDisk –CanPool $true | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk1 True OK Healthy Auto-Select 20 GB
PS C:\> Get-EventLog -LogName System -Source Disk -After "3/29/2014 6:20 PM” | ? EntryType -ne "Information" | FL
Index : 14813
EntryType : Warning
InstanceId : 2147745949
Message : Disk 1 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR16, 1}
Source : disk
TimeGenerated : 3/29/2014 6:24:13 PM
TimeWritten : 3/29/2014 6:24:13 PM
UserName :
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 6:20 PM") } | FL
TimeCreated : 3/29/2014 6:24:13 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874345F-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
Notes:
As we saw previously, we survive the failure of a single disk just fine. Volume is healthy, but the virtual disk and pool are degraded.
We note that there is a brand new disk available that can be pooled, but Storage Spaces will not automatically attempt to use it.
1.14 - Retire old disk, add new disk, repair and verify
Steps to perform on the guest:
Get-PhysicalDisk | ? OperationalStatus –eq “Lost Communication” | Set-PhysicalDisk –Usage Retired
Add-PhysicalDisk –StoragePoolFriendlyName Pool1 -PhysicalDisks (Get-PhysicalDisk –CanPool $True)
Repair-VirtualDisk Space1
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT –AutoSize
Get-StoragePool Pool1 | Get-PhysicalDisk | FT –AutoSize
Remove-PhysicalDisk -StoragePoolFriendlyName Pool1 -PhysicalDisks (Get-PhysicalDisk | ? OperationalStatus -eq "Lost Communication" )
Get-StoragePool Pool1 | Get-PhysicalDisk | FT –AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Output:
PS C:\> Get-PhysicalDisk | ? OperationalStatus –eq “Lost Communication” | Set-PhysicalDisk –Usage Retired
PS C:\> Add-PhysicalDisk –StoragePoolFriendlyName Pool1 -PhysicalDisks (Get-PhysicalDisk –CanPool $True)
PS C:\> Repair-VirtualDisk Space1
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 26.9 GB 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 27 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Degraded Warning False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT –AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Retired 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Remove-PhysicalDisk -StoragePoolFriendlyName Pool1 -PhysicalDisks (Get-PhysicalDisk | ? OperationalStatus -eq "Lost Communication" )
Confirm
Are you sure you want to perform this action?
Removing a physical disk will cause problems with the fault tolerance capabilities of the following storage pool:"Pool1".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"):
PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
Notes:
After we add the new disk to the pool and command a repair, things go back to normal.
The old disk can either be retired (remains showing in the pool) or it can be completely removed from the pool.
1.15 - Add the old disk back
Steps to perform on the host:
Add-VMHardDiskDrive –VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 3 –Path D:\demo\disk1.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Add-VMHardDiskDrive –VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 3 –Path D:\demo\disk1.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1r.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
VM2 SCSI 0 3 D:\demo\disk1.VHDX
Notes:
Simulating an odd but plausible situation. You have cleaned up the pool and repaired everything after a disk went bad, but someone put that old disk back in.
Since you’ve already removed it from the pool, what is the result?
1.16 - Verify effect on the pool, reset disk
Steps to perform on the guest:
Get-PhysicalDisk | FT –AutoSize
Get-StoragePool Pool1 | Get-PhysicalDisk
Get-PhysicalDisk | ? OperationalStatus -eq "Unrecognized Metadata" | Reset-PhysicalDisk
Get-PhysicalDisk | FT -AutoSize
Get-EventLog -LogName System -Source Disk -After "3/29/2014 6:20 PM” | ? EntryType -ne "Information" | FL
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 6:20 PM") } | FL
Output:
PS C:\> Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk0 False OK Healthy Auto-Select 32 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk5 False Unrecognized MetadataUnhealthy Unknown 20 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-StoragePool Pool1 | Get-PhysicalDisk
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-PhysicalDisk | ? OperationalStatus -eq "Unrecognized Metadata" | Reset-PhysicalDisk
PS C:\> Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk0 False OK Healthy Auto-Select 32 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk5 True OK Healthy Auto-Select 20 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 7:00PM") } | FL
TimeCreated : 3/29/2014 7:10:53 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 201
Message : Physical drive {5A464324-FBA5-A176-5DE8-DE8FE63E3951} has invalid meta-data. Resetting the health
status, using command line or GUI, might bring the physical drive to the primordial pool. Return Code:
STATUS_SUCCESS
Notes:
So you can see that the old disk comes back as unhealthy, with a status of “unrecognized metadata”.
It has already been removed from the pool and contains metadata that is inconsistent with the current state of the pools and virtual disks in the system.
There is an interesting event log in the Storage Spaces event highlighting the situation. The recommendation is to reset the health status.
We go ahead and reset it as suggested, making it healthy again and available to add back to the pool if necessary (that last portion is not shown).
2 - Three-way mirror with 5 disks
Part 2 of this post focuses on testing the resiliency of a 3-way mirrored virtual disk on a storage pool with 5 physical disks. It follows the same format as Part 1.
We are going to start fresh with a new set of disks. At this point, if you’re following along on your system, you should:
- Inside the VM, remove all virtual disks and pools from Part 1.
- On the host, remove the data VHDX files from the VM, then delete them.
- Inside the VM, clear the System log and the Storage Spaces.
2.1 - Attach 5 disks to a VM
Steps to perform on the host:
1..5 | % { New-VHD –Path D:\demo\disk$_.VHDX –Dynamic –SizeBytes 20GB }
1..5 | % { Add-VMHardDiskDrive –VMName VM2 –Path D:\demo\disk$_.VHDX -ControllerType SCSI}
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> 1..5 | % { New-VHD –Path D:\demo\disk$_.VHDX –Dynamic –SizeBytes 20GB } | Out-Null
PS C:\> 1..5 | % { Add-VMHardDiskDrive –VMName VM2 –Path D:\demo\disk$_.VHDX -ControllerType SCSI}
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
VM2 SCSI 0 3 D:\demo\disk4.VHDX
VM2 SCSI 0 4 D:\demo\disk5.VHDX
Notes:
Nothing much. Just your regular VM with 1 IDE boot disk and 5 SCSI data disks.
2.2 - Create pool and 3-way mirrored space
Steps to perform on the guest:
$s = Get-StorageSubSystem -FriendlyName *Spaces*
$d = Get-PhysicalDisk -CanPool $true
New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d
$p = Get-StoragePool Pool1
Set-ResiliencySetting -StoragePool $p -Name Mirror -NumberofColumnsDefault 1 -NumberOfDataCopiesDefault 3
$p | New-VirtualDisk -FriendlyName Space1 -ResiliencySettingName Mirror –UseMaximumSize
$L ="X”
$N = (Get-VirtualDisk -FriendlyName Space1 | Get-Disk).Number
Set-Disk -Number $N -IsReadOnly 0
Set-Disk -Number $N -IsOffline 0
Initialize-Disk -Number $N -PartitionStyle MBR
New-Partition -DiskNumber $N -DriveLetter $L -UseMaximumSize
Start-Sleep –Seconds 10
Initialize-Volume -DriveLetter $L -FileSystem NTFS -Confirm:$false
Output:
PS C:\> $s = Get-StorageSubSystem -FriendlyName *Spaces*
PS C:\> $d = Get-PhysicalDisk -CanPool $true
PS C:\> New-StoragePool -FriendlyName Pool1 -StorageSubSystemFriendlyName $s.FriendlyName -PhysicalDisks $d
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> $p = Get-StoragePool Pool1
PS C:\> Set-ResiliencySetting -StoragePool $p -Name Mirror -NumberofColumnsDefault 1 -NumberOfDataCopiesDefault 3
PS C:\> $p | New-VirtualDisk -FriendlyName Space1 -ResiliencySettingName Mirror –UseMaximumSize
FriendlyName ResiliencySettingNa OperationalStatus HealthStatus IsManualAttach Size
me
------------ ------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 30 GB
PS C:\> $L ="X”
PS C:\> $N = (Get-VirtualDisk -FriendlyName Space1 | Get-Disk).Number
PS C:\> Set-Disk -Number $N -IsReadOnly 0
PS C:\> Set-Disk -Number $N -IsOffline 0
PS C:\> Initialize-Disk -Number $N -PartitionStyle MBR
PS C:\> New-Partition -DiskNumber $N -DriveLetter $L -UseMaximumSize
Disk Number: 6
PartitionNumber DriveLetter Offset Size Type
--------------- ----------- ------ ---- ----
1 X 1048576 30 GB Logical
PS C:\> Start-Sleep –Seconds 10
PS C:\> Initialize-Volume -DriveLetter $L -FileSystem NTFS -Confirm:$false
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 29.9 GB 30 GB
Notes:
Created a storage pool and storage space. Resulting disk is initialized and mounted as volume X:.
2.3 - Verify health of the pool and space
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 29.9 GB 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk2 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk5 False OK Healthy Auto-Select 19.25 GB
Notes:
All is healthy: volume, storage space (virtual disk), storage pool and physical disks.
2.4 - Remove 2 disks
Steps to perform on the host:
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 2 D:\demo\disk3.VHDX
VM2 SCSI 0 3 D:\demo\disk4.VHDX
VM2 SCSI 0 4 D:\demo\disk5.VHDX
Notes:
Removed 2 of the 5 SCSI disks for the VM.
Since this is a 3-way mirror, we’re going to hit it right away with a double disk failure. But it should be able to survive that.
2.5 - Verify effect on the pool and space
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-EventLog -LogName System -Source Disk | ? EntryType -ne "Information" | FL
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Level -lt 4 | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 29.9 GB 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror Incomplete Warning False 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Degraded Warning False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk5 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-EventLog -LogName System -Source Disk | ? EntryType -ne "Information" | FL
Index : 14841
EntryType : Warning
InstanceId : 2147745949
Message : Disk 2 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR22, 2}
Source : disk
TimeGenerated : 3/29/2014 8:38:11 PM
TimeWritten : 3/29/2014 8:38:11 PM
UserName :
Index : 14840
EntryType : Warning
InstanceId : 2147745949
Message : Disk 1 has been surprise removed.
Category : (0)
CategoryNumber : 0
ReplacementStrings : {\DR21, 1}
Source : disk
TimeGenerated : 3/29/2014 8:38:10 PM
TimeWritten : 3/29/2014 8:38:10 PM
UserName :
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Level -lt 4 | FL
TimeCreated : 3/29/2014 8:38:11 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:38:10 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:31:56 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 103
Message : The capacity consumption of the storage pool {18743521-B794-11E3-80B7-00155D010205} has exceeded the
threshold limit set on the pool. Return Code: STATUS_SUCCESS
Notes:
As expected, the 3-way mirrored space was able to survive the double disk failure. The storage space is now incomplete and the storage pool is degraded, but the volume is still healthy.
Applications won’t see the event, but again there are many alarm bells ringing in the event logs. You better act fast because the next failure will be fatal…
2.6 - Add back the removed disks
Steps to perform on the host:
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1 –Path D:\demo\disk2.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 –Path D:\demo\disk1.VHDX
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1 –Path D:\demo\disk2.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 0 D:\demo\disk1.VHDX
VM2 SCSI 0 1 D:\demo\disk2.VHDX
VM2 SCSI 0 2 D:\demo\disk3.VHDX
VM2 SCSI 0 3 D:\demo\disk4.VHDX
VM2 SCSI 0 4 D:\demo\disk5.VHDX
Notes:
Added the two VHDX files back to restore the VM to the original state.
2.7 - Verify that the pool and space recover
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Message -like "*repair*" | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 29.9 GB 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk2 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk5 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? Message -like "*repair*" | FL
TimeCreated : 3/29/2014 8:45:51 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:45:47 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:31:56 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
Notes:
Storage Spaces, seeing the drives coming back, attempts a repair and succeeds.
After that, all is well again: virtual disks, storage pools and physical disks all report as healthy.
This shows that we can survive two disk failures in a way that is completely transparent to the applications.
2.8 - Remove 3 disks
Steps to perform on the host:
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 2
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 1
PS C:\> Remove-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 2
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 3 D:\demo\disk4.VHDX
VM2 SCSI 0 4 D:\demo\disk5.VHDX
Notes:
Removing 3 of the 5 disks now. There are now only two surviving disks. That should be too much to bear.
2.9 - Verify effect on the pool and space
Steps to perform on the guest:
Get-Volume X | FT –AutoSize
Get-VirtualDisk Space1 | FT -AutoSize
Get-VirtualDisk Space1 | Get-StoragePool | FT -AutoSize
Get-VirtualDisk Space1 | Get-PhysicalDisk | FT -AutoSize
Get-EventLog -LogName System -Source Disk -After "3/29/2014 8:40 PM” | ? EntryType -ne "Information" | FT TimeGenerated, Source, Index, EventID, EntryType, Message -AutoSize
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 8:40 PM") } | FL
Output:
PS C:\> Get-Volume X | FT –AutoSize
Get-Volume : No MSFT_Volume objects found with property 'DriveLetter' equal to 'X'. Verify the value of the property and retry.
At line:1 char:1
+ Get-Volume X | FT –AutoSize
+ ~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (X:Char) [Get-Volume], CimJobException
+ FullyQualifiedErrorId : CmdletizationQuery_NotFound_DriveLetter,Get-Volume
PS C:\> Get-VirtualDisk Space1 | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror Detached Unhealthy False 30 GB
PS C:\> Get-VirtualDisk Space1 | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 Read-only Unhealthy False False
PS C:\> Get-VirtualDisk Space1 | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk-1 False Lost Communication Warning Auto-Select 19.25 GB
PhysicalDisk5 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-EventLog -LogName System -Source Disk -After "3/29/2014 8:40 PM” | ? EntryType -ne "Information" | FT TimeGenerated, Source, Index, EventID, EntryType, Message -AutoSize
TimeGenerated Source Index EventID EntryType Message
------------- ------ ----- ------- --------- -------
3/29/2014 8:50:21 PM disk 14847 157 Warning Disk 6 has been surprise removed.
3/29/2014 8:50:21 PM disk 14846 157 Warning Disk 3 has been surprise removed.
3/29/2014 8:50:20 PM disk 14845 157 Warning Disk 2 has been surprise removed.
3/29/2014 8:50:19 PM disk 14844 157 Warning Disk 1 has been surprise removed.
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Level -lt 4) -and ($_.TimeCreated -gt "3/29/2014 8:40 PM") } | FL
TimeCreated : 3/29/2014 8:50:21 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:50:21 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 203
Message : Physical drive {06724EEA-84C4-283C-1EF9-F26D195E0D7A} failed an IO operation. Return Code: This device
does not exist.. Additional related events may be found in the System event log for Disk 3.
This drive may need to be replaced. To view its reliability counters, run this command in PowerShell:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{06724EEA-84C4-283C-1EF9-F26D195E0D7A}" } |
Get-StorageReliabilityCounter
This drive may be located using the following information:
Drive Manufacturer: Msft
Drive Model Number: Virtual Disk
Drive Serial Number:
More information can be obtained using this PowerShell command:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{06724EEA-84C4-283C-1EF9-F26D195E0D7A}" }
If this drive is in an enclosure, it may be located using the following information:
Enclosure Manufacturer:
Enclosure Model Number:
Enclosure Serial Number:
Enclosure Slot: -1
It may also be located by running this command in PowerShell:
Get-PhysicalDisk | ?{ $_.ObjectId -Match "{06724EEA-84C4-283C-1EF9-F26D195E0D7A}" } |
Enable-PhysicalDiskIndication
TimeCreated : 3/29/2014 8:50:21 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 303
Message : Drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or are
missing. As a result, no copy of data is available. Return Code: This device does not exist.
TimeCreated : 3/29/2014 8:50:20 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:50:19 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:45:49 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 8:45:46 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 304
Message : One or more drives hosting data for storage space {1874353C-B794-11E3-80B7-00155D010205} have failed or
are missing. As a result, at least one copy of data is not available. However, at least one copy of
data is still available. Return Code: STATUS_SUCCESS
Notes:
As expected, the volume is gone and any applications running here would have stopped.
The virtual disk is detached, the volume is dismounted, the pool is unhealthy and 3 physical disks are missing.
Again we get more disk failures and Storage Spaces failures as I would expect. Not sure why that happens.
2.10 - Add back the 3 removed disks
Steps to perform on the host:
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 5 –Path D:\demo\disk1.VHDX
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 6 –Path D:\demo\disk2.VHDX
Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 7 –Path D:\demo\disk3.VHDX
Get-VMHardDiskDrive -VMName VM2
Output:
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 5 –Path D:\demo\disk1.VHDX
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 6 –Path D:\demo\disk2.VHDX
PS C:\> Add-VMHardDiskDrive -VMName VM2 -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 7 –Path D:\demo\disk3.VHDX
PS C:\> Get-VMHardDiskDrive -VMName VM2
VMName ControllerType ControllerNumber ControllerLocation DiskNumber Path
------ -------------- ---------------- ------------------ ---------- ----
VM2 IDE 0 0 c:\demo\v2.vhdx
VM2 SCSI 0 3 D:\demo\disk4.VHDX
VM2 SCSI 0 4 D:\demo\disk5.VHDX
VM2 SCSI 0 5 D:\demo\disk1.VHDX
VM2 SCSI 0 6 D:\demo\disk2.VHDX
VM2 SCSI 0 7 D:\demo\disk3.VHDX
Notes:
Simulating putting the 3 drives back in. This should fix things.
Just for fun, adding them in different controller locations, the equivalent of putting it back in a drive bay different than where it was originally.
Storage Spaces relies on the metadata stored in the disk itself, so this should not make any difference.
2.11 - Verify that the pool and space recover
Steps to perform on the guest:
Get-Volume X | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Message -like "*repair*") -and ($_.TimeCreated -gt "3/29/2014 9:00 PM") } | FL
Output:
PS C:\> Get-Volume X | FT -AutoSize
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ------------- ----
X NTFS Fixed Healthy 29.9 GB 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | FT -AutoSize
FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach Size
------------ --------------------- ----------------- ------------ -------------- ----
Space1 Mirror OK Healthy False 30 GB
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-StoragePool | FT -AutoSize
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Pool1 OK Healthy False False
PS C:\> Get-Volume X | Get-Partition | Get-Disk | Get-VirtualDisk | Get-PhysicalDisk | FT -AutoSize
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
PhysicalDisk6 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk1 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk4 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk3 False OK Healthy Auto-Select 19.25 GB
PhysicalDisk5 False OK Healthy Auto-Select 19.25 GB
PS C:\> Get-WinEvent -LogName Microsoft-Windows-Stor* | ? { ($_.Message -like "*repair*") -and ($_.TimeCreated -gt "3/29/2014 9:00 PM") } | FL
TimeCreated : 3/29/2014 9:13:14 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 9:13:08 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
TimeCreated : 3/29/2014 9:13:05 PM
ProviderName : Microsoft-Windows-StorageSpaces-Driver
Id : 308
Message : A repair attempt for storage space {1874353C-B794-11E3-80B7-00155D010205} was initiated by the driver.
Return Code: STATUS_SUCCESS
Notes:
Again, without any need for repair commands, the physical disks are detected and everything goes back to a healthy state.
Keep in mind that your application probably will need to be restarted and you should follow the required recovery steps, since there was down time.
3 - Conclusion
This is it. I hope this tour of mirrored storage spaces resiliency was helpful.
The idea was to save you the trouble of finding this out for yourself, but I’m sure you’ll still be interested in experimenting with it further.
The goal was also to prepare you for an eventual disk failure. You will likely be a bit more stressed at that time, so knowing what to expect and where to look might be helpful.
I have lots of additional ideas to try like trying more columns, adding storage tiers to the mix, using parity spaces or actually using an actual bad disk to go beyond just surprise removals.
But I guess that will have to wait for another opportunity. This one is already too long as it is… :-)
For additional content related to Storage Spaces, check these:
![]()
