We're currently using Windows Server Backup (bleh) to primarily backup system state on our servers. We are taking a "full bare-metal backup" each night via WSB, which backs up to an attached USB device.
In addition to this, we also employ the use of Crashplan's PRO service to get frequent backups and push them up to Crashplan's servers.
My Windows Server Backup backups have been failing after we installed additional memory over the weekend. I've got errors on several of my VSS writers and each time the backup tries to run it throws a specific error related to VSS. Here's what I'm seeing:
Errors-
Backup
The backup operation that started at '2014-03-27T19:59:38.567000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348075'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
VSS
Volume Shadow Copy Service error: Error calling a routine on the Shadow Copy Provider {b5946137-7b9f-4925-af80-51abd60b20d5}. Routine returned E_INVALIDARG. Routine details GetSnapshot({00000000-0000-0000-0000-000000000000},000000000021D060).
Operation:
Get Shadow Copy Properties
Context:
Execution Context: Coordinator
** The event ID noted is 12294 **
When I run vssadmin list writers at a command prompt, I get the following results:
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Task Scheduler Writer'
Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
State: [1] Stable
Last error: No error
Writer name: 'VSS Metadata Store Writer'
Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
State: [1] Stable
Last error: No error
Writer name: 'Performance Counters Writer'
Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {eff91499-f794-4211-863c-3d822caeedd0}
State: [10] Failed
Last error: Timed out
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {0d6d2b70-b159-448b-a129-b0ce5a2f4947}
State: [10] Failed
Last error: Timed out
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {68585e38-eeee-4b7b-8bc7-d39191486bf5}
State: [10] Failed
Last error: Timed out
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {8a945781-8e9b-4931-9cdf-d0d1a8470391}
State: [10] Failed
Last error: Timed out
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {49dfe97c-7c19-4b2a-9ac6-48d8736a81f0}
State: [10] Failed
Last error: Timed out
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {52b516ab-cc3f-476a-8a38-3a17711cbeb3}
State: [10] Failed
Last error: Timed out
Writer name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {5612603b-f804-4b38-817e-2753cb222e3a}
State: [10] Failed
Last error: Timed out
Troubleshooting
I've tried to recreate the backup set, that didn't fix it (this was before I realized VSS was involved). So then I tried to re-register the VSS config files, in case something had happened there using the below script:
cd /d %windir%\system32
net stop vss
net stop swprv
regsvr32 /s ole32.dll
regsvr32 /s oleaut32.dll
regsvr32 /s vss_ps.dll
vssvc /register
regsvr32 /s /i swprv.dll
regsvr32 /s /i eventcls.dll
regsvr32 /s es.dll
regsvr32 /s stdprov.dll
regsvr32 /s vssui.dll
regsvr32 /s msxml.dll
regsvr32 /s msxml3.dll
regsvr32 /s msxml4.dll
vssvc /register
net start swprv
net start vss
I dug out my Windows Internals, Sixth Edition (Part 2) to make sure I had a good understanding of exactly what VSS was doing behind the scenes. And, although I feel smarter now, it really didn't help me troubleshoot this problem.
I have to believe that something with increasing system memory has impacted VSS somehow, however I'm at a loss trying to troubleshoot this any further.
Anyone out there ever had a similar experience?