So the other day I got a ticket opened by my companies Managed Service Provider. There was a backup failure on one of our databases. The response from the hosting provider was:
This failed due to a "Error: 3041, Severity: 16, State: 1." error. This is because you must perform a full database backup before you back up the transaction log for a database in SQL Server. If you require any further assistance or have any queries regarding the above please do not hesitate to contact us at any time or update this ticket.
Now the database in question has been on the server for months, if not longer. And quickly looking at the database properties would tell you that a full database backup has been done, so this response was pretty much BS. Looking at the ERRORLOG for that time shows three error messages.
Error: 3041, Severity: 16, State: 1. BACKUP failed to complete the command BACKUP DATABASE prod_phreesia_print. Check the backup application log for detailed messages. Error: 18210, Severity: 16, State: 1. BackupMedium::ReportIoError: write failure on backup device '920a8f3f-be50-4e54-9b0f-f1bfeddea12a'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.). Error: 18210, Severity: 16, State: 1. BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device '920a8f3f-be50-4e54-9b0f-f1bfeddea12a'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
Looking at these three error messages (which all were logged during the same second) gives a pretty good idea as to what happened. The tape backup system threw an error causing the full or differential backup to fail. Without looking through the system any deep I know that this had to be either the full or differential backup that threw the error because the full and differential backups go directly to tape while the log backups go to disk. Since the backup device listed in a guid, that tells me that it is a backup doing directly to tape via the tape backup solution. Since the tape backups are managed by the MSP, one would think they would be able to quickly match the two errors together.
But apparently I’m expecting to much from a company which my company pays over $40k a month to.
DennyContact the Author | Contact DCAC