Hey Checkyourlogs fans, today I encountered a very weird error when attempting to take an incremental backup of a SQL Server using the Veeam Agent for Windows. When running the job, it would immediately fail with an error message that said: “Failed to load backup meta from file: Blah veeam Backup.vbk.” This was a very interesting error message because I had just completed a full backup earlier that morning.
Here was the error message from the Veeam Console.
Digging in the UI told me to have a look at the logs for more details.
So, I went to the default log location on my Veeam Server “C:\Programdata\Veeam\Backup\<JOBNAME>
Inside the logfile, I could see that there was an error message that said failed to open the backup file. I originally thought that the issue was caused by the Repository being offline.
Next, I went to my Backup Repository and had a look, and I could see that the backup file was there.
My VBK and VIB files were there and online. I remember reading that if Deduplication Jobs are running on Veeam Backup Target Volumes that Replicas and Backups could fail. So I decided to check if there were any Deduplication Jobs running at this time.
Indeed, there was a Deduplication Optimization pass occurring right at this time. I decided to kill the deduplication job by running get-dedupjob | stop-dedupjob. The Backup Target was running Windows Server 2019 with Deduplication Enabled.
Last I started the backup again and this time with success.
I hope this blog post saves you a ton of time.
Thanks,
Dave
I have a 50 TB repository with very large backup files (some are 6 TB) even though I do per VM backup files. I get this error constantly due to the deduplication optimization locking the files. I have learned to just kill the dedup when this happens and rerun the job, but have been secretly hoping that Microsoft would find a way around this issue. Let me know if you find a better way than having to stop the deduplication all the time.
Have you tried re-configuring the Deduplication Schedules and having the Dedup jobs running during the day when the server is typically dormant for backups? This is what we do in production. Just search the blog for dedup I have a blog on how to adjust these settings and optimize your Dedup jobs.
Yes, but the jobs don’t finish in time and often run for 24 hours. Perhaps my repositories are just too big? (I have 2 of them with 50 TB). I followed the article that you had and even allowed the processor and memory to 90% and they still don’t finish in time.
From what I’ve read in various places 2x50tb and up to 6tb files would definitely not be a good fit for nfs dedup even in 2019’s improved version.
I’d probably run refs instead
Yah because Dedup only looks at the first 4 TB of a file anyways.
Are you running Per VM Backup files?