Hi, the confirmation that beta ReFS driver version 10.0.14393.1934 fixed all issues, including from those with very large volumes. And reported are not just those repository server lockups fixed, but also a significantly improved performance of all backup operations. These were all excellent news, and got even better news in response: looks like the new ReFS driver is scheduled to be shipped in the next (February) cumulative update! So finally, the driver will become available for everyone and supported for production use. Many of IT Experts (especially Veeam) had spent much time working with Microsoft on this issue and finally, thanks to them! Customer Feedback: One of my customers found a temporary way around the problem. It has significantly increased the memory (RAM) on the target server. The result is ReFS bugs are now limited.
ReFS deduplication functionality with Windows Server 2016 version 1709 was incompatible with Veeam B&R v9.5 advanced ReFS integration. Specifically, the issue is that block cloning API calls fail on deduplicated backup files with data integrity streams enabled which does look like a bug on Microsoft side. Thus, to make ReFS volumes with deduplication enabled actually usable for backup repositories, Veeam Software has decided to temporarily disable fast cloning functionality on such volumes in the Veeam Backup and Replication 9.5 Update 3. Then and after some weeks, the reason it took so long to fix is that there are multiple separate issues. It was to identify and separate all these individual issues when some were outside of ReFS even, for example, caused by NTFS-specific optimization in the OS memory management. And it also makes perfect sense now why only some of users were affected. For example, small ReFS volumes should be much less impacted by these bugs.
ReFS volumes with deduplication:
With Windows Server 2016, some features are quite interesting in the context of Veeam specifically (What’s new). For example, support for built-in deduplication on ReFS volumes. Remember however that ReFS deduplication engine is not currently BlockClone aware, Veeam recommends you avoid enabling on ReFS repositories, as the deduplication process will have to “inflate” the processed backup files so results are somewhat unpredictable from both performance and capacity perspective. On the other hand, ReFS volumes with deduplication enabled will likely make it the perfect target for archiving use cases, it’ll be interesting to test it with the upcoming Veeam “Archive Tier” and “NAS backup” functionality (Veeam Availability Suite v10).
ReFS and SAN-attached storage:
Here’s one other Microsoft Ignite finding. Basically, most recently Microsoft has updated their ReFS documentation by adding a note stating nothing less than “ReFS is not supported on SAN-attached storage”. This made no sense to the IT community because both unsupported SAN LUN and fully supported basic disks are absolutely identical block devices from a file system perspective, so some additional information is available at Ignite. The technical reason behind this note appears to be the fact that Microsoft Support has seen some SAN storage devices which do not honor the flush command, which is extensively used by file systems and apps to guarantee that the data has landed to the disk and is consistent. So this is not really a ReFS-specific issue. Those “questionable performance optimizations” of RAID controllers on some low-end SAN storage which delay or simply ignore flush commands is the reason Veeam also does not support such storage as backup targets. But from Microsoft perspective, they cannot possibly test every RAID controller out there, so they opted for such a blanket statement. Meaning there’s nothing to worry about for those using proven enterprise-grade storage, while those with consumer-grade devices should probably consider enabling pass-through RAID controller mode regardless of the file system behind the repository, so that all I/O goes straight to the disks.
General Windows Server 2016 licensing information, including licensing criteria and terminology can be viewed here.
Here Windows Server 2016 datasheet: Download
Ransomware – Protect your Backup Server Guide!
Ransomware: Save your customers with Veeam B&R!
Veeam B&R 9.5 U3 – New protection feature here!
You can learn more about Veeam Backup & Replication v10 here.