Hello, before tackling the subject of your presence, allow me to briefly introduce myself: I am Philippe DUPUIS, I‘m a french Data Management and Security engineer working with Christopher (Be indulgent if you notice some English grammar failures, my mother tongue is French 😊). I specialize in Data Protection and Governance, Disaster Recovery, Security, Cloud & Virtualization technologies. I’m really enjoying writing my first blog post on this amazing blog.
My presentation is done, let’s talk about one of the best new features of Veeam v11: « Immutable Linux repositories ». Reminder, we did a french webinar last month about the new features of Veeam v11 (Blog Post).
In recent times, IT infrastructure attacks have become more sophisticated (Targeted and opportunistic attacks). They are the capacity to encrypt live data and entire virtual machines, but they also have learned to delete or encrypt entire backups too. If that happens and you do not have an offline backup like tape or any other air-gapped solution, you will be in the perfect storm.
We are in a world where existing threats evolve to perfect the vectors to deliver their malicious payload with results more and more dramatic. To answer this news problematics Veeam Backup & Replication v11 (VBR v11) keeps improving and moving ahead. Veeam enforces its defensive capabilities to deal with the ransomware threat. This software is smarter these days, it’s able to recognize backup systems and can trigger actions like backup file deletion.
The question is how to protect our backups? Veeam had such capabilities via 3rd parties with tape which is a perfect air-gap solution, on-premise storage solution like Exagrid or S3 compatible storage with object-lock like AWS S3 buckets as offsite backup copies. Always keep in mind to design your backup infrastructure secure by design. You could rewatch our french webinar about it right here ==> Click.
In VBR v11, they introduced the ability to build your own immutable, hardened backup repositories. This can be accomplished using any Linux server with storage and the XFS file system.
As a reminder, immutability in this context means, a backup file cannot be changed, altered, or deleted without having root access within the Linux host before the defined timespan has passed.
Pretty simple, all you need is a Linux repository formatted with XFS and VBR v11. Please notice: When using this new feature, the Linux server cannot be used as a backup proxy.
For the Linux distribution, you are free to choose from CentOS 8.2 and 8.3, Debian 10.x, RHEL 8.2 or later, SLES 15 SP2, Ubuntu 18.04 LTS, and 20.04 LTS.
XFS with Reflink clone need to be enabled. XFS Reflink achieves the same benefits as ReFS in terms of speed and space consumption also called Fast Clone.
Last but not least, the backup chains must be compatible with immutable files. Because backup files cannot be changed or deleted during the specified period of immutability, the backup chain only can create new files without changing any of the existing ones. In summary only forward incremental with periodic synthetic or active full backup fulfill this requirement. If you use or plan to use a backup copy job, the GFS setting is required.
Linux Server Setup:
There is no difference between a hardened repository and other repositories. You could use it as a standard repository or as an extent in a Scale-Out Repository.
First, you have to deploy a new server, keep in mind to use the K.I.S.S (Keep It Simple and Straightforward) principle for your designs, once the server installation is done, update it! Always keep a system up to date.
Then, set up your storage and create a partition with an XFS volume. Create a mount point for the partition.
Create a dedicated user and set his permissions, so the veeam transport service user has the correct rights to the Veeam mount(s). Set permissions on the repository directory to only that account. At this point reboot your server and check everything once more. Now you can add the repository to VBR v11.
To add the Linux repository server to VBR managed servers, we will use a new feature « single-use credentials for the hardened repository ». When using it, Veeam does not store the credential information in his database. They are used only for deploying Veeam Data Mover Service to the host. These credentials reduce the rights for the Veeam Data Mover Service.
- Launch the “Add Server” wizard.
2. Specify the DNS Name or IP address.
3. Choose the new authentication method.
4. The Veeam Data Mover service will be installed
5. Once the installation completed, you are able to disable SSH. The Veeam components will now use server and client certificates to authenticate each other.
6. Click the button « Finish ».
At this part you could remove the sudo right on the user used to install the Veeam Component. Time to add our Linux backup repository with the wizard,
- Select « Direct attached storage »
5. Click next. Enable the fast cloning on the XFS volume, and turn on the immutable feature. Specify how days files should be immutable. The minimum time span for immutability is 7 days.
6. Enable advanced options, like « User per VM backup files ».
7. As usual specify mount server information
8. Click Next, and the required Veeam components will install. If you get any errors here, be sure to verify the rights of the single-use user you used to add the server to the Veeam managed servers. Ensure it has “RWX” rights on the XFS volume.
Congratulation, the configuration is done. Time to test this new repository with a backup job Remember the backup chain must be compatible with this new repository ! But don’t worry Veeam is here to warning us with unsupported configuration. Here an example with a forever incremental backup chain.
You won’t notice a difference in the backup process.
Let’s try to delete the backup files to test the new protection
Try again, you can’t do that! You are protected. The backup files can’t be removed from a compromised VBR host. Veeam gives you the information on when the files could be deleted.
Here ends the first part about Hardened Linux Repository. In the second part, we will abort, how it works, make some behavior tests and I will give you some advice to enhance the security.
Step by Step Guide Veeam B&R 11 Upgrade: Guide.
Veeam CDP and Application consistency: Blog Post.
Veeam improves the engine in version 11: Blog Post.
Veeam B&R v11 and ReFS: Blog Post.
Veeam B&R 11 – Continuous Data Protection: Blog Post.
Microsoft Teams Backup with VBO v5: Blog Post.
Protect your Backup against Ransomware: Blog Post.