Hey Checkyourlogs Fans,
When dealing with Veeam Backup & Replication (VBR), backup failures can be frustrating, especially when the error messages seem complex. Recently, while troubleshooting a backup job failure after upgrading a VBR server to version 12.3, we encountered the following error in the logs:
[17.03.2025 02:34:11.146] <40> Error (3) Native method execution failed: test proxy rpc connection – The RPC server is unavailable.
[17.03.2025 02:34:11.146] <40> Error (3) RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [VBRP001.Techmentor.com:6162].
This error repeated multiple times for target machines, indicating a critical connectivity issue between the VBR server and the backup proxy.
Identifying the Problem
The key error message, “The RPC server is unavailable,” is a common issue that occurs when:
- The required services on the VBR server are not running.
- The firewall is blocking RPC communication.
- There are DNS or network connectivity issues.
In our case, the error occurred immediately after upgrading the VBR server to Veeam Backup & Replication 12.3. Upon further investigation, we found that the Veeam Data Mover Service was not running on the VBR server, causing the backup job to fail.
Solution: Restarting the VBR Server
In this case, the simplest and most effective solution was to reboot the VBR server. Restarting ensured that all Veeam services were initialized correctly after the upgrade. Once the server rebooted, the backup job ran successfully without further errors.
Checking for Service Status
Before rebooting, manually checking if the core Veeam services are running is always a good idea. You can do this using the following steps:
- Open the Services application (services.msc).
- Locate the following services:
- Veeam Backup Service
- Veeam Data Mover Service
- Veeam vPower NFS Service (if applicable)
- Veeam Backup Proxy Service
- If any of these services are stopped, try manually starting them.
- review the Windows Event Viewer logs for additional insights if services fail to start.
Verifying Connectivity
If rebooting does not resolve the issue, you can verify network connectivity and firewall rules using these methods:
1. Check RPC Port Availability
Run the following PowerShell command to check if the required port (6162) is open:
Test-NetConnection -ComputerName VBRP001.Techmentor.com -Port 6162
If the result shows that the port is closed, check the firewall rules and ensure Veeam services are allowed.
2. Verify DNS Resolution
Incorrect DNS settings can cause connectivity issues. Use the nslookup command to ensure the VBR server resolves correctly:
nslookup VBRP001.Techmentor.com
If DNS is unresolved, check your DNS configuration and ensure proper forward and reverse lookup zones exist.
Lessons Learned
This troubleshooting experience highlights a few key takeaways:
- Upgrades can sometimes disable or fail to restart critical services. Always verify service status after an upgrade.
- The simplest solution is often the best one. A quick server reboot can save hours of troubleshooting.
- Logs are your best friends. Checking logs at C:\ProgramData\Veeam\Backup\<JobName>\task.<Server>.Techmentor.com provided the crucial clue that the issue was with the RPC connection.
- Network and firewall settings matter. Always verify port accessibility and DNS resolution when troubleshooting connectivity issues.
Final Thoughts
If you ever encounter the “RPC Server Unavailable” error in Veeam, start by checking service status and considering a reboot before diving into deeper network troubleshooting. Hopefully, this post will help you resolve similar issues quickly and efficiently.
Sometimes, the most straightforward fixes are right in front of us.
Happy troubleshooting!
– Dave