separate windows drives/mount points ? Yes, there is a way to setup and run backups on multiple drives at the same time. Talk to support and they will give you the procedure. I had a Windows 2k8 R2 cluster with 8 drives, each drive was 10TB so you can imagine running one backup would not cut it. Support gave me the procedure that allowed me to run 8 backup simultaneously. You do have to be careful with CPU/Memory/Disk I/O utilization when you do that. Mine was running on pretty beefy Cisco UCS blade so it was fine.
i am not sure you are talking to the right people or they understand what you are trying to do. Level 2 support had the procedure. The procedure basically involves setting up multiple avamar services running on the same Windows host, each service will be using a different port, appear as different avamar client and require special dataset for each drive letter you want to backup.
You will need work with EMC-support and at an escalation. The Two parameters to put into the Dataset, those values are hidden, purposely not documented, but only in internal documents, as it may cause other issues like over CPU and RAM utilization.
Starting in 5.x (year 2009 or about); in your dataset for the file-backup, please , parallel=true; multicore=true – both have to be set. Now keeping in mind as the avtar will be processing multiple drives (# of streams based on the number of drives you have in the dataset), thus we engage support is that CPU utilization on the "client" doing the backup will go up higher during the backup- window. So you may want to experiment by putting in 3 independent drives in the dataset and seeing (example f:\ g:\, h:\)
Also the other issue you may be having; as the version of Avamar Client has not been documented in your posting, also if Avamar is integrated to Data Domain, that the client "cache files", may be "off" which is based on the number of files and that "cache" files May need tuning as well. Need to look at the daily backup log, also back in the Dataset, at least enable "Report Advanced statistics" .
So in general if the RAM on the client is not limited and the CPU is not, one should expect at least 100 GB/HR of backup and lets say 1 Million files per 1TB (1000GB)- even without parallel turned on ... so there is something else going on in your environment, may be client-cashes that may need tuning, so please review the "backup log" for your backup and search for the word, "booted" (notes my MAC for 273 GB has ~1 Million files, yet only 327.5 MB was unique / changed data)
Here is my Mac backup o May 7th, backup log
2016-05-07 20:24:21 avtar Info <5156>: Backup #308 timestamp 2016-05-07 20:19:48, 1,034,180 files, 133,483 directories, 273.8 GB (3,053 files, 327.5 MB, 0.12% new)
2016-05-07 20:24:21 avtar Info <7539>: Label "Mac-Laptop-Sch-630pm-130am-Laptop-Mac-Policy-1462660200004", scheduled to expire after 07/06/16 (2016-07-06 22:30:00 UTC), daily backup
2016-05-07 20:24:21 avtar Info <6083>: Backed-up 273.8 GB in 109.80 minutes: 150 GB/hour (565,134 files/hour)
Also look at:
2016-05-07 20:24:21 avtar Info <5587>: Updating cache files in /usr/local/avamar/var
2016-05-07 20:24:21 avtar Info <5069>: - Writing cache file "/usr/local/avamar/var/f_cache.dat"
2016-05-07 20:24:22 avtar Info <5546>: Cache update complete /usr/local/avamar/var/f_cache.dat (352.0 MiB of 1024 MiB max)
2016-05-07 20:24:22 avtar Stats <6151>: File cache: entries added/updated 1,032,839, booted 0
2016-05-07 20:24:22 avtar Stats <18857>: Backup statistics for filename cache (monolithic) in '/usr/local/avamar/var/f_cache.dat'
2016-05-07 20:24:22 avtar Stats <0000>: -- elements: 1,032,839 entered, 1,032,839 looked up, 0 booted (not recorded)
2016-05-07 20:24:22 avtar Stats <18859>: -- elements (lookup results): 3,051 missed
2016-05-07 20:24:22 avtar Stats <18860>: -- elements (hash table probes): minimum 1, maximum 21, mean 1.211 sdev 0.662
2016-05-07 20:24:22 avtar Info <5069>: - Writing cache file "/usr/local/avamar/var/p_cache.dat"
2016-05-07 20:24:24 avtar Info <5546>: Cache update complete /usr/local/avamar/var/p_cache.dat (384.0 MiB of 512 MiB max)
2016-05-07 20:24:24 avtar Stats <6152>: Hash cache: entries added/updated 184,376, booted 0
- Feel free to review the Avamar Operational Best Practices Document that is on Avamar System http://Youravamarsystem, ignore the browser certificate warning and continue, then "Documentation" link, then Search or scroll for "Avamar Operational Best Practices" guide to review the "cache" guidelines and or as I started to get EMC support engaged.
Slow backup performance is most often caused by some factor other than lack of parallelism. Implementing and managing parallelism has a number of caveats and challenges, especially for methods like running multiple avtars in parallel. The reason people are insisting you work with support is that support can help you analyze your environment and determine if there is some other factor that needs to be addressed before opening the parallelism can of worms. There are also several different ways to implement parallelism and if parallelism is actually needed, they can point you toward the appropriate type to use.
Shooting in the dark is not going to help you. Please work with support on this. You will likely need to work with L2 support.
FYI, setting --parallel for Data Domain integrated backups is known to cause issues and is not supported.
Hi Suhas - I see that you've opened an SR with support. A colleague of mine has taken ownership of it and I'll be working with him to get a better understanding of the situation.
In order to facilitate this you will be asked to provide a full set of information as described in the following article
- KB 303720 - Information required by Support when investigating Avamar client backup performance issues
Please provide this information as soon as you can and in its entirety so we have the best opportunity to be effective.
Note that there are many many possible causes of backups not completing within the available backup window and we'll be using the following article as a general troubleshooting template. I recommend reviewing it to understand more about these types of challenges and the things we will look for
- KB 335029 - How to understand Avamar Client backup performance and identify performance bottlenecks
In some cases a client might be backing up with a high rate of performance, but the amount of data to be handled means that there simply isn't enough time within the available window. If that's the case, there may be a need for some careful optimisation.
Finally, I should add that if this client has never previously backed up successfully within the backup window, then we will need to engage the Professional Services team to work directly on it with you as a client implementation. In that case, Support will collaborate with the Professional Services delivery person.
Just to wrap this one up...
The main reason the client was experiencing slow backup performance was due to file cache misses.
2016-05-15 07:24:31 avtar Warning <5650>: CAPACITY WARNING: The file cache is full; filecachemax=511MB, should be at least 2817MB
Due to the large size of the dataset and the client having a 32 bit architecture it wasn't possible to size the monolithic cache large enough to accommodate entries for all files that were being backed up.
The Operational Best Practices Guide documents a suggestion for this type of scenario "using cacheprefix"
Also refer to