Thursday, October 31, 2013

robocopy data transfer is very sluggish

We often see DBAs talking that the robocopy data transfer is very sluggish.

Recently I was involved in a project where 14 TB data was being transferred. My DBA informed me that the data transfer rate is just 325 KB/sec. With this rate of transfer it would take 1.4 years to transfer 14 TB of data. There was an absolute need fix either n\w design or robocopy. I chose to review robocopy command.

My DBA was using command with /Z option and without any /MT multi threading. The original command looked like as follows:

robocopy \\SourceServer\Sourcefolder \\DestinationServer\Destinationfolder /Z

I recommended following changes and confirm the speed

robocopy \\SourceServer\Sourcefolder \\DestinationServer\Destinationfolder /MT:64
/Z actually slows the speed a lot. This is absolutely great option to avoid any data discrepancy but I opted not to use this option and in case of any failures, the copy will be retried.

The speed was 595 MB/sec after this change. The copy time decreased from 1.4 years to whopping 6.8 hours. I am completely ok to retry this if any bit transfer is flipped because of n\w issue and causes corruption.

No comments:

Post a Comment