Throttling throughput in rsync is actually really simple to do. All the information you need to know is in the man pages for rsync, but it is so very surprising to see how many people asking for an answer on the web.
You may have a need from time to time to rsync some data from one location to another, but you may be concerned of either saturating your link to your remote location or getting a hefty bill at the end of the month for over usage.
This is normally common if you were to be syncing via a VPN or internet uplink.
For example, lets say you have a contract with your internet provider to give you internet connectivity and your contractual arrangement is for 10MB/s. However, your supplier gives you a 1Gbps link on a 95 percentile agreement.
Basically this means you are paying for your 10MB/s, but you won’t be hit with a bill for excess usage unless you go over 10MB/s for more than 5% of your allotted billing period.
Getting a huge bill is always a good motivator to not going over your limit so bandwidth throttling will come in handy.
With rsync this is very easy, and you can use the “–bwlimit” option for any process. Be it rsyncing local data to another local disk, or rsyncing over a network connection. Its all the same.
Lets say your normal command looks something like this. (To show how this works, I am rsyncing my RHEL 6 DVD iso to another folder on my laptop.
rsync -avzP /home/mac/Downloads/rhel-server-6.3-x86_64-dvd.iso /home/mac/Desktop/
To limit the throughput, you would add the “–bwlimit” flag. Below I have limited it to 2000KB/s
rsync -avzP –bwlimit=2000 /home/mac/Downloads/rhel-server-6.3-x86_64-dvd.iso /home/mac/Desktop/
Lets have a look at an unlimited rsync
mac@stark:~$ rsync -avzSP /home/mac/Downloads/rhel-server-6.3-x86_64-dvd.iso /home/mac/Desktop/ sending incremental file list rhel-server-6.3-x86_64-dvd.iso 3589636096 100% 16.74MB/s 0:03:24 (xfer#1, to-check=0/1) sent 3590841223 bytes received 31 bytes 17473680.07 bytes/sec total size is 3589636096 speedup is 1.00 mac@stark:~$
In the above scenario, we can see the average throughput was 16.74MB/s and it took 3 minutes and 24 seconds to complete.
Now lets use an extremely low limit in comparison to the above so we can really see the difference. In the below scenario I have limited the throughput to 2000KBps which works out to be 1.95MB/s. Do achieve this, I have simply added “–bwlimit=2000”.
mac@stark:~$ rsync -avzP --bwlimit=2000 /home/mac/Downloads/rhel-server-6.3-x86_64-dvd.iso /home/mac/Desktop/ sending incremental file list rhel-server-6.3-x86_64-dvd.iso 3589636096 100% 1.90MB/s 0:29:59 (xfer#1, to-check=0/1) sent 3590841223 bytes received 31 bytes 1995466.10 bytes/sec total size is 3589636096 speedup is 1.00 mac@stark:~$
Now we can see that the average throughput is 1.90MB/s and this took 29 minutes and 59 seconds.
Its as simple as that folks.