Throttling rsync (curiosity question mostly)

Issues related to configuring your network
Post Reply
lightman47
Posts: 1523
Joined: 2014/05/21 20:16:00
Location: Central New York, USA

Throttling rsync (curiosity question mostly)

Post by lightman47 » 2023/02/23 17:38:20

RHEL 8/9 and CentOS 7

I have backup scripts that utilize rsync. When I am running a backup it pretty much consumes my network connection, be it it wifi or sometimes even cat-5. I am wondering if there is a way to (within the script) 'throttle' rsync by like 10% or whatever? I'd just like it to not over-ride other programs also trying to use the connection.

I've looked into the rsync switches but didn't see anything obvious to me.

Thank you.

User avatar
TrevorH
Site Admin
Posts: 33280
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Throttling rsync (curiosity question mostly)

Post by TrevorH » 2023/02/23 18:16:37

You mean like --bwlimit=RATE limit socket I/O bandwidth ?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

lightman47
Posts: 1523
Joined: 2014/05/21 20:16:00
Location: Central New York, USA

Re: Throttling rsync (curiosity question mostly)

Post by lightman47 » 2023/02/23 19:06:48

Thank you -- clearly I (again) got distracted while looking - I'm always "doing 18 things at a time".

lightman47
Posts: 1523
Joined: 2014/05/21 20:16:00
Location: Central New York, USA

Re: Throttling rsync (curiosity question mostly)

Post by lightman47 » 2023/03/31 16:43:37

This works really great - UNTIL rsync reads the end of the file and then the cache begins emptying unrestricted. The result is back to a "hung" network until the cache empties.

Without turning off write-caching entirely, can I somehow 'adjust' it? I looked at https://linuxconfig.org/improve-hard-dr ... ck-caching

Thanks

User avatar
TrevorH
Site Admin
Posts: 33280
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Throttling rsync (curiosity question mostly)

Post by TrevorH » 2023/03/31 17:14:49

You could try playing with the various sysctls like these ones:

vm.dirty_writeback_centisecs = 200
vm.dirty_ratio = 15
vm.dirty_background_ratio = 3
vm.dirty_expire_centisecs = 500
vm.swappiness = 3
vm.vfs_cache_pressure = 170
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply