Answers

Got a burning question about open source or the kernel? Whatever your level, email it to answers@linuxformat.com upgraded to v2.0; we’re still finding bugs…

Neil Bothwick has been

Q Speed is not everything

We need to copy a great deal of data from the client’s live server. While doing so, using the cp -R command, the server slows down quite a bit. Copying after 5pm, out of office hours, helps but we also need to copy during the day, at a lower speed, so people can use the server without annoying delays. How do we slow down the file copy command?

Corey Richardson

The short answer is that you do not use cp to copy the files. By using Rsync, you can limit the bandwidth used in file transfers, thanks to its --bwlimit option. The option takes a number that is interpreted as KiB per second, although you can also use a suffix to give a different unit, for example --bwlimit=2.5m.

Rsync has a number of other advantages over plain cp . Firstly, it can operate over an SSH connection, as it does by default when connecting to a remote host, which means all data travels through a secure tunnel. This may be important when transferring company data. Also, Rsync is well suited to recovering from interrupted transfers; run it again and it picks up where it left off. To make a copy of a directory on the remote server, you would use something like: $ rsync --bwlimit=5m user@server:/path/to/dir//path/to/local/copy/

Ensure that both paths include the trailing slash. This makes the local directory a copy of the remote one, copying only those files that are new or changed, another saving in bandwidth.

As you make the distinction between copying during and after office hours, you may also find --stop-at useful. This tells Rsync to abort the transfer at the given time, so you could run at higher speed at night, then resume at a lower rate when people in the office need the bandwidth. Running the following would transfer at a higher speed until 8am, then stop that transfer and resume it at a lower speed: $ rsync --bwlimit=10m --stop-at=08:00 user@server:/path/to/dir//path/to/local/copy/ $ rsync --bwlimit=2m user@server:/path/to/dir//path/to/local/copy/ If all files had been transferred before 8am, the second invocation would simply exit without copying anything.