Rsync is a UNIX command that can keep two file folders synchronized. (For Windows users, see Use Linux Commands and Shell Scripts directly in Windows.) In part one of this article, I will describe what rsync is and how it works, and I’ll give examples of how it can be used. In part two, I’ll go through a detailed example, step by step.

Rsync is basically a copy command on steroids. The number of options available for rsync is exhaustive (something on the order of 100 different switches). See the man pages for details (”man rsync”).

rsync -vrut --filter='. searchcopy_filelist.txt'
work /cygdrive/e/work_search

As is common with Unix commands, the command line switches can be spelled out, in which case they begin with two hyphens (”–filter”), or they are abbreviated to a single letter, in which case they began with one hyphen (”-f”). single letter versions of the command line switches can be combined, thus “-vrut” is the equivalent of “-v -r -u -t”. The two different styles of switches can be intermixed. Casing is significant. (”-b” and “-B” are two completely different switches.)

The example above (which is supposed to be all on one line) consists of five switches plus the specification of the source folder and the specification of the destination folder.

• -v is the “verbose” switch
• -r is the “recursive” switch, which says to process subfolders
• -u is the “update” switch, which says not to overwrite corresponding files that already exist in the destination folder, if they have a newer timestamp
• -t is the “times” switch, which says to preserve the timestamp when making a copy (as opposed to using the current time)
• –filter is the switch that allows for precise control of determining which files and/or folders are transferred. In this case, we are using a rule that says read more rules from a file, namely “searchcopy_filelist.txt”
• The first command line argument that does not begin with a dash, namely “work” in this case, is the name of the source folder (relative to the current folder).
• The last command line argument that does not begin with a dash, namely “/cygdrive/e/work_search” in this case, is the name of the destination folder
• If there are more than two command line switches that do not begin with a dash, then all but the last one are considered source folders, which are processed successively.

Lickety Split: Rsync’s main claim to fame is that it’s fast when operating remotely because, if the remote target system already has an old copy of a particular file, then Rsync will only transmit the differences. This process is referred to as the rsync remote-update protocol. This means that rsync can be executed as frequently as desired without sacrificing bandwidth resources.

Precise Control: Rsync is also quite powerful with respect to specifying exactly which files and folders are to be synchronized and how. The two folders being synchronized do not need to be exact copies. One can be a subset, or superset, of the other, according to (partial) matches on subfolder names, the file names, and/or the filename extensions. The matches are defined using regular expressions, thus there is quite a bit of power there.

Local vs. Remote: The R in “rsync” stands for remote, but the folders could be local to each other as well. As far as rsync is concerned, “local” means that both file folders are controlled by the same computer, either on the same disk drive or different disk drives. “Remote” means that the file folders are controlled by two different computers connected across a network. For rsync to work remotely, two copies of rsync need to be running. First, rsync needs to be started in daemon mode at one end of the connection, and then another copy of rsync would be fired up (in client mode) at the other end.

Possible Uses: Here’s a short list of some possible uses for rsync, ending with an old classic. (Note that the terms “original” and “mirror” are my designations. The rsync documentation refers to them as “source” and “destination”.) In all of these examples, the assumption is that the original location is regularly being updated with changes and new additions, and that the mirror(s) need to keep up with those changes.

This list is by no means exhaustive. It is only intended to spark the imagination. In the case of making a shadow copy of the source code from a set of working folders, for example, there could be many different motivations. Perhaps, there is a legal obligation to place a copy of the source code into escrow. Perhaps, there is simply a need to make a snapshot of the files for posterity (although a version control system such as Subversion would be a better choice for that). Or, perhaps, a subset of the working folders sans library code/examples would be easier to search and navigate for developers newly assigned to the team who are not yet familiar with the CodeBase.

Read more: Uncategorized, Productivity, Linux