\chapter{File Deduplication using Base Jobs}
\index[general]{Base Jobs}
\index[general]{File Deduplication}
\label{basejobs}
A base job is sort of like a Full save except that you will want the FileSet to
contain only files that are unlikely to change in the future (i.e.  a snapshot
of most of your system after installing it).  After the base job has been run,
when you are doing a Full save, you specify one or more Base jobs to be used.
All files that have been backed up in the Base job/jobs but not modified will
then be excluded from the backup.  During a restore, the Base jobs will be
automatically pulled in where necessary.

This is something none of the competition does, as far as we know (except
perhaps BackupPC, which is a Perl program that saves to disk only).  It is big
win for the user, it makes Bacula stand out as offering a unique optimization
that immediately saves time and money.  Basically, imagine that you have 100
nearly identical Windows or Linux machine containing the OS and user files.
Now for the OS part, a Base job will be backed up once, and rather than making
100 copies of the OS, there will be only one.  If one or more of the systems
have some files updated, no problem, they will be automatically restored.

A new Job directive \texttt{Base=Jobx, Joby...} permits to specify the list of
files that will be used during Full backup as base.

\begin{verbatim}
Job {
   Name = BackupLinux
   Level= Base
   ...
}

Job {
   Name = BackupZog4
   Base = BackupZog4, BackupLinux
   Accurate = yes
   ...
}
\end{verbatim}

In this example, the job \texttt{BackupZog4} will use the most recent version
of all files contained in \texttt{BackupZog4} and \texttt{BackupLinux}
jobs. Base jobs should have run with \texttt{level=Base} to be used.

By default, Bacula will compare permissions bits, user and group fields,
modification time, size and the checksum of the file to choose between the
current backup and the BaseJob file list. You can change this behavior with the
\texttt{BaseJob} FileSet option. This option works like the \texttt{verify=}
one, that is described in the \ilink{FileSet}{FileSetResource} chapter.

\begin{verbatim}
FileSet {
  Name = Full
  Include = {
    Options {
       BaseJob  = pmugcs5
       Accurate = mcs
       Verify   = pin5
    }
    File = /
  }
}
\end{verbatim}

\textbf{Important note}: The current implementation doesn't permit to scan
volume with \textbf{bscan}. The result wouldn't permit to restore files
easily. It is recommended to not prune File or Job records with Basejobs.

\smallskip

Added in version 8.0.5, the new ``M'' option letter for the Accurate directive
in the FileSet Options block, which allows comparing the modification time and/or
creation time against the last backup timestamp. This is in contrast to the
existing options letters ``m'' and/or ``c'', mtime and ctime, which are checked
against the stored catalog values, that could vary accross different machines
when using the BaseJob feature.

The advantage of the new ``M'' option letter for Jobs that refer to BaseJobs is
that it will backup files based on the last backup time, which is more useful,
because the mtime/ctime timestamps may differ on various Clients, causing
unnecessary files to be backed up.

\begin{verbatim}
  Job {
    Name = USR
    Level = Base
    FileSet = BaseFS
    ...
  }

  Job {
    Name = Full
    FileSet = FullFS
    Base = USR
    ...
  }

  FileSet {
    Name = BaseFS
    Include {
      Options {
        Signature = MD5
      }
      File = /usr
    }
  }

  FileSet {
    Name = FullFS
    Include {
      Options {
        Accurate = Ms      # check for mtime/ctime and Size
        Signature = MD5
      }
      File = /home
      File = /usr
    }
  }
\end{verbatim}
