<!------------------------------------------------------------------------
logs.config, v2.0


This is the configuration file for Traffic Server logging.  This file
defines log files, their formats, filters, and processing options.  The
format of this file is modeled after XML, the eXtensible Markup Language.
White-spaces, blank lines, and all comments are ignored.
-------------------------------------------------------------------------->

<!------------------------------------------------------------------------
LogFormats

An event log format specifies which fields are to be gathered from each
protocol event access.  The following tags are possible for LogFormat
specifications (a '*' denotes a tag that is required):
 
* <Name = "valid_format_name"/>
      Valid format names include anything except (squid, common, extended,
      extended2), which are the pre-defined formats.  There is no default.

* <Format = "valid_format_specification"/>
       A valid format specification is a printf-style string that describes
       what each log entry looks like when formatted for ascii output.
       Placeholders for valid Inktomi field names are specified using the
       notation '%<field>'.  The specified field can be of two types:
           (1) simple field: '%<field>', for example %<cqu>.
           (2) container field: '%<field>container', which is a field within
               a container (such as an http header or an Inktomi stat).  See
               documentation for valid container names.
       To limit the length of field's output, we can use slice notation with
       above two types:
           (1) simple field with slice: '%<field[start:end]>',
           (2) container field with slice: '%<field>container[start:end]'.
           Here is same example about slice notation:
            '%<cqup>[:]'    # the whole characters of <cqup>.
            '%<cqup[0:30]>' # the first 30 characters of <cqup>.
            '%<cqup[-10:]>' # the last 10 characters of <cqup>.
            '%<cqup[:-5]>'  # everything except the last 5 characters of <cqup>.
          Note: slice notation makes sense only when:
            * the target field is a string type, and
            * it shouldn't be ip/timestamp which are converted
              to string from integer internally.
       If you want to include quotes within the format string, escape them
       with a backslash. For example, to quote the client request url (cqu),
       you would type something like
       <Format = "\"%<cqu>\" %<pssc>"/>

  <Interval = "aggregate_interval_secs"/>
      This tag is needed when the format contains any of the aggregate
      operators.  The "aggregate_interval_secs" value is a number 
      representing the number of seconds over which the entry is aggregated 
      before being produced.  The valid set of aggregate operators are:
            COUNT	'*' can be used for the field value
	    SUM
	    AVG
	    FIRST
	    LAST

Example1: a simple format that contains 3 common fields:

<LogFormat>
    <Name = "minimal"/>
    <Format = "%<chi> : %<cqu> : %<pssc>"/>
</LogFormat>

Example2: a summary format that contains aggregate operators:

<LogFormat>
    <Name = "summary"/>
    <Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>"/>
    <Interval = "10"/>
</LogFormat>

-------------------------------------------------------------------------->

<!------------------------------------------------------------------------
LogFilters

Filters are used to include or exclude certain entries from being
logged, based on the value of a field within that entry.  A filter
contains the following tags (a '*' denotes a tag that is required):

* <Name = "valid_filter_name"/>
      All filters must be uniquely named.

* <Condition = "valid_log_field valid_filter_operator value"/>

      A condition establishes a test that will be performed for each record.
      The "valid_log_field" is compared to the "value" using the specified
      "valid_filter_operator". The result of this comparison is either true,
      or false. The action of the filter (see below) is based on this truth
      value.

      The "valid_log_field"s are all Inktomi fields that are not a combination
      of other (simpler) fields. 

      "valid_filter_operator"s are:
	MATCH
	CASE_INSENSITIVE_MATCH
	CONTAIN
	CASE_INSENSITIVE_CONTAIN
      These operators are all equivalent to MATCH for integer fields, but
      provide different functionality for string fields:
        MATCH: the field and the value must be identical strings
        CASE_INSENSITIVE_MATCH: the only difference between field and value
                                is the case
        CONTAIN: value is a substring of field (case matters)
        CASE_INSENSITIVE_CONTAIN: value is a substring of field (case does 
                                  not matter)

* <Action = "valid_action"/>
      This defines the action that the filter takes, and can be either
      ACCEPT, or REJECT. ACCEPT accepts the records that satisfy the Condition,
      REJECT rejects the records that satisfy the condition.

Example1: only log REFRESH_HIT entries

<LogFilter>
    <Name = "only_refresh_hits"/>
    <Condition = "pssc MATCH REFRESH_HIT"/>
    <Action = "ACCEPT"/>
</LogFilter>

Example2: do not log requests for domain unwanted.com

<LogFilter>
    <Name = "not_unwanted_domain"/>
    <Condition = "shn CASE_INSENSITIVE_CONTAIN unwanted.com"/>
    <Action = "REJECT"/>
</LogFilter>

-------------------------------------------------------------------------->

<!------------------------------------------------------------------------
 LogObject

 This section defines the LogObjects that will be collected on this Traffic
 Server.  Each LogObject has several tags from the following list. Only
 the Format and the Filename fields are mandatory.

  <Format = "valid_format_name"/>
      Valid format names include the pre-defined logging formats:
      (squid, common, extended, extended2), and any previously-defined
      custom LogFormats).  There is no default for this tag.

  <Filename = "file_name"/>
      This tag specifies the name of the file to which this log object
      writes. This can be a local file, or a file on a different machine
      if this Traffic Server is a collation client (see the LogHostName 
      tag below.)
      All filenames are relative to the default logging directory. If
      "file_name" has no file extension, then a ".log" extension will
      be added to it if the log is an ascii log (see the "Mode" tag
      below), and a ".blog" extension if it is a binary log. If
      "file_name" has a file extension then it is not modified at all
      regardless of the "Mode" tag. If "file_name" does not have a
      file extension, and you do not want one, then write a single dot
      at the end of "file_name". For example <Filename = "no_ext."/>
      will generate the name "no_ext" without the dot at the end. If
      you want a single dot at the end of the log filename, then you
      should specify two dots at the end of the Filename value. For
      example <Filename="with_dot.."/> will generate the filename
      "with_dot."

  <Mode = "valid_logging_mode"/>
      Valid logging modes include ascii, binary, and ascii_pipe.
      ascii: write log in human readable form (plain ascii).
      binary: write log in a binary format that can later be read using 
      the logcat utility.
      ascii_pipe: do not write log to a regular file, but to a named UNIX
      pipe (this option is only available on Linux and Solaris).

  <Filters = "list of valid filter names"/>
      A comma separated list of valid filter names to apply to this
      object. Valid filter names include any previously-defined LogFilter.
      If more than one filter is specified, then ALL filters should 
      accept a record for it to be logged (in other words, filters are
      ANDed together.)

  <Protocols = "list of protocols"/>
      The list of protocols to log is a comma separated list of the
      protocols that the object logs. Valid protocol names are http,
      icp, and mco. If the object has no Protocol tag, then it logs
      all protocols. The Protocol tag simply provides an easy way to
      create a filter that accepts the specified protocols.

  <ServerHosts = "list of servers"/>
      This tag provides an easy way to create a filter that logs only the
      requests to hosts in the comma separated list. Only entries from the 
      named servers will be included in the log file. (Servers can only
      be specified by name, not by ip.)

  <CollationHosts = "list of collation hosts"/>
      This tag specifies a comma separated list of collation hosts
      to which all of the log entries for the LogObject will be forwarded.
      Specify the hosts using the format host:port, where host can be
      either a name or an ip. For example:
      <CollationHosts = "host1.company.com:5000, 209.131.52.129:6000"/>

      Also, we can introduce failover hosts for each collation host by
      using '|' delimiter, For example:
      <CollationHosts = "host1:5000|host2:5000|host3:6000, 209.131.52.129:6000"/>

      In the example above, host2/host3 are failover hosts for host1. When host1
      disconnected, log entries will be sent to host2, and then if host2 failed
      again, log entries will be sent to host3 until host1 or host2 comes back.

  <Header = "header"/>
      This tag specifies a string to be written at the beginning of
      the log file, just before the first record.

  <RollingEnabled = "rolling_enabled_value"/>
      This tag specifies the automatic rolling mode for the LogObject.
      It overrides the configuration variable rolling_enabled. 
      The possible values of rolling_enabled_value are as follows:
      0: do not roll
         do not automatically roll this log file
      1: roll based on time
         roll this log file at a certain time frequency, specified by the
         RollingIntervalSec, and RollingOffsetHr tags (see below)
      2: roll based on size
         roll when the log file surpasses a certain size, specified by
         the RollingSizeMb tag (see below)
      3: roll based on time or size
         roll when either the specified rolling time is reached or the
         specified file size is reached
      4: roll based on time and size
	 roll the log file when the specified rolling time is reached
	 if the size of the file equals or exceeds the specified size

  <RollingIntervalSec = "seconds"/>
      This tag specifies the seconds between consecutive log file 
      rolls for the LogObject. It overrides the configuration variable
      rolling_interval_sec.

  <RollingOffsetHr = "hour"/>
      This tag specifies an hour (from 0 to 23) at which rolling is
      guaranteed to "align". Rolling may start before then, but a
      rolled file will be produced at that time. The impact of this
      setting is only noticeable if the rolling interval is larger
      than one hour. This tag overrides the configuration variable
      rolling_offset_hr.

  <RollingSizeMb = "log_size_in_mb"/>
      This tag specifies the size (in megabytes) the log file must reach
      before it is rolled if rolling is based on size.

  Please note: 

  - The "Format" and "Filename" tags are mandatory, all others are optional.

  - Any repeated tag will be ignored. For example, if there are two "Format"
    tags, the second one will be ignored.

  - If you do not see the expected results from the LogObjects you
    define, check the diags.log file for syntax errors that may have
    prevented the objects from being created.

  - You can specify any of the RollingEnabled, RollingIntervalSec, and
    RollingOffsetHr tags for one object. The values you do not specify
    will be taken from the equivalent configuration variables in the
    records.config file.


  Examples:
  =========

  Example1: create a local log file for the minimal format defined above.

  <LogObject>
      <Format = "minimal"/>
      <Filename = "minimal"/>
  </LogObject>

  Example2: create a local log file for the minimal format defined above, but
  do not log the domain unwanted.com (use the filter defined
  above). Create a binary log rather than an ascii log.
 
  <LogObject>
      <Format = "minimal"/>
      <Filename = "minimal_without_unwanted"/>
      <Filters = "not_unwanted_domain"/>
      <Mode = "binary"/>
  </LogObject>

  Example3: like example2, but log only REFRESH_HIT entries (from all domains 
  except unwanted.com) and use the default ascii mode.

  <LogObject>
      <Format = "minimal"/>
      <Filename = "minimal_without_unwanted"/>
      <Filters = "not_unwanted_domain, only_refresh_hits"/>
  </LogObject>

  Example4: create a remote log on the collation host
  remote.company.com (listening on port 4000). Use the "minimal"
  format defined above, and name the log "minimal_remote".

  <LogObject>
      <Format = "minimal"/>
      <Filename = "minimal_remote"/>
      <CollationHosts = "remote.company.com:4000"/>
  </LogObject>

  Example5: create a local log using the pre-defined "squid" format.
  Log only the http and icp protocols, and log only entries from the
  servers s1.com1.com, and s2.com2.com

  <LogObject>
      <Format = "squid"/>
      <Filename = "squid_http_icp_selected_servers"/>
      <Protocols = "http, icp"/>
      <ServerHosts = "s1.com1.com, s2.com2.com"/>
  </LogObject>

  Example6: write to a UNIX named pipe using the pre-defined "squid" format.
  The name of the pipe will be "squid.pipe".

  <LogObject>
      <Format = "squid"/>
      <Filename = "squid"/>
      <Mode = "ascii_pipe"/>
  </LogObject>

  Example7: create a log using the pre-defined "common" format, and
  have the traffic_server roll it at 8AM and 8PM regardless of the
  setting of the rolling_enabled configuration variable.

  <LogObject>
      <Format = "common"/>
      <Filename = "common-8h"/>
      <RollingEnabled = "1"/>
      <RollingIntervalSec = "43200"/>
      <RollingOffsetHr = "8"/>
  </LogObject>

  Example8: create a log using the pre-defined "common" format, and
  have the traffic_server roll it every hour if rolling has been
  enabled through the rolling_enabled configuration variable (note
  that the RollingOffsetHr setting is irrelevant in this case).

  <LogObject>
      <Format = "common"/>
      <Filename = "common-1h"/>
      <RollingIntervalSec = "3600"/>
  </LogObject>

-------------------------------------------------------------------------->


<!------------------------------------------------------------------------

WebTrends Enhanced Log Format
-----------------------------

The following <LogFormat> is compatible with the WebTrends Enhanced Log
Format. If you want to generate a log that can be parsed by WebTrends 
reporting tools, simply create a <LogObject> that uses this format. 

-------------------------------------------------------------------------->

<LogFormat>
  <Name = "welf"/>
  <Format = "id=firewall time=\"%<cqtd> %<cqtt>\" fw=%<phn> pri=6 proto=%<cqus> duration=%<ttmsf> sent=%<psql> rcvd=%<cqhl> src=%<chi> dst=%<shi> dstname=%<shn> user=%<caun> op=%<cqhm> arg=\"%<cqup>\" result=%<pssc> ref=\"%<{Referer}cqh>\" agent=\"%<{user-agent}cqh>\" cache=%<crc>"/>
</LogFormat>


<!------------------------------------------------------------------------

squid Log Format with seconds resolution timestamp
--------------------------------------------------

The following <LogFormat> is the squid format but with a seconds-only
timestamp (cqts) instead of a seconds and milliseconds timestamp
(cqtq).  The default squid format (enabled by setting the
configuration variable proxy.records.log.squid_log_enabled or through
the GUI) used to generate seconds-only timestamps. Starting in Traffic
Server 5.2, it generates millisecond resolution timestamps that are
compliant with the format of the squid cache.
If you wish to generate a squid log with seconds resolution timestamp
simply create a <LogObject> that uses this format.

-------------------------------------------------------------------------->

<LogFormat>
  <Name = "squid_seconds_only_timestamp"/>
  <Format = "%<cqts> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun> %<phr>/%<pqsn> %<psct>"/>
</LogFormat>


