Leevid PowerCPS Server¶

Before you start¶

Before you get started with Traffic Server you may have to decide which version you want to use. Traffic Server follows the Semantic Versioning guidelines, in summary

A version is made of a version-triplet: MAJOR.MINOR.PATCH

As of v4.0.0, there are no longer any development (or unstable) releases. All releases are considered stable and ready for production use, releases within a major version are always upgrade compatible. More details are available on the Wiki page.

Sometimes we speak of trunk, master or HEAD, all of which are used interchangeably: trunk or master or sometimes TIP or HEAD, refer to the latest code in a Git Version Control System. Master is always kept releasable, and compatible with the current major release version. Incompatible changes are sometimes committed on a next-major release branch, for example we have the 5.0.x branch where changes incompatible with 4.x are managed.

If your distribution does not come with a prepackaged Traffic Server, please go to downloads to choose the version that you consider most appropriate for yourself. If you want to really be on the bleeding edge you can clone our git repository.

Please note that while we do have a GitHub Mirror that you can also use to submit pull requests, it may not be entirely up-to-date.

Building Traffic Server¶

In order to build Traffic Server from source you will need the following (development) packages:

• pkgconfig
• libtool
• gcc (>= 4.3 or clang > 3.0)
• make (GNU Make!)
• openssl
• tcl
• expat
• pcre
• libcap
• flex (for TPROXY)
• hwloc
• lua
• curses
• curl (both for traffic_top)

if you’re building from a git clone, you’ll also need

• git
• autoconf
• automake

We will show-case a build from git:

git clone https://git-wip-us.apache.org/repos/asf/trafficserver.git

Next, we cd trafficserver and run:

autoreconf -if

This will generate a configure file from configure.ac, so now we can run that:

./configure --prefix=/opt/ats

Note well, that by default Traffic Server uses the user nobody, as well as user’s primary group as Traffic Server user. If you want to change that, you can override it here:

./configure --prefix=/opt/ats --with-user=tserver

If dependencies are not in standard paths (/usr/local or /usr), you need to pass options to configure to account for that:

./configure --prefix=/opt/ats --with-user=tserver --with-lua=/opt/csw

Most configure path-options accept a format of "INCLUDE_PATH:LIBRARY_PATH":

./configure --prefix=/opt/ats --with-user=tserver --with-lua=/opt/csw \
   --with-pcre=/opt/csw/include:/opt/csw/lib/amd64

We can run make to build the project. We highly recommend to run make check to verify the build’s general sanity:

make
make check

We can finally run make install to install (you may have to switch to root to do this):

sudo make install

We also recommend to run a regression test. Please note that this will only work successfully with the default layout:

cd /opt/ats
sudo bin/traffic_server -R 1

After you have installed Traffic Server on your system, you can do any of the following:

Start Traffic Server¶

To start Traffic Server manually, issue the trafficserver command, passing in the attribute start. This command starts all the processes that work together to process Traffic Server requests as well as manage, control, and monitor the health of the Traffic Server system.

To run the trafficserver start command, e.g.:

bin/trafficserver start

At this point your server is up and running in the default configuration of a Reverse Proxy and HTTP Redirects.

Start Traffic Line¶

Traffic Line provides a quick way of viewing Traffic Server statistics and configuring the Traffic Server system via command-line interface. To execute individual commands or script multiple commands, refer to traffic_line.

Traffic Line commands take the following form:

bin/traffic_line -command argument

For a list of traffic_line commands, enter:

bin/traffic_line -h

Please note that traffic_line, while a fine tool for an administrator, is a poor choice for automation, especially that of monitoring. See our chapter on Monitoring Traffic for how to do that better.

Stop Traffic Server¶

To stop Traffic Server, always use the trafficserver command, passing in the attribute stop. This command stops all the Traffic Server processes (traffic_manager, traffic_server, and traffic_cop). Do not manually stop processes, as this can lead to unpredictable results.:

bin/trafficserver stop

Understanding HTTP Web Proxy Caching¶

Internet users direct their requests to web servers all over the Internet. A caching server must act as a web proxy server so it can serve those requests. After a web proxy server receives requests for web objects, it either serves the requests or forwards them to the origin server (the web server that contains the original copy of the requested information). The Traffic Server proxy supports explicit proxy caching, in which the user’s client software must be configured to send requests directly to the Traffic Server proxy. The following overview illustrates how Traffic Server serves a request.

1. Traffic Server receives a client request for a web object.

2. Using the object address, Traffic Server tries to locate the requested object in its object database (cache).

3. If the object is in the cache, then Traffic Server checks to see if the object is fresh enough to serve. If it is fresh, then Traffic Server serves it to the client as a cache hit (see the figure below).

   

   A cache hit

4. If the data in the cache is stale, then Traffic Server connects to the origin server and checks if the object is still fresh (a revalidation). If it is, then Traffic Server immediately sends the cached copy to the client.

5. If the object is not in the cache (a cache miss) or if the server indicates the cached copy is no longer valid, then Traffic Server obtains the object from the origin server. The object is then simultaneously streamed to the client and the Traffic Server local cache (see the figure below). Subsequent requests for the object can be served faster because the object is retrieved directly from cache.

   

   A cache miss

Caching is typically more complex than the preceding overview suggests. In particular, the overview does not discuss how Traffic Server ensures freshness, serves correct HTTP alternates, and treats requests for objects that cannot/should not be cached. The following sections discuss these issues in greater detail.

Ensuring Cached Object Freshness¶

When Traffic Server receives a request for a web object, it first tries to locate the requested object in its cache. If the object is in cache, then Traffic Server checks to see if the object is fresh enough to serve. For HTTP objects, Traffic Server supports optional author-specified expiration dates. Traffic Server adheres to these expiration dates; otherwise, it picks an expiration date based on how frequently the object is changing and on administrator-chosen freshness guidelines. Objects can also be revalidated by checking with the origin server to see if an object is still fresh.

Scheduling Updates to Local Cache Content¶

To further increase performance and to ensure that HTTP objects are fresh in the cache, you can use the Scheduled Update option. This configures Traffic Server to load specific objects into the cache at scheduled times. You might find this especially beneficial in a reverse proxy setup, where you can preload content you anticipate will be in demand.

To use the Scheduled Update option, you must perform the following tasks.

• Specify the list of URLs that contain the objects you want to schedule for update,
• the time the update should take place,
• and the recursion depth for the URL.
• Enable the scheduled update option and configure optional retry settings.

Traffic Server uses the information you specify to determine URLs for which it is responsible. For each URL, Traffic Server derives all recursive URLs (if applicable) and then generates a unique URL list. Using this list, Traffic Server initiates an HTTP GET for each unaccessed URL. It ensures that it remains within the user-defined limits for HTTP concurrency at any given time. The system logs the completion of all HTTP GET operations so you can monitor the performance of this feature.

Traffic Server also provides a Force Immediate Update option that enables you to update URLs immediately without waiting for the specified update time to occur. You can use this option to test your scheduled update configuration (refer to Forcing an Immediate Update).

Pushing Content into the Cache¶

Traffic Server supports the HTTP PUSH method of content delivery. Using HTTP PUSH, you can deliver content directly into the cache without client requests.

PUSH http://www.company.com HTTP/1.0
Content-length: 84

HTTP/1.0 200 OK
Content-type: text/html
Content-length: 17

<HTML>
a
</HTML>

Pinning Content in the Cache¶

The Cache Pinning Option configures Traffic Server to keep certain HTTP objects in the cache for a specified time. You can use this option to ensure that the most popular objects are in cache when needed and to prevent Traffic Server from deleting important objects. Traffic Server observes Cache-Control headers and pins an object in the cache only if it is indeed cacheable.

To set cache pinning rules

1. Make sure the following variable in records.config is set

   • proxy.config.cache.permit.pinning

2. Add a rule in cache.config for each URL you want Traffic Server to pin in the cache. For example:

   url_regex=^https?://(www.)?apache.org/dev/ pin-in-cache=12h

3. Run the command traffic_line -x to apply the configuration changes.

To Cache or Not to Cache?¶

When Traffic Server receives a request for a web object that is not in the cache, it retrieves the object from the origin server and serves it to the client. At the same time, Traffic Server checks if the object is cacheable before storing it in its cache to serve future requests.

Caching HTTP Objects¶

Traffic Server responds to caching directives from clients and origin servers, as well as directives you specify through configuration options and files.

Forcing Object Caching¶

You can force Traffic Server to cache specific URLs (including dynamic URLs) for a specified duration, regardless of Cache-Control response headers.

To force document caching

1. Add a rule for each URL you want Traffic Server to pin to the cache cache.config:

   url_regex=^https?://(www.)?apache.org/dev/ ttl-in-cache=6h

2. Run the command traffic_line -x to apply the configuration changes.

Caching HTTP Alternates¶

Some origin servers answer requests to the same URL with a variety of objects. The content of these objects can vary widely, according to whether a server delivers content for different languages, targets different browsers with different presentation styles, or provides different document formats (HTML, XML). Different versions of the same object are termed alternates and are cached by Traffic Server based on Vary response headers. You can specify additional request and response headers for specific Content-Types that Traffic Server will identify as alternates for caching. You can also limit the number of alternate versions of an object allowed in the cache.

Using Congestion Control¶

The Congestion Control option enables you to configure Traffic Server to stop forwarding HTTP requests to origin servers when they become congested. Traffic Server then sends the client a message to retry the congested origin server later.

To use the Congestion Control option, you must perform the following tasks:

1. Set the variable proxy.config.http.congestion_control.enabled to 1
   • Create rules in the congestion.config file to specify:
   • which origin servers Traffic Server tracks for congestion
   • the timeouts Traffic Server uses, depending on whether a server is congested
   • the page Traffic Server sends to the client when a server becomes congested
   • if Traffic Server tracks the origin servers per IP address or per hostname
2. Run the command traffic_line -x to apply the configuration changes.

Using Transaction Buffering Control¶

By default I/O operations are run at full speed, as fast as either Traffic Server, the network, or the cache can go. This can be problematic for large objects if the client side connection is significantly slower. In such cases the content will be buffered in ram while waiting to be sent to the client. This could potentially also happen for POST requests if the client connection is fast and the origin server connection slow. If very large objects are being used this can cause the memory usage of Traffic Server to become very large.

This problem can be ameloriated by controlling the amount of buffer space used by a transaction. A high water and low water mark are set in terms of bytes used by the transaction. If the buffer space in use exceeds the high water mark, the connection is throttled to prevent additional external data from arriving. Internal operations continue to proceed at full speed until the buffer space in use drops below the low water mark and external data I/O is re-enabled.

Although this is intended primarily to limit the memory usage of Traffic Server it can also serve as a crude rate limiter by setting a buffer limit and then throttling the client side connection either externally or via a transform. This will cause the connection to the origin server to be limited to roughly the client side connection speed.

Traffic Server does network I/O in large chunks (32K or so) and therefore the granularity of transaction buffering control is limited to a similar precision.

The buffer size calculations include all elements in the transaction, including any buffers associated with transform plugins.

Transaction buffering control can be enabled globally by using configuration variables or by TSHttpTxnConfigIntSet() in a plugin.

Be careful to always have the low water mark equal or less than the high water mark. If you set only one, the other will be set to the same value.

If using TSHttpTxnConfigIntSet(), it must be called no later than TS_HTTP_READ_RESPONSE_HDR_HOOK.

Reducing Origin Server Requests (Avoiding the Thundering Herd)¶

When an object can not be served from cache, the request will be proxied to the origin server. For a popular object, this can result in many near simultaneous requests to the origin server, potentially overwhelming it or associated resources. There are several features in Traffic Server that can be used to avoid this scenario.

Reverse Proxy and HTTP Redirects¶

As a reverse proxy cache, Traffic Server serves requests on behalf of origin servers. Traffic Server is configured in such a way that it appears to clients like a normal origin server.

Understanding Reverse Proxy Caching¶

With forward proxy caching, Traffic Server handles web requests to distant origin servers on behalf of the clients requesting the content. Reverse proxy caching (also known as server acceleration or virtual web hosting) is different because Traffic Server acts as a proxy cache on behalf of the origin servers that store the content. Traffic Server is configured to be the origin server which the client is trying to connect to. In a typical scenario the advertised hostname of the origin server resolves to Traffic Server, which acts as the real origin server.

Reverse Proxy Solutions¶

There are many ways to use Traffic Server as a reverse proxy. Below are a few example scenarios.

You can use Traffic Server in reverse proxy mode to:

• Offload heavily-used origin servers
• Deliver content efficiently in geographically distant areas
• Provide security for origin servers that contain sensitive information

Offloading Heavily-Used Origin Servers¶

Traffic Server can absorb requests to the main origin server and improve the speed & quality of web serving by reducing load and hot spots on backup origin servers. For example, a web hoster can maintain a scalable Traffic Server serving engine with a set of low-cost, low-performance, less-reliable PC origin servers as backup servers. In fact, a single Traffic Server can act as the virtual origin server for multiple backup origin servers, as shown in the figure below.

Traffic Server as reverse proxy for a pair of origin servers

Delivering Content in Geographically-Dispersed Areas¶

Traffic Server can be used in reverse proxy mode to accelerate origin servers that provide content to areas not located within close geographical proximity. Caches are typically easier to manage and are more cost-effective than replicating data. For example, Traffic Server can be used as a mirror site on the far side of a trans-Atlantic link to serve users without having to fetch the request and content across expensive international connections. Unlike replication, for which hardware must be configured to replicate all data and to handle peak capacity, Traffic Server dynamically adjusts to optimally use the serving and storing capacity of the hardware. Traffic Server is also designed to keep content fresh automatically, thereby eliminating the complexity of updating remote origin servers.

Providing Security for an Origin Server¶

Traffic Server can be used in reverse proxy mode to provide security for an origin server. If an origin server contains sensitive information that you want to keep secure inside your firewall, then you can use a Traffic Server outside the firewall as a reverse proxy for that origin server. When outside clients try to access the origin server, the requests instead go to Traffic Server. If the desired content is not sensitive, then it can be served from the cache. If the content is sensitive and not cacheable, then Traffic Server obtains the content from the origin server (the firewall allows only Traffic Server access to the origin server). The sensitive content resides on the origin server, safely inside the firewall.

HTTP Reverse Proxy¶

In reverse proxy mode, Traffic Server serves HTTP requests on behalf of a web server. The figure below illustrates how Traffic Server in reverse proxy mode serves an HTTP request from a client browser.

HTTP reverse proxy

The figure above demonstrates the following steps:

1. A client browser sends an HTTP request addressed to a host called www.host.com on port 80. Traffic Server receives the request because it is acting as the origin server (the origin server’s advertised hostname resolves to Traffic Server).
2. Traffic Server locates a map rule in the remap.config file and remaps the request to the specified origin server (realhost.com).
3. Traffic Server opens an HTTP connection to the origin server. (If the request is not able to be served from cache)
4. If the request is a cache hit and the content is fresh, then Traffic Server sends the requested object to the client from the cache. Otherwise, Traffic Server obtains the requested object from the origin server, sends the object to the client, and saves a copy in its cache.

To configure HTTP reverse proxy, you must perform the following tasks:

• Create mapping rules in the remap.config file (refer to Creating Mapping Rules for HTTP Requests).

  # remap.config
  map http://www.host.com http://realhost.com

• Enable the reverse proxy option (refer to Enabling HTTP Reverse Proxy).

In addition to the tasks above, you can also Setting Optional HTTP Reverse Proxy Options.

Redirecting HTTP Requests¶

You can configure Traffic Server to redirect HTTP requests without having to contact any origin servers. For example, if you redirect all requests for http://www.ultraseek.com to http://www.server1.com/products/portal/search/, then all HTTP requests for www.ultraseek.com go directly to www.server1.com/products/portal/search.

You can configure Traffic Server to perform permanent or temporary redirects. Permanent redirects notify the browser of the URL change (by returning the HTTP status code ``301``) so that the browser can update bookmarks. Temporary redirects notify the browser of the URL change for the current request only (by returning the HTTP status code ``307`` ).

To set redirect rules

1. For each redirect you want to set enter a mapping rule in the remap.config file
2. Run the command traffic_line -x to apply the configuration changes.

redirect http://www.server1.com http://www.server2.com

Configuration¶

In order to configure Apache Traffic Server as forward proxy you will have to edit records.config and set

• proxy.config.url_remap.remap_required to 0

If your proxy is serving as pure forward proxy, you will also want to set

• proxy.config.reverse_proxy.enabled to 0

Other configuration variables to consider:

• proxy.config.http.no_dns_just_forward_to_parent
• proxy.config.http.forward.proxy_auth_to_parent
• proxy.config.http.insert_squid_x_forwarded_for

Security Considerations¶

It’s important to note that once your Apache Traffic Server is configured as forward proxy it will indiscriminately accept proxy requests from anyone. That means, if it’s reachable on the internet, you have configured an Open Proxy. Most of the time, this is not what you want, so you’ll have to make sure it’s either only reachable within your NAT or is secured by firewall rules that permit only those clients to access it which you want to it to access.

Configuring Browsers Manually¶

To manually configure a browser to send HTTP requests to Traffic Server, clients must provide the following information:

• The fully-qualified hostname or IP address of the Traffic Server node
• The Traffic Server proxy server port (port 8080)

In addition, clients can specify not to use Traffic Server for certain sites - in such cases, requests to the listed sites go directly to the origin server. The procedures for manual configuration vary among browser versions; refer to specific browser documentation for complete proxy configuration instructions. You do not need to set any special configuration options on Traffic Server if you want to accept requests from manually-configured browsers.

Using a PAC File¶

A PAC file is a specialized JavaScript function definition that a browser calls to determine how requests are handled. Clients must specify (in their browser settings) the URL from which the PAC file is loaded. You can store a PAC file on Traffic Server (or on any server in your network) and then provide the URL for this file to your clients.

If you want to store a PAC file on the Traffic Server system, then you must perform the following configuration:

• Either copy an existing PAC file into the Traffic Server config directory or enter a script that defines the proxy server configuration settings in the proxy.pac file provided. The file is empty by default. A sample script is provided in Sample PAC File.
• Specify the port Traffic Server uses to serve the PAC file. The default port is 8083, see proxy.config.admin.autoconf_port.

function FindProxyForURL(url, host)
{
  if (isPlainHostName(host)) || (localHostOrDomainIs(host, ".company.com")) {
    return "DIRECT";
  }
  else
    return "PROXY myproxy.company.com:8080; DIRECT";
}

Understanding Cache Hierarchies¶

A cache hierarchy consists of cache levels that communicate with each other. Traffic Server supports several types of cache hierarchies. All cache hierarchies recognize the concept of parent and child. A parent cache is a cache higher up in the hierarchy, to which Traffic Server can forward requests. A child cache is a cache for which Traffic Server is a parent.

Traffic Server supports the following hierarchical caching options:

Parent Caching¶

If a Traffic Server node cannot find a requested object in its cache, then it searches a parent cache (which itself can search other caches) before finally retrieving the object from the origin server. You can configure a Traffic Server node to use one or more parent caches so that if one parent is unavailable, then another parent is availale to service requests. This is called Parent Failover. Traffic Server will support parent caching for HTTP and HTTPS requests.

Note: If you do not want all requests to go to the parent cache, then simply configure Traffic Server to route certain requests (such as requests containing specific URLs) directly to the origin server. Simply set parent proxy rules in parent.config

The figure below illustrates a simple cache hierarchy with a Traffic Server node configured to use a parent cache. In the following scenario, a client sends a request to a Traffic Server node that is a child in the cache hierarchy (because it’s configured to forward missed requests to a parent cache). The request is a cache miss, so Traffic Server then forwards the request to the parent cache, where it is a cache hit. The parent sends a copy of the content to the Traffic Server, where it is cached and then served to the client. Future requests for this content can now be served directly from the Traffic Server cache (until the data is stale or expired).

Parent caching

Note: If the request is a cache miss on the parent, then the parent retrieves the content from the origin server (or from another cache, depending on the parent’s configuration). The parent caches the content and then sends a copy to Traffic Server (its child), where it is cached and served to the client.

url_regex=politics prefix=/viewpoint go_direct=true

dest_host=host1 scheme=http parent="parent1;parent2" round-robin=strict

The Traffic Server Cache¶

The Traffic Server cache consists of a high-speed object database called the object store. The object store indexes objects according to URLs and associated headers. This enables Traffic Server to store, retrieve, and serve not only web pages, but also parts of web pages - which provides optimum bandwidth savings. Using sophisticated object management, the object store can cache alternate versions of the same object (versions may differ because of dissimilar language or encoding types). It can also efficiently store very small and very large documents, thereby minimizing wasted space. When the cache is full, Traffic Server removes stale data to ensure the most requested objects are kept readily available and fresh.

Traffic Server is designed to tolerate total disk failures on any of the cache disks. If the disk fails completely, then Traffic Server marks the entire disk as corrupt and continues using the remaining disks. An alarm is then created to indicate which disk failed. If all of the cache disks fail, then Traffic Server goes into proxy-only mode.

You can perform the following cache configuration tasks:

• Change the total amount of disk space allocated to the cache: refer to Changing Cache Capacity.
• Partition the cache by reserving cache disk space for specific protocols and origin servers/domains: refer to Partitioning the Cache.
• Delete all data in the cache: refer to Clearing the Cache.
• Override cache directives for a requested domain name, regex on a url, hostname or ip, with extra filters for time, port, method of the request (and more). ATS can be configured to never cache; always cache; ignore no-cache directives, etc. These are configured in cache.config.

The RAM Cache¶

Traffic Server maintains a small RAM cache of extremely popular objects. This RAM cache serves the most popular objects as quickly as possible and reduces load on disks, especially during temporary traffic peaks. You can configure the RAM cache size to suit your needs, as described in Changing the Size of the RAM Cache below.

The RAM cache supports two cache eviction algorithms, a regular LRU (Least Recently Used) and the more advanced CLFUS (Clocked Least Frequently Used by Size, which balances recentness, frequency and size to maximize hit rate – similar to a most frequently used algorithm). The default is to use CLFUS, and this is controlled via proxy.config.cache.ram_cache.algorithm.

Both the LRU and CLFUS RAM caches support a configuration to increase scan resistance. In a typical LRU, if you request all possible objects in sequence, you will effectively churn the cache on every request. The option proxy.config.cache.ram_cache.use_seen_filter can be set to add some resistance against this problem.

In addition, CLFUS also supports compressing in the RAM cache itself. This can be useful for content which is not compressed by itself (e.g. images). This should not be confused with Content-Encoding: gzip, this feature is only thereto save space internally in the RAM cache itself. As such, it is completely transparent to the User-Agent. The RAM cache compression is enabled with the option proxy.config.cache.ram_cache.compress. The default is 0, which means no compression. Other possible values are 1 for fastlz, 2 for libz and 3 for liblzma.

Changing the Size of the RAM Cache¶

Traffic Server provides a dedicated RAM cache for fast retrieval of popular small objects. The default RAM cache size is automatically calculated based on the number and size of the cache partitions you have configured. If you’ve partitioned your cache according to protocol and/or hosts, then the size of the RAM cache for each partition is proportional to the size of that partition.

You can increase the RAM cache size for better cache hit performance. However, if you increase the size of the RAM cache and observe a decrease in performance (such as increased latencies), then it’s possible that the operating system requires more memory for network resources. In such instances, you should return the RAM cache size to its previous value.

To change the RAM cache size:

1. Stop Traffic Server.
2. Set the variable proxy.config.cache.ram_cache.size to specify the size of the RAM cache. The default value of -1 means that the RAM cache is automatically sized at approximately 1MB per gigabyte of disk.
3. Restart Traffic Server. If you increase the RAM cache to a size of 1GB or more, then restart with the trafficserver command (refer to Start Traffic Server).

Changing Cache Capacity¶

You can increase or reduce the total amount of disk space allocated to the cache without clearing the content. To check the size of the cache (in bytes), enter the command traffic_line -r proxy.process.cache.bytes_total.

Partitioning the Cache¶

You can manage your cache space more efficiently and restrict disk usage by creating cache volumes with different sizes for specific protocols. You can further configure these volumes to store data from specific origin servers and/or domains. The volume configuration must be the same on all nodes in a cluster.

Configuring the Cache Object Size Limit¶

By default, Traffic Server allows objects of any size to be cached. You can change the default behavior and specify a size limit for objects in the cache via the steps below:

1. Set proxy.config.cache.max_doc_size to specify the maximum size allowed for objects in the cache in bytes. 0 (zero) if you do not want a size limit.
2. Run the command traffic_line -x to apply the configuration changes.

Clearing the Cache¶

When you clear the cache, you remove all data from the entire cache - including data in the host database. You should clear the cache before performing certain cache configuration tasks, such as partitioning. You cannot clear the cache when Traffic Server is running.

To clear the cache:

1. Stop Traffic Server (refer to Stopping Traffic Server)

2. Enter the following command to clear the cache:

   traffic_server -Cclear
   

   The clear command deletes all data in the object store and the host database. Traffic Server does not prompt you to confirm the deletion.

3. Restart Traffic Server (refer to Starting Traffic Server).

Removing an Object From the Cache¶

Traffic Server accepts the custom HTTP request method PURGE when removing a specific object from cache. If the object is found in the cache and is successfully removed, then Traffic Server responds with a 200 OK HTTP message; otherwise, a 404 File Not Found message is returned.

In the following example, Traffic Server is running on the domain example.com and you want to remove the image remove_me.jpg from cache. Because by default we do not permit PURGE requests from any other IP, we connect to the daemon via localhost:

$ curl -X PURGE -H 'Host: example.com' -v "http://localhost/remove_me.jpg"
* About to connect() to localhost port 80 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 80 (#0)

> PURGE /remove_me.jpg HTTP/1.1
> User-Agent: curl/7.19.7
> Host: example.com
> Accept: */*
>
< HTTP/1.1 200 Ok
< Date: Thu, 08 Jan 2010 20:32:07 GMT
< Connection: keep-alive

The next time Traffic Server receives a request for the removed object, it will contact the origin server to retrieve it (i.e., it has been purged from the Traffic Server cache).

Note: The procedure above only removes an object from a specific Traffic Server cache. Users may still see the old (removed) content if it was cached by intermediary caches or by the end-users’ web browser.

Inspecting the Cache¶

Traffic Server provides a Cache Inspector utility that enables you to view, delete, and invalidate URLs in the cache (HTTP only). The Cache Inspector utility is a powerful tool that’s capable of deleting all the objects in your cache; therefore, make sure that only authorized administrators are allowed to access this utility, see Controlling Client Access to the Proxy Cache and the @src_ip option in remap.config.

Traffic Server Monitoring Tools¶

Traffic Server provides the following tools to monitor system performance and analyze network traffic:

• Traffic Server can send email that’s triggered by alarms that signal any detected failure conditions; refer to Working with Traffic Manager Alarms.
• The Traffic Line command-line interface provides an alternative method of viewing Traffic Server performance and network traffic information; refer to Viewing Statistics from Traffic Line.
• The Traffic Shell command-line tool provides yet another alternative method of viewing Traffic Server performance and network traffic information; refer to Starting Traffic Shell.

Working with Traffic Manager Alarms¶

Traffic Server signals an alarm when it detects a problem. For example, the space allocated to event logs could be full or Traffic Server may not be able to write to a configuration file.

Viewing Statistics from Traffic Line¶

You can use the Traffic Line command-line interface to view statistics about Traffic Server performance and web traffic. In addition to viewing statistics, you can also configure, stop, and restart the Traffic Server system. For additional information, refer to Configure Traffic Server Using Traffic Line and Description. You can view specific information about a Traffic Server node or cluster by specifying the variable that corresponds to the statistic you want to see.

To view a statistic, enter the following command::

traffic_line -r variable

where variable is the variable representing the information you want to view. For a list of variables you can specify, refer to Traffic Line Variables.

For example, the following command displays the document hit rate for the Traffic Server node::

traffic_line -r proxy.node.cache_hit_ratio

If the Traffic Server bin directory is not in your path, then prepend the Traffic Line command with ./ (for example: traffic_line -r variable).

Configure Traffic Server Using Traffic Line¶

Traffic Line enables you to quickly and easily change your Traffic Server configuration via command-line interface. Alternatively, you can also use traffic_shell to configure Traffic Server.

traffic_line -r var

traffic_line -s var -v value

Configure Traffic Server Using Configuration Files¶

As an alternative to using Traffic Line or Traffic Shell, you can change Traffic Server configuration options by manually editing specific variables in the records.config file. After modifying the records.config file, Traffic Server must reread the configuration files: enter the Traffic Line command traffic_line -x. You may need to restart Traffic Server to apply some of the configuration changes.

The following is a sample portion of the records.config file:

Sample records.config file

In addition to the records.config file, Traffic Server provides other configuration files that are used to configure specific features. You can manually edit all configuration files as described in Configuration File Reference.

Understanding Traffic Server Clusters¶

A Traffic Server cluster consists of multiple Traffic Server nodes. The nodes in a cluster share configuration information and can form a single logical cache. Traffic Server detects the addition and deletion of nodes in the cluster automatically and can detect when a node is unavailable. Traffic Server uses its own protocol for clustering, which is multicast for node location and heartbeat, but unicast for all data exchange within the cluster. Traffic Server has two clustering modes:

• Management-only mode; refer to Management-Only Clustering below.
• Full-clustering mode; refer to Full Clustering

Management-Only Clustering¶

In management-only clustering mode, Traffic Server cluster nodes share configuration information. You can administer all the nodes at the same time. Traffic Server uses a multicast management protocol to provide a single system image of your Traffic Server cluster. Information about cluster membership, configuration, and exceptions is shared across all nodes, and the traffic_manager process automatically propagates configuration changes to all the nodes.

Full Clustering¶

In full-clustering mode, as well as sharing configuration information, a Traffic Server cluster distributes its cache across its nodes into a single, virtual object store, rather than replicating the cache node by node. Traffic Server can provide an enormous aggregate cache size and can maximize cache hit rate by storing objects only once across the entire cluster.

A fully clustered Traffic Server maps objects to specific nodes in the cluster. When a node receives a request, it checks to see if the request is a hit somewhere in the cluster. If the request is a hit on a different node, the node handling the request obtains the object from the hit node and serves it to the client. Traffic Server uses its own communication protocol to obtain an object from sibling cluster nodes.

If a node fails or is shut down and removed, Traffic Server removes references to the missing node on all nodes in the cluster.

Full clustering recommends a dedicated network interface for cluster communication to get better performance.

Enabling Clustering Mode¶

Before you put a node into a cluster, please make sure the following things are in order:

• You are using the same operation system on all nodes:
  • Using the same distribution, e.g.: RHEL 5.5
  • Have same kernel, e.g.: 2.6.18-194.17.1.el5
  • The same architecture, e.g.: x86_64
• You have the same version of Traffic Server installed
• The same hardware
• On the same switch or same VLAN.

Traffic Server does not apply the clustering mode change to all the nodes in the cluster. You must change the clustering mode on each node individually. You may following these instructions:

1. Setup the same cluster name, with proxy.config.proxy_name, e.g. MyCluster.
2. Set proxy.local.cluster.type to 1, to enable cluster mode. The following values of this configuration are valid

3. Setup a proxy.config.cluster.ethernet_interface, e.g.: eth0. This should be replaced by your real interface; we recommends a dedicated interface here. Refer to proxy.local.cluster.type for a full description.

4. Enable configuration changes:

   traffic_line -x
   

5. Restart traffic server:

   traffic_line -L
   

   The traffic_server and traffic_manager processes will need to restart after the change of proxy.local.cluster.type and proxy.config.cluster.ethernet_interface have taken place.

Traffic Server will join the cluster in about 10 seconds, and you can run traffic_line -r proxy.process.cluster.nodes to check the hosts in the cluster, or check out the cluster.config in the configuration directory. This configuration is generated by the system, and should not be edited. It contains a list of the machines that are currently members of the cluster, for example:

# 3
127.1.2.3:80
127.1.2.4:80
127.1.2.5:80

After successfully joining of a cluster, all changes of global configurations on any node, will take effect on all nodes. This means you can make changes on any cluster node member, and they are automatically distributed to all members.

Deleting Nodes from a Cluster¶

To delete a node from the Traffic Server cluster, just roll back proxy.local.cluster.type to the default value 3 and reload.

Performance tweak for busy Cluster¶

Starting from v3.2.0, Apache Traffic Server can handle multiple internal cluster connections, and we can tweak the number of Cluster threads. Each of the thread will keep one connection to all of peering cluster machines.

traffic_line -s proxy.config.cluster.threads -v 10

Controlling Client Access to the Proxy Cache¶

You can configure Traffic Server to allow only certain clients to use the proxy cache by editing a configuration file.

1. Add a line in ip_allow.config for each IP address or range of IP addresses allowed to access Traffic Server.
2. Run the command traffic_line -x to apply the configuration changes.

Configuring DNS Server Selection (Split DNS)¶

The Split DNS option enables you to configure Traffic Server to use multiple DNS servers, as dictated by your security requirements. For example, you might configure Traffic Server to use one set of DNS servers to resolve hostnames on your internal network, while allowing DNS servers outside the firewall to resolve hosts on the Internet. This maintains the security of your intranet, while continuing to provide direct access to sites outside your organization.

To configure Split DNS, you must do the following:

• Specify the rules for performing DNS server selection based on the destination domain, the destination host, or a URL regular expression.
• Enable the Split DNS option.

To do this, we

1. Add rules to splitdns.config.
2. In records.config set the variable proxy.config.dns.splitDNS.enabled to 1 to enable split DNS.
3. Run the command traffic_line -x to apply the configuration changes.

Using SSL Termination¶

The Traffic Server SSL termination option enables you to secure connections in reverse proxy mode between a client and a Traffic Server and/or Traffic Server and an origin server.

The following sections describe how to enable and configure the SSL termination option.

• Enable and configure SSL termination for client/Traffic Server connections: Client and Traffic Server Connections
• Enable and configure SSL termination for Traffic Server/origin server connections: Traffic Server and Origin Server Connections
• Enable and configure SSL termination for both client/Traffic Server and Traffic Server/origin server connections: Client and Traffic Server Connections Traffic Server and Origin Server Connections, respectively.

Client and Traffic Server Connections¶

The figure below illustrates communication between a client and Traffic Server (and between Traffic Server and an origin server) when the SSL termination option is enabled & configured for client/Traffic Server connections only.

Client and Traffic Server communication using SSL termination

The figure above depicts the following:

1. The client sends an HTTPS request for content. Traffic Server receives the request and performs the SSL ‘handshake’ to authenticate the client (depending on the authentication options configured) and determine the encryption method that will be used. If the client is allowed access, then Traffic Server checks its cache for the requested content.
2. If the request is a cache hit and the content is fresh, then Traffic Server encrypts the content and sends it to the client. The client decrypts the content (using the method determined during the handshake) and displays it.
3. If the request is a cache miss or cached content is stale, then Traffic Server communicates with the origin server via HTTP and obtains a plain text version of the content. Traffic Server saves the plain text version of the content in its cache, encrypts the content, and sends it to the client. The client decrypts and displays the content.

To configure Traffic Server to use the SSL termination option for client/Traffic Server connections, you must do the following:

• Obtain and install an SSL server certificate from a recognized certificate authority. The SSL server certificate contains information that enables the client to authenticate Traffic Server and exchange encryption keys.
• Configure SSL termination options:
  • Set the port number used for SSL communication using proxy.config.http.server_ports.
  • Edit ssl_multicert.config to specify the filename and location of the SSL certificates and private keys.
  • (Optional) Configure the use of client certificates: Client certificates are located on the client. If you configure Traffic Server to require client certificates, then Traffic Server verifies the client certificate during the SSL handshake that authenticates the client. If you configure Traffic Server to not require client certificates, then access to Traffic Server is managed through other Traffic Server options that have been set (such as rules in ip_allow.config).
  • (Optional) Configure the use of Certification Authorities (CAs). CAs add security by verifying the identity of the person requesting a certificate.

In order to accomplish this, we

1. Edit the following variables in the SSL Termination section of records.config
   • proxy.config.http.server_ports
   • proxy.config.ssl.client.certification_level
   • proxy.config.ssl.server.cert.path
   • proxy.config.ssl.server.private_key.path
   • proxy.config.ssl.CA.cert.path
2. Run the command traffic_line -L to restart Traffic Server on the local node or traffic_line -M to restart Traffic Server on all the nodes in a cluster.

Traffic Server and Origin Server Connections¶

The figure below illustrates communication between Traffic Server and an origin server when the SSL termination option is enabled for Traffic Server/origin server connections.

Traffic Server and origin server communication using SSL termination

The figure above depicts the following:

Step 1: If a client request is a cache miss or is stale, then Traffic Server sends an HTTPS request for the content to the origin server. The origin server receives the request and performs the SSL handshake to authenticate Traffic Server and determine the encryption method to be used.

Step 2: If Traffic Server is allowed access, then the origin server encrypts the content and sends it to Traffic Server, where it is decrypted (using the method determined during the handshake). A plain text version of the content is saved in the cache.

Step 3: If SSL termination is enabled for client /Traffic Server connections, then Traffic Server re-encrypts the content and sends it to the client via HTTPS, where it is decrypted and displayed. If SSL termination is not enabled for client/Traffic Server connections, then Traffic Server sends the plain text version of the content to the client via HTTP.

To configure Traffic Server to use the SSL termination option for Traffic Server and origin server connections, you must do the following:

• Obtain and install an SSL client certificate from a recognized certificate authority. The SSL client certificate contains information that allows the origin server to authenticate Traffic Server (the client certificate is optional).
• Configure SSL termination options:
• Enable the SSL termination option.
  • Set the port number used for SSL communication.
  • Specify the filename and location of the SSL client certificate (if you choose to use a client certificate).
  • Specify the filename and location of the Traffic Server private key (if the private key is not located in the client certificate file). Traffic Server uses its private key during the SSL handshake to decrypt the session encryption keys. The private key must be stored and protected against theft.
  • Configure the use of CAs. CAs allow the Traffic Server that’s acting as a client to verify the identity of the server with which it is communicating, thereby enabling exchange of encryption keys.

In order to accomplish this, we:

1. Edit the following variables in the SSL Termination section of records.config:
   • proxy.config.ssl.auth.enabled
   • proxy.config.http.server_ports
   • proxy.config.ssl.client.verify.server
   • proxy.config.ssl.client.cert.filename
   • proxy.config.ssl.client.cert.path
   • proxy.config.ssl.client.private_key.filename
   • proxy.config.ssl.client.private_key.path
   • proxy.config.ssl.client.CA.cert.filename
   • proxy.config.ssl.client.CA.cert.path
2. Run the command traffic_line -L to restart Traffic Server on the local node or traffic_line -M to restart Traffic Server on all the nodes in a cluster.

Understanding Traffic Server Log Files¶

Traffic Server records information about every transaction (or request) it processes and every error it detects in log files. Traffic Server keeps three types of log files:

• Error log files record information about why a particular transaction was in error.

• Event log files (also called access log files) record information about the state of each transaction Traffic Server processes.

• System log files record system information, including messages about the state of Traffic Server and errors/warnings it produces. This kind of information might include a note that event log files were rolled, a warning that cluster communication timed out, or an error indicating that Traffic Server was restarted.

  All system information messages are logged with the system-wide logging facility syslog under the daemon facility. The syslog.conf(5) configuration file (stored in the /etc directory) specifies where these messages are logged. A typical location is /var/log/messages (Linux).

  The syslog(8) process works on a system-wide basis, so it serves as the single repository for messages from all Traffic Server processes (including traffic_server, traffic_manager, and traffic_cop).

  System information logs observe a static format. Each log entry in the log contains information about the date and time the error was logged, the hostname of the Traffic Server that reported the error, and a description of the error or warning.

  Refer to Error Messages for a list of the messages logged by Traffic Server.

By default, Traffic Server creates both error and event log files and records system information in system log files. You can disable event logging and/or error logging by setting the configuration variable proxy.config.log.logging_enabled (in the records.config file) to one of the following values:

• 0 to disable both event and error logging
• 1 to enable error logging only
• 2 to enable transaction logging only
• 3 to enable both transaction and error logging

Understanding Event Log Files¶

Event log files record information about every request that Traffic Server processes. By analyzing the log files, you can determine how many people use the Traffic Server cache, how much information each person requested, what pages are most popular, and so on. Traffic Server supports several standard log file formats, such as Squid and Netscape, as well as user-defined custom formats. You can analyze the standard format log files with off-the-shelf analysis packages. To help with log file analysis, you can separate log files so they contain information specific to protocol or hosts. You can also configure Traffic Server to roll log files automatically at specific intervals during the day or when they reach a certain size.

The following sections describe the Traffic Server logging system features and discuss how to:

• Manage your event log files

  You can choose a central location for storing log files, set how much disk space to use for log files, and set how and when to roll log files. Refer to Managing Event Log Files.

• Choose different event log file formats

  You can choose which standard log file formats you want to use for traffic analysis, such as Squid or Netscape. Alternatively, you can use the Traffic Server custom format, which is XML-based and enables you to institute more control over the type of information recorded in log files. Refer to Choosing Event Log File Formats.

• Roll event log files automatically

  Configure Traffic Server to roll event log files at specific intervals during the day or when they reach a certain size; this enables you to identify and manipulate log files that are no longer active. Refer to Rolling Event Log Files.

• Separate log files according to protocols and hosts

  Configure Traffic Server to create separate log files for different protocols. You can also configure Traffic Server to generate separate log files for requests served by different hosts. Refer to Splitting Event Log Files.

• Collate log files from different Traffic Server nodes

  Designate one or more nodes on the network to serve as log collation servers. These servers, which might be standalone or part of Traffic Server, enable you to keep all logged information in well-defined locations. Refer to Collating Event Log Files.

• View statistics about the logging system

  Traffic Server provides statistics about the logging system; you can access these statistics via Traffic Line. Refer to Viewing Logging Statistics.

• Interpret log file entries for the log file formats

  Refer to Example Event Log File Entries.

<LogFormat>
  <Name = "summary"/>
  <Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>"/>
  <Interval = "10"/>
</LogFormat>

<Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)> : %<cqu>"/>

%Y%M%D.%Hh%Mm%Ss-%Y%M%D.%Hh%Mm%Ss

squid.log.mymachine.20110912.12h00m00s-20000913.12h00m00s.old

squid.log.mymachine.20110912.12h00m00s-19980912.03h00m00s.old

squid.log.mymachine.20110912.02h59m47s-19980912.06h00m00s.old

Collating Event Log Files¶

You can use the Traffic Server log file collation feature to collect all logged information in one place. Log collation enables you to analyze a set of Traffic Server clustered nodes as a whole (rather than as individual nodes) and to use a large disk that might only be located on one of the nodes in the cluster. Traffic Server collates log files by using one or more nodes as log collation servers and all remaining nodes as log collation clients. When a Traffic Server node generates a buffer of event log entries, it first determines if it is the collation server or a collation client. The collation server node writes all log buffers to its local disk, just as it would if log collation was not enabled. Log collation servers can be standalone or they can be part of a node running Traffic Server.

The collation client nodes prepare their log buffers for transfer across the network and send the buffers to the log collation server. When the log collation server receives a log buffer from a client, it writes it to its own log file as if it was generated locally. For a visual representation of this, see the figure below.

Log collation

If log clients cannot contact their log collation server, then they write their log buffers to their local disks, into orphan log files. Orphan log files require manual collation.

注解

Log collation can have an impact on network performance. Because all nodes are forwarding their log data buffers to the single collation server, a bottleneck can occur. In addition, collated log files contain timestamp information for each entry, but entries in the files do not appear in strict chronological order. You may want to sort collated log files before doing analysis.

To configure Traffic Server to collate event log files, you must perform the following tasks:

• Either Configure Traffic Server Node to Be a Collation Server or install & configure a Standalone Collator
• Configure Traffic Server Nodes to Be a Collation Clients.
• Add an attribute to the LogObject specification in the logs_xml.config file if you are using custom log file formats; refer to Collating Custom Event Log Files.

Configuring Traffic Server to Be a Collation Server¶

To configure a Traffic Server node to be a collation server, simply edit a configuration file via the steps below.

1. In the records.config file, edit the following variables
   • proxy.local.log.collation_mode (1 for server mode)
   • proxy.config.log.collation_port
   • proxy.config.log.collation_secret
2. Run the command traffic_line -x to apply the configuration changes.

注解

If you modify the collation_port or secret after connections between the collation server and collation clients have been established, then you must restart Traffic Server.

Using a Standalone Collator¶

If you do not want the log collation server to be a Traffic Server node, then you can install and configure a standalone collator (SAC) that will dedicate more of its power to collecting, processing, and writing log files.

To install and configure a standalone collator:

1. Configure your Traffic Server nodes as log collation clients; refer to Configuring Traffic Server to Be a Collation Client.

2. Copy the traffic_sac binary from the Traffic Server bin directory and

3. Copy the libtsutil.so libraries from the Traffic Server lib directory to the machine serving as the standalone collator.

4. Create a directory called config in the directory that contains the traffic_sac binary.

5. Create a directory called internal in the config directory you created in Step 4 (above). This directory is used internally by the standalone collator to store lock files.

6. Copy the records.config file from a Traffic Server node configured to be a log collation client to the config directory you created in Step 4 on the standalone collator. The records.config file contains the log collation secret and the port you specified when configuring Traffic Server nodes to be collation clients. The collation port and secret must be the same for all collation clients and servers.

7. In the records.config file, edit the following variable

   • proxy.config.log.logfile_dir

8. Enter the following command:

   traffic_sac -c config

Configuring Traffic Server to Be a Collation Client¶

To configure a Traffic Server node to be a collation client, follow the steps below. If you modify the collation_port or secret after connections between the collation clients and the collation server have been established, then you must restart Traffic Server.

1. In the records.config file, edit the following variables:
   • proxy.local.log.collation_mode: 2 to configure this node as log collation client and send standard formatted log entries to the collation server. For XML-based formatted log entries, see logs_xml.config file; refer to Using the Custom Format.
   • proxy.config.log.collation_host
   • proxy.config.log.collation_port
   • proxy.config.log.collation_secret
   • proxy.config.log.collation_host_tagged
   • proxy.config.log.max_space_mb_for_orphan_logs
2. Run the command traffic_line -x to apply the configuration changes.

Collating Custom Event Log Files¶

If you use custom event log files, then you must edit the logs_xml.config file (in addition to configuring a collation server and collation clients).

To collate custom event log files

1. On each collation client, edit the :file:`logs_xml.config

2. Add the CollationHost attribute to the LogObject specification:

   <LogObject>
     <Format = "squid"/>
     <Filename = "squid"/>
     <CollationHosts="ipaddress:port"/>
   </LogObject>

   where ipaddress is the hostname or IP address of the collation server to which all log entries (for this object) are forwarded, and port is the port number for communication between the collation server and collation clients.

3. Run the command traffic_line -L to restart Traffic Server on the local node or traffic_line -M to restart Traffic Server on all the nodes in a cluster.

Viewing Logging Statistics¶

Traffic Server generates logging statistics that enable you to see the following information:

• How many log files (formats) are currently being written.
• The current amount of space used by the logging directory, which contains all event and error logs.
• The number of access events written to log files since Traffic Server installation. This counter represents one entry in one file; if multiple formats are being written, then a single event creates multiple event log entries.
• The number of access events skipped (because they were filtered) since Traffic Server installation.
• The number of access events written to the event error log since Traffic Server installation.

You can retrieve the statistics via the Traffic Line command-line interface; refer to Monitoring Traffic.

Viewing Log Files¶

You can view the system, event, and error log files Traffic Server creates. You can also delete a log file or copy it to your local system if you have the correct user permissions. Traffic Server displays only one MB of information in the log file. If the log file you select to view is bigger than 1MB, then Traffic Server truncates the file and displays a warning message indicating that the file is too big.

Online Event Log XML Builder¶

If you need any assistance building your event log, you can try out our online log builder. This is a work in progress, so any comments, critique or suggestions are most welcome.

Example Event Log File Entries¶

This section shows an example log file entry in each of the standard log formats supported by Traffic Server: Squid, Netscape Common, Netscape Extended, and Netscape Extended-2.

Squid Format¶

The following figure shows a sample log entry in a squid.log file.

Sample log entry in squid.log

Field	Symbol	Description
1	cqtq	The client request timestamp in Squid format; the time of the client request in seconds since January 1, 1970 UTC (with millisecond resolution).
2	ttms	The time Traffic Server spent processing the client request; the number of milliseconds between the time the client established the connection with Traffic Server and the time Traffic Server sent the last byte of the response back to the client.
3	chi	The IP address of the client’s host machine.
4	crc/pssc	The cache result code; how the cache responded to the request: HIT, MISS, and so on. Cache result codes are described here. The proxy response status code (the HTTP response status code from Traffic Server to client).
5	psql	The length of the Traffic Server response to the client in bytes, including headers and content.
6	cqhm	The client request method: GET, POST, and so on.
7	cauc	The client request canonical URL; blanks and other characters that might not be parsed by log analysis tools are replaced by escape sequences. The escape sequence is a percentage sign followed by the ASCII code number of the replaced character in hex.
8	caun	The username of the authenticated client. A hyphen (-) means that no authentication was required.
9	phr/pqsn	The proxy hierarchy route; the route Traffic Server used to retrieve the object.
10	psct	The proxy response content type; the object content type taken from the Traffic Server response header.

Squid log in XML¶

This is the equivalent XML configuration for the log above:

<LogFormat>
  <Name = "squid"/>
  <Format = "%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc>
             %<caun> %<phr>/%<pqsn> %<psct>"/>
</LogFormat>

Netscape Common¶

Sample log entry in common.log

Field	Symbol	Description
1	chi	The IP address of the client’s host machine.
2	–	This hyphen (-) is always present in Netscape log entries.
3	caun	The authenticated client username. A hyphen (-) means no authentication was required.
4	cqtd	The date and time of the client request, enclosed in brackets.
5	cqtx	The request line, enclosed in quotes.
6	pssc	The proxy response status code (HTTP reply code).
7	pscl	The length of the Traffic Server response to the client in bytes.

Netscape Common in XML¶

This is the equivalent XML configuration for the log above:

<LogFormat>
  <Name = "common"/>
  <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>"/>
</LogFormat>

Netscape Extended¶

Sample log entry in extended.log

In addition to field 1-7 from the Netscape Common log format above, the Extended format also adds the following fields:

Field	Symbol	Description
8	sssc	The origin server response status code.
9	sshl	The server response transfer length; the body length in the origin server response to Traffic Server, in bytes.
10	cqbl	The client request transfer length; the body length in the client request to Traffic Server, in bytes.
11	pqbl	The proxy request transfer length; the body length in the Traffic Server request to the origin server.
12	cqhl	The client request header length; the header length in the client request to Traffic Server.
13	pshl	The proxy response header length; the header length in the Traffic Server response to the client.
14	pqhl	The proxy request header length; the header length in Traffic Server request to the origin server.
15	sshl	The server response header length; the header length in the origin server response to Traffic Server.
16	tts	The time Traffic Server spent processing the client request; the number of seconds between the time that the client established the connection with Traffic Server and the time that Traffic Server sent the last byte of the response back to the client.

Netscape Extended in XML¶

This is the equivalent XML configuration for the log above:

<LogFormat>
  <Name = "extended"/>
  <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>
     %<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts>"/>
</LogFormat>

Netscape Extended2¶

Sample log entry in extended2.log

In addition to field 1-16 from the log formats above, the Extended2 format also adds the following fields:

Field	Symbol	Description
17	phr	The proxy hierarchy route; the route Traffic Server used to retrieve the object.
18	cfsc	The client finish status code: FIN if the client request completed successfully or INTR if the client request was interrupted.
19	pfsc	The proxy finish status code: FIN if the Traffic Server request to the origin server completed successfully or INTR if the request was interrupted.
20	crc	The cache result code; how the Traffic Server cache responded to the request: HIT, MISS, and so on. Cache result codes are described here.

Netscape Extended2 in XML¶

This is the equivalent XML configuration for the log above:

<LogFormat>
  <Name = "extended2"/>
  <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>
             %<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts> %<phr> %<cfsc> %<pfsc> %<crc>"/>
</LogFormat>

Squid- and Netscape-format: Cache Result Codes¶

The following table describes the cache result codes in Squid and Netscape log files.

TCP_HIT

A valid copy of the requested object was in the cache and Traffic Server sent the object to the client.

TCP_MISS

The requested object was not in cache, so Traffic Server retrieved the object from the origin server (or a parent proxy) and sent it to the client.

TCP_REFRESH_HIT

The object was in the cache, but it was stale. Traffic Server made an if-modified-since request to the origin server and the origin server sent a 304 not-modified response. Traffic Server sent the cached object to the client.

TCP_REF_FAIL_HIT

The object was in the cache but was stale. Traffic Server made an if-modified-since request to the origin server but the server did not respond. Traffic Server sent the cached object to the client.

TCP_REFRESH_MISS

The object was in the cache but was stale. Traffic Server made an if-modified-since request to the origin server and the server returned a new object. Traffic Server served the new object to the client.

TCP_CLIENT_REFRESH

The client issued a request with a no-cache header. Traffic Server obtained the requested object from the origin server and sent a copy to the client. Traffic Server deleted the previous copy of the object from cache.

TCP_IMS_HIT

The client issued an if-modified-since request and the object was in cache & fresher than the IMS date, or an if-modified-since request to the origin server revealed the cached object was fresh. Traffic Server served the cached object to the client.

TCP_IMS_MISS

The client issued an if-modified-since request, and the object was either not in cache or was stale in cache. Traffic Server sent an if-modified-since request to the origin server and received the new object. Traffic Server sent the updated object to the client.

TCP_SWAPFAIL

The object was in the cache but could not be accessed. The client did not receive the object.

ERR_CLIENT_ABORT

The client disconnected before the complete object was sent.

ERR_CONNECT_FAIL

Traffic Server could not reach the origin server.

ERR_DNS_FAIL

The Domain Name Server (DNS) could not resolve the origin server name, or no DNS could be reached.

ERR_INVALID_REQ

The client HTTP request was invalid. (Traffic Server forwards requests with unknown methods to the origin server.)

ERR_READ_TIMEOUT

The origin server did not respond to Traffic Server’s request within the timeout interval.

ERR_PROXY_DENIED

Client service was denied.

ERR_UNKNOWN

The client connected, but subsequently disconnected without sending a request.

Custom Logging Fields¶

The following list describes Traffic Server custom logging fields.

{HTTP header field name}cqh

Logs the information in the requested field of the client request HTTP header. For example, %<{Accept-Language}cqh> logs the Accept-Language: field in client request headers.

{HTTP header field name}pqh

Logs the information in the requested field of the proxy request HTTP header. For example, %<{Authorization}pqh> logs the Authorization: field in proxy request headers.

{HTTP header field name}psh

Logs the information in the requested field of the proxy response HTTP header. For example, %<{Retry-After}psh> logs the Retry-After: field in proxy response headers.

{HTTP header field name}ssh

Logs the information in the requested field of the server response HTTP header. For example, %<{Age}ssh> logs the Age: field in server response headers.

caun

The client authenticated username; result of the RFC931/ident lookup of the client username.

cfsc

The client finish status code; specifies whether the client request to Traffic Server was successfully completed (FIN) or interrupted (INTR).

chi

The IP address of the client’s host machine.

chih

The IP address of the client’s host machine in hexadecimal.

chp

The port number of the client’s host machine.

cps

Client Protocol Stack, the output would be the conjunction of protocol names in the stack spliced with ‘+’, such as “TLS+SPDY”.

cqbl

The client request transfer length; the body length in the client request to Traffic Server (in bytes).

cqhl

The client request header length; the header length in the client request to Traffic Server.

cqhm

The HTTP method in the client request to Traffic Server: GET, POST, and so on (subset of cqtx).

cqhv

The client request HTTP version.

cqtd

The client request timestamp. Specifies the date of the client request in the format yyyy-mm-dd, where yyyy is the 4-digit year, mm is the 2-digit month, and dd is the 2-digit day.

cqtn

The client request timestamp; date and time of the client’s request (in the Netscape timestamp format).

cqtq

The client request timestamp, with millisecond resolution.

cqts

The client-request timestamp in Squid format; the time of the client request since January 1, 1970 UTC. Time is expressed in seconds, with millisecond resolution.

cqtt

The client request timestamp. The time of the client request in the format hh:mm:ss, where hh is the two-digit hour in 24-hour format, mm is the two-digit minutes value, and ss is the 2-digit seconds value (for example, 16:01:19).

cqtx

The full HTTP client request text, minus headers; for example,

GET http://www.company.com HTTP/1.0

In reverse proxy mode, Traffic Server logs the rewritten/mapped URL (according to the rules in the remap.config file), _not_ the pristine/unmapped URL.

cqu

The universal resource identifier (URI) of the request from client to Traffic Server (subset of cqtx ).

In reverse proxy mode, Traffic Server logs the rewritten/mapped URL (according to the rules in the remap.config file), _not_ the pristine/unmapped URL.

cquc

The client request canonical URL. This differs from cqu in that blanks (and other characters that might not be parsed by log analysis tools) are replaced by escape sequences. The escape sequence is a percentage sign followed by the ASCII code number in hex.

See cquuc.

cqup

The client request URL path; specifies the argument portion of the URL (everything after the host). For example, if the URL is http://www.company.com/images/x.gif, then this field displays /images/x.gif

See cquup.

cqus

The client request URL scheme.

cquuc

The client request unmapped URL canonical. This field records a URL before it is remapped (reverse proxy mode).

cquup

The client request unmapped URL path. This field records a URL path before it is remapped (reverse proxy mode).

cquuh

The client request unmapped URL host. This field records a URL’s host before it is remapped (reverse proxy mode).

crat

The Retry-After time in seconds, if specified by the origin server.

crc

The cache result code; specifies how the cache responded to the request (HIT, MISS, and so on).

csscl

The cached response length (in bytes) from origin server to Traffic Server.

csshl

The cached header length in the origin server response to Traffic Server (in bytes).

csshv

The cached server response HTTP version (1.0, 1.1, etc.).

csssc

The cached HTTP response status code from origin server to Traffic Server.

cwr

The cache write result (-, WL_MISS, INTR`, ERR or FIN)

cwtr

The cache write transform result

fsiz

The size of the file (n bytes) as seen by the origin server.

pfsc

The proxy finish status code; specifies whether the Traffic Server request to the origin server was successfully completed (FIN), interrupted (INTR) or timed out (TIMEOUT).

phn

The hostname of the Traffic Server that generated the log entry in collated log files.

phi

The IP of the Traffic Server that generated the log entry in collated log files.

phr

The proxy hierarchy route; the route Traffic Server used to retrieve the object.

pqbl

The proxy request transfer length; the body length in Traffic Server’s request to the origin server.

pqhl

The proxy request header length; the header length in Traffic Server’s request to the origin server.

pqsi

The proxy request server IP address (0 on cache hits and parent-ip for requests to parent proxies).

pqsn

The proxy request server name; the name of the server that fulfilled the request.

pscl

The length of the Traffic Server response to the client (in bytes).

psct

The content type of the document from server response header: (for example, img/gif ).

pshl

The header length in Traffic Server’s response to the client.

psql

The proxy response transfer length in Squid format (includes header and content length).

pssc

The HTTP response status code from Traffic Server to the client.

shi

The IP address resolved from the DNS name lookup of the host in the request. For hosts with multiple IP addresses, this field records the IP address resolved from that particular DNS lookup.

This can be misleading for cached documents. For example: if the first request was a cache miss and came from ``IP1`` for server ``S`` and the second request for server ``S`` resolved to ``IP2`` but came from the cache, then the log entry for the second request will show ``IP2``.

shn

The hostname of the origin server.

sscl

The response length (in bytes) from origin server to Traffic Server.

sshl

The header length in the origin server response to Traffic Server (in bytes).

sshv

The server response HTTP version (1.0, 1.1, etc.).

sssc

The HTTP response status code from origin server to Traffic Server.

ttms

The time Traffic Server spends processing the client request; the number of milliseconds between the time the client establishes the connection with Traffic Server and the time Traffic Server sends the last byte of the response back to the client.

ttmsh

Same as ttms but in hexadecimal.

ttmsf

The time Traffic Server spends processing the client request as a fractional number of seconds. Time is specified in millisecond resolution; however, instead of formatting the output as an integer (as with ttms), the display is formatted as a floating-point number representing a fractional number of seconds.

For example: if the time is 1500 milliseconds, then this field displays 1.5 while the ttms field displays 1500 and the tts field displays 1.

tts

The time Traffic Server spends processing the client request; the number of seconds between the time at which the client establishes the connection with Traffic Server and the time at which Traffic Server sends the last byte of the response back to the client.

Logging Format Cross-Reference¶

The following sections illustrate the correspondence between Traffic Server logging fields and standard logging fields for the Squid and Netscape formats.

Traffic Server Error Messages¶

The following table lists messages that can appear in system log files. This list is not exhaustive; it simply describes common warning messages that can occur and which might require your attention.

Traffic Server Process Fatal¶

Accept port is not between 1 and 65535. Please check configuration

The port specified in the records.config file that accepts incoming HTTP requests is not valid.

Self loop is detected in parent proxy configuration

The name and port of the parent proxy match that of Traffic Server. This creates a loop when Traffic Server attempts to send the request to the parent proxy.

Traffic Server Alarm Messages¶

[Rollback::Rollback] Config file is read-only: <filename>

Go to the Traffic Server config directory and check the indicated file permissions; change if necessary.

[Rollback::Rollback] Unable to read or write config file <filename>

Go to the Traffic Server config directory and make sure the indicated file exists. Check permissions and modify if necessary.

[Traffic Manager] Configuration File Update Failed: <error number>

Go to the Traffic Server config directory and check the indicated file permissions; change if necessary.

[Traffic Manager] Mgmt <==>Proxy conn. closed

An informational message to inform you that the traffic_server process is down.

Access logging suspended - configured space allocation exhausted.

The space allocated to the event log files is full; you must either increase the space or delete some log files so that access logging to continue. To prevent this error, consider rolling log files more frequently and enabling the autodelete feature.

Access logging suspended - no more space on the logging partition.

The entire partition containing the event logs is full; you must delete or move some log files to enable access logging to continue. To prevent this error, consider rolling log files more frequently and enabling the autodelete feature.

Created zero length place holder for config file <filename>

Go to the Traffic Server config directory and check the indicated file. If it is indeed zero in length, then use a backup copy of the configuration file.

Traffic Server could not open logfile <filename>

Check permissions for the indicated file and the logging directory.

Traffic Server failed to parse line <line number> of the logging config file <filename>

Check your custom log configuration file; there could be syntax errors. Refer to Custom Logging Fields for correct custom log format fields.

vip_config binary is not setuid root, manager will be unable to enable virtual ip addresses

The traffic_manager process is not able to set virtual IP addresses. You must setuid root for the vip_config file in the Traffic Server bin directory.

HTML Messages Sent to Clients¶

Traffic Server returns detailed error messages to browser clients when there are problems with the HTTP transactions requested by the browser. These Traffic Server response messages correspond to standard HTTP response codes, but provide more information. A list of the more frequently-encountered HTTP response codes is provided in Standard HTTP Response Messages. You can customize the Traffic Server response messages (typically in proxy/config/body_factory/default/, but set by proxy.config.body_factory.template_sets_dir).

The following table lists the hard-coded Traffic Server HTTP messages, with corresponding HTTP response codes and customizable files.

Access Denied

403 You are not allowed to access the document at location URL. access#denied

Cache Read Error

500 Error reading from cache; please retry request. cache#read_error

Connection Timed Out

504 Too much time has elapsed since the server has sent data. timeout#inactivity

Content Length Required

400 Could not process this request because Content-Length was not specified. request#no_content_length

Cycle Detected

400 Your request is prohibited because it would cause an HTTP proxy cycle. request#cycle_detected

Forbidden

403 <port number> is not an allowed port for SSL connections (you have made a request for a secure SSL connection to a forbidden port number). access#ssl_forbidden

Host Header Required

400 An attempt was made to transparently proxy your request, but this attempt failed because your browser did not send an HTTP Host header. Manually configure your browser to use http://<proxy name>:<proxy port> as the HTTP proxy. Alternatively, end users can upgrade to a browser that supports the HTTP Host header field. interception#no_host

Host Header Required

400 Because your browser did not send a Host HTTP header field, the virtual host being requested could not be determined. To access the website correctly, you must upgrade to a browser that supports the HTTP Host header field. request#no_host

HTTP Version Not Supported

505 The origin server <server name> is using an unsupported version of the HTTP protocol. response#bad_version

Invalid Content Length

400 Could not process this request because the specified Content-Length was invalid (less than 0).. request#invalid_content_length

Invalid HTTP Request

400 Could not process this <client request> HTTP method request for URL. request#syntax_error

Invalid HTTP Response

502 The host <server name> did not return the document URL correctly. response#bad_response

Malformed Server Response

502 The host <server name> did not return the document URL correctly. response#bad_response

Malformed Server Response Status

502 The host <server name> did not return the document URL correctly. response#bad_response

Maximum Transaction Time exceeded

504 Too much time has elapsed while transmitting document URL. timeout#activity

No Response Header From Server

502 The host <server name> did not return the document URL correctly. response#bad_response

Not Cached

504 This document was not available in the cache, and you (the client) only accept cached copies. cache#not_in_cache

Not Found on Accelerator

404 The request for URL on host <server name> was not found. Check the location and try again. urlrouting#no_mapping

NULL

502 The host <hostname> did not return the document URL correctly. response#bad_response

Proxy Authentication Required

407 Please log in with username and password. access#proxy_auth_required

Server Hangup

502 The server <hostname> closed the connection before the transaction was completed. connect#hangup

Temporarily Moved

302 The document you requested, URL, has moved to a new location. The new location is <new URL>. redirect#moved_temporarily

Transcoding Not Available

406 Unable to provide the document URL in the format requested by your browser. transcoding#unsupported

Tunnel Connection Failed

502 Could not connect to the server <hostname>. connect#failed_connect

Unknown Error

502 The host <hostname> did not return the document URL correctly. response#bad_response

Unknown Host

500 Unable to locate the server named <hostname>; the server does not have a DNS entry. Perhaps there is a misspelling in the server name or the server no longer exists; double-check the name and try again. connect#dns_failed

Unsupported URL Scheme

400 Cannot perform your request for the document URL because the protocol scheme is unknown. request#scheme_unsupported

Before you start¶

There is no single option to that will guarantee maximum performance of Apache Traffic Server in every use-case. There are however numerous options that help tune its performance under different loads and in its - often vastly different - use-cases.

Building Traffic Server¶

A lot of speed can be gained or lost depending on the way ATS is built.

Tuning the Machine¶

Operating Systems Options¶

Optimal Use of Memory¶

Tuning different Thread types¶

Tuning Plugin Execution¶

FAQs¶

curl -H"Expect:" http://www.example.com/

struct curl_slist *header_list=NULL;
header_list = curl_slist_append(header_list, "Expect:");
curl_easy_setopt(my_curlp, CURLOPT_HTTPHEADER, header_list);

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));

Troubleshooting Tips¶

Data Missing

This document resulted from a POST operation and has expired from the cache. You can repost the form data to recreate the document by pressing the Reload button.

The document contains no data; Try again later, or contact the server's Administrator...

WARNING: Maximum document size exceeded

Feb 20 23:53:40 louis traffic_manager[4414]: ERROR ==> [drainIncomingChannel] Unknown message: 'GET http://www.telechamada.pt/ HTTP/1.0'
Feb 20 23:53:46 louis last message repeated 1 time
Feb 20 23:53:58 louis traffic_manager[4414]: ERROR ==> [drainIncomingChannel] Unknown message: 'GET http://www.ip.pt/ HTTP/1.0'

traffic_cop[16056]: encountered "var/trafficserver/no_cop" file...exiting

trafficserver start

WARNING: interface is ignored: Operation not permitted

WARNING: errno 105 is ENOBUFS (low on kernel memory), consider a memory upgrade

kernel: eth0: can't fill rx buffer (force 0)!

kernel: recvmsg bug: copied E01BA916 seq E01BAB22