# Server log formats

Server Log Monitoring reads the access logs of your web server, load balancer or CDN. This page lists the log formats that VitalSentinel reads, the fields that each log line must have, and example server settings. Use it before you connect an SFTP, FTPS, bucket or upload source. To connect a source, see [Server logs](/connect/server-logs).

<Callout type="info">
  The Cloudflare Worker does not use log files. If your site runs behind Cloudflare, the Worker collects the requests directly, and this page does not apply to it.
</Callout>

## How VitalSentinel finds the log format [#how-vitalsentinel-finds-the-log-format]

You do not select a log format. VitalSentinel examines the first lines of each file and finds the format automatically. Until it confirms the format, the **Log source** card in Server Log Monitoring **Settings** shows **Log format** as **Auto-detect**. After the first file, the card shows the format name, for example **Combined (Apache/Nginx)**.

VitalSentinel also detects compression from the file content, not from the file name. It reads plain text files and gzip (`.gz`) files. For SFTP, FTPS and bucket sources, it also reads bzip2, zstd and zip files.

If VitalSentinel cannot detect the format of a custom Nginx log, the app asks for your Nginx `log_format` line. See [Custom Nginx log formats](#custom-nginx-log-formats).

## Log formats that VitalSentinel reads [#log-formats-that-vitalsentinel-reads]

The **Log format** row of the **Log source** card uses these names:

| Log format | Written by |
|------------|------------|
| **Combined (Apache/Nginx)** | Apache and Nginx with the default `combined` format |
| **Custom Nginx log_format** | Nginx with your own `log_format`, when you give VitalSentinel the format line |
| **IIS (W3C extended)** | Microsoft IIS, with a `#Fields:` header line in each file |
| **Newline-delimited JSON** | Servers and tools that write one JSON object for each request, on one line |
| **Cloudflare log fields** | Cloudflare Logpush, with the HTTP requests dataset |
| **AWS Application Load Balancer** | AWS Application Load Balancer access logs |
| **AWS CloudFront** | AWS CloudFront standard logs in the W3C format |

VitalSentinel does not read other formats, for example the Common log format, Parquet files or CSV exports.

### The Common log format does not work [#the-common-log-format-does-not-work]

The Common log format (Apache `common`) has no User-Agent field. VitalSentinel uses the User-Agent to find the bot that sent each request, so it rejects Common logs. The error on the **Log source** card starts with "this log has no User-Agent or Referer field (Apache/Nginx Common format)."

To fix it, change your server to the Combined format. The next files that the server writes then work. Old files in the Common format stay unusable.

## Fields that VitalSentinel needs [#fields-that-vitalsentinel-needs]

Each log line must have these fields:

- **Time** of the request, with a time zone.
- **Request path**, the URL that the client asked for.
- **Status code**, the HTTP answer of your server, for example `200` or `404`.
- **User-Agent**, the name that the client sends. VitalSentinel uses it to identify Googlebot, AI crawlers and other bots.
- **Client IP address**. VitalSentinel uses it to verify that a bot is real and not a fake bot.

These fields are optional, but each one unlocks more reports: request method, query string, referrer, response size and response time. Content type, cache status, host name, country and network (ASN) are also optional.

If a source does not record a field, the **Log source** card shows **Capabilities**, with the reports that the missing field disables. See [Server logs](/connect/server-logs).

## Apache and Nginx combined format [#apache-and-nginx-combined-format]

The Combined format is the default format of Apache and Nginx. In most cases, you do not change anything. The Nginx `combined` format is:

```nginx
log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';
```

The Apache `combined` format is:

```apache
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog "logs/access_log" combined
```

VitalSentinel also detects a Combined line with more quoted fields at the end. An example is the Nginx `main` format, which adds `"$http_x_forwarded_for"`. VitalSentinel ignores these extra fields.

### Make sure your server uses the combined format [#make-sure-your-server-uses-the-combined-format]

Look at your server configuration:

- **Nginx**: find the `access_log` line. If it ends with `combined`, `main` or no format name, the format works.
- **Apache**: find the `CustomLog` line. If it ends with `combined`, the format works. If it ends with `common`, change it to `combined`.

Reload the server after a change. The new lines use the new format.

## Custom Nginx log formats [#custom-nginx-log-formats]

You can use your own Nginx `log_format`, for example to record the response time. VitalSentinel cannot always detect a custom format automatically. If it cannot, the next collection run fails, and the **Log source** card shows the error.

To give VitalSentinel your format:

1. On the **Log source** card, click **Edit**.
2. Click **Test connection**. The form shows a warning and the field **Nginx log_format directive**.
3. Paste the exact `log_format` line from your Nginx configuration.
4. Click **Test connection** again, then click **Save changes**.

The warning starts with "The connection works, but the last collection run could not process this source's log format".

A custom format must include `$time_local` or `$time_iso8601`. Without a time variable, VitalSentinel cannot read the lines.

### Example format with response time [#example-format-with-response-time]

This format adds the response time and the content type to the combined format:

```nginx
log_format vitalsentinel '$remote_addr - $remote_user [$time_local] '
                         '"$request" $status $body_bytes_sent '
                         '"$http_referer" "$http_user_agent" '
                         '$request_time "$sent_http_content_type"';

access_log /var/log/nginx/access.log vitalsentinel;
```

When VitalSentinel asks for the format line, paste the whole `log_format vitalsentinel ...;` statement.

### Nginx variables that VitalSentinel reads [#nginx-variables-that-vitalsentinel-reads]

VitalSentinel reads these Nginx variables in a custom format. It ignores other variables.

| Data | Variables |
|------|-----------|
| Time | `$time_local`, `$time_iso8601` |
| Client IP | `$remote_addr`, `$http_x_forwarded_for` |
| Request | `$request`, `$request_method`, `$request_uri`, `$uri`, `$document_uri`, `$args`, `$query_string` |
| Answer | `$status`, `$body_bytes_sent`, `$bytes_sent`, `$sent_http_content_type`, `$upstream_cache_status` |
| Time taken | `$request_time`, `$upstream_response_time` |
| Client | `$http_user_agent`, `$http_referer` |
| Host | `$host`, `$http_host` |

VitalSentinel uses `$http_x_forwarded_for` only when the format has no `$remote_addr`. Any client can write a false value into this header.

## Log the real client IP behind a proxy or CDN [#log-the-real-client-ip-behind-a-proxy-or-cdn]

If a CDN, a load balancer or a reverse proxy is in front of your server, the server sees the address of the proxy. Then every log line shows the same few IP addresses. VitalSentinel cannot verify the bots, and it can mark real Googlebot requests as fake.

VitalSentinel reads the client address field of the log line (`$remote_addr` in Nginx, `%h` or `%a` in Apache). It does not read an extra `X-Forwarded-For` column of a Combined line. Thus, configure the server to put the real client IP into the client address field.

<Callout type="warn">
  Trust only the IP addresses of your own proxy or CDN. If you trust all addresses, any client can send a false IP address in the header.
</Callout>

### Nginx real client IP [#nginx-real-client-ip]

Nginx sets the real client IP with the realip module. Nginx must include this module (`--with-http_realip_module`). To make sure, run `nginx -V` and look for the module in the output.

For a load balancer or proxy that sends `X-Forwarded-For`, add this to the `http` block:

```nginx
set_real_ip_from  10.0.0.0/8;      # the address range of your proxy
real_ip_header    X-Forwarded-For;
real_ip_recursive on;
```

For Cloudflare, use the `CF-Connecting-IP` header. Add one `set_real_ip_from` line for each Cloudflare IP range. Cloudflare publishes the ranges at https://www.cloudflare.com/ips/ and changes them from time to time.

```nginx
set_real_ip_from  192.0.2.0/24;    # repeat for each Cloudflare range
real_ip_header    CF-Connecting-IP;
```

After the change, `$remote_addr` in the `combined` format contains the real client IP.

### Apache real client IP [#apache-real-client-ip]

Apache sets the real client IP with the `mod_remoteip` module.

1. Turn on the module. On Debian and Ubuntu, run `sudo a2enmod remoteip`.
2. Add `RemoteIPHeader` and one `RemoteIPTrustedProxy` line for each address range of your proxy.
3. In the `combined` format, replace `%h` with `%a`.
4. Restart Apache.

```apache
RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 10.0.0.0/8
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
```

For Cloudflare, use `RemoteIPHeader CF-Connecting-IP` and add each range from https://www.cloudflare.com/ips/.

## IIS, JSON and other formats [#iis-json-and-other-formats]

### IIS (W3C extended) [#iis-w3c-extended]

Use the W3C log file format of IIS. Each file must start with a `#Fields:` header line, because the header tells VitalSentinel the order of the columns. In the IIS logging fields, select at least `date`, `time`, `cs-uri-stem`, `sc-status`, `c-ip` and `cs(User-Agent)`.

### Newline-delimited JSON [#newline-delimited-json]

Each line must be one JSON object for one request. VitalSentinel finds the fields by their names. These are some of the names that it reads:

| Data | Field names |
|------|-------------|
| Time | `timestamp`, `time`, `ts`, `@timestamp`, `datetime` |
| Path | `path`, `uri`, `url`, `request_path`, `request_uri` |
| Status | `status`, `status_code`, `response_status` |
| User-Agent | `user_agent`, `ua`, `useragent`, `http_user_agent` |
| Client IP | `ip`, `remote_addr`, `client_ip`, `remote_ip` |
| Method | `method`, `http_method`, `request_method` |

The time must be an ISO 8601 (RFC 3339) value with a time zone, for example `2026-10-24T08:15:00Z`, or a Unix timestamp in seconds or milliseconds. Each line must have a status field.

## CDN and cloud provider logs [#cdn-and-cloud-provider-logs]

### Cloudflare Logpush [#cloudflare-logpush]

If your site is on Cloudflare, the Cloudflare Worker is the simpler way to connect. As an alternative, Cloudflare Logpush can send the **HTTP requests** dataset to an R2, Amazon S3 or Google Cloud Storage bucket. Connect that bucket as a bucket source.

VitalSentinel reads the native Logpush field names, for example `EdgeStartTimestamp`, `ClientRequestHost`, `ClientRequestURI`, `EdgeResponseStatus`, `ClientRequestUserAgent` and `ClientIP`. Include these fields in the Logpush job. `EdgeStartTimestamp` can use the ISO 8601 or the Unix nanoseconds format.

Logpush availability and cost depend on your Cloudflare plan. Check the Cloudflare documentation.

### AWS Application Load Balancer [#aws-application-load-balancer]

AWS writes Application Load Balancer access logs to an S3 bucket. VitalSentinel reads these files as AWS writes them. Connect the bucket as an **Amazon S3** source, with the log folder as the **Prefix**.

### AWS CloudFront [#aws-cloudfront]

CloudFront standard logging can write access logs to an S3 bucket. Use the W3C output format with tab-separated fields. Each file must have its `#Fields:` header line.

Include at least the fields `date`, `time`, `cs-uri-stem`, `sc-status`, `c-ip` and `cs(User-Agent)`. Also include `x-edge-location`, because VitalSentinel uses it to tell CloudFront logs from IIS logs. VitalSentinel does not read the JSON or Parquet output of CloudFront.

### Other CDNs [#other-cdns]

VitalSentinel has no format for other CDNs, for example Fastly, Akamai or Bunny CDN. If your CDN can write the Combined format or newline-delimited JSON to a bucket, connect that bucket. If not, click **Request a connector** in the setup. See [Server logs](/connect/server-logs).

## Where servers and hosting panels keep access logs [#where-servers-and-hosting-panels-keep-access-logs]

### Servers that you manage [#servers-that-you-manage]

| Server | Usual location of the access log |
|--------|----------------------------------|
| Nginx | `/var/log/nginx/access.log`, with rotated copies such as `access.log.1` and `access.log.2.gz` |
| Apache on Debian and Ubuntu | `/var/log/apache2/access.log` |
| Apache on RHEL and CentOS | `/var/log/httpd/access_log` |
| IIS | `%SystemDrive%\inetpub\logs\LogFiles\W3SVC<site id>\` |

Your server can use a different path. The `access_log` line (Nginx) or the `CustomLog` line (Apache) shows the real path.

### cPanel [#cpanel]

In cPanel, go to **Metrics** > **Raw Access**. There you can download the access log of each domain as a `.gz` file. Upload this file with **Manual upload**. Your hosting provider can turn off this feature.

In **Configure Logs**, you can archive the logs to your home directory. cPanel then keeps them in `/home/username/logs`. If your host gives you SFTP access, you can connect this folder as an SFTP source.

### Plesk [#plesk]

In Plesk, go to **Websites and Domains**, select the domain, then click **Logs** > **Manage Log Files**. Click the download icon next to a log file. Upload the file with **Manual upload**.

On Linux, Plesk keeps these access logs for each domain: `access_log` and `access_ssl_log` (Apache), and `proxy_access_log` and `proxy_access_ssl_log` (Nginx). Use the SSL logs for HTTPS traffic. In **Log Rotation**, Plesk can compress and delete old logs.

### Managed hosting [#managed-hosting]

Many managed hosts give a log download in their dashboard or SFTP access to a logs folder. The log format of a managed host can be different from the Combined format. Upload one file first, with **Manual upload**, to make sure that VitalSentinel can read it.

## Related [#related]

- [Server logs](/connect/server-logs) - connect a log source and fix connection problems
- [Server Log Monitoring](/server-logs/overview) - what the module reports
- [Settings](/server-logs/settings) - hit allowance, excluded traffic and sections
- [Monitoring IPs and user agents](/help/monitoring-ips) - the IP addresses to allow on your server