Visualizing DigitalOcean Spaces Access Logs With GoAccess

DigitalOcean Spaces is an S3-compatible object storage service that supports detailed access logging. Spaces can automatically generate access logs for buckets and store the logs in a bucket you specify. Access logs include:

These access logs capture reads, writes, and deletions of objects within the bucket, requests made to origin endpoints, and accesses to the bucket’s CDN endpoints if CDN is enabled.

However, there’s a gap.

While Spaces provides the logs, it does not natively offer a built-in analytics or visualization layer. This is where tools like GoAccess come in, allowing teams to transform raw log files into meaningful, interactive dashboards. This tutorial takes raw, S3-style access logs from Spaces and turns them into a human-readable, real-time dashboard using lightweight, open-source tooling.

In this tutorial, you will learn how to:

• Collect access logs from DigitalOcean Spaces
• Sync logs locally using the AWS CLI (S3-compatible)
• Parse and visualize logs using GoAccess
• Generate a single HTML dashboard to analyze Spaces traffic

This guide is especially useful for developers, DevOps engineers, and platform teams who want better visibility into Spaces usage.

• You can turn DigitalOcean Spaces access logs into a searchable HTML analytics dashboard with GoAccess in a few repeatable steps.
• Access logs are delivered asynchronously, so your workflow should include periodic sync and report refresh.
• A consistent S3 log format and a quick validation pass before report generation prevent most parsing failures.
• Internal runbooks should include log retention, key scoping, and scheduled report updates for production use.

Before you begin, make sure you have:

• A DigitalOcean account
• At least one Spaces bucket with access logging enabled. Refer to the Spaces access logs docs
• A Droplet running Ubuntu 22.04 or newer
• A Spaces access key and secret key
• Basic familiarity with the Linux command line
• Optional: Review How to Use SSH to Connect to a Remote Server in Ubuntu if you need a quick SSH refresher.

Create a new Droplet using Ubuntu. A basic Droplet (1 vCPU / 1 GB RAM) is sufficient for moderate log volumes. Once created, SSH into the Droplet.

If you are new to Spaces setup, review How to Create a Spaces Bucket before continuing.

GoAccess is a real-time web log analyzer and interactive viewer. We’ll use it to visualize our Spaces access logs in a browser-friendly HTML report.

Install GoAccess:

This updates your package list and installs GoAccess on your server:

sudo apt update
sudo apt install -y goaccess

Verify the installation:

Check that GoAccess was installed correctly by printing its version:

goaccess --version

This should output the installed version number, confirming it’s ready to use.

The AWS CLI v2 is required to interact with DigitalOcean Spaces, which uses S3-compatible APIs. With this tool, you’ll be able to download your access logs programmatically.

Install AWS CLI v2:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
sudo apt install -y unzip
unzip /tmp/awscliv2.zip -d /tmp
sudo /tmp/aws/install
aws --version

• The first command downloads the AWS CLI installer as a zip file.
• The next commands extract and install it.
• aws --version confirms the CLI is installed and accessible.

You’ll need to provide credentials so the AWS CLI can access your Spaces bucket.

Set up the credentials folder and open the credentials file:

mkdir -p ~/.aws
nano ~/.aws/credentials

• mkdir -p ~/.aws ensures the configuration directory exists.
• nano ~/.aws/credentials creates or opens the credentials file for editing.

Add your Spaces keys:

[default]
aws_access_key_id = YOUR_SPACES_ACCESS_KEY
aws_secret_access_key = YOUR_SPACES_SECRET_KEY

Replace YOUR_SPACES_ACCESS_KEY and YOUR_SPACES_SECRET_KEY with your actual credentials from DigitalOcean.

Configure the region and output format:

nano ~/.aws/config

[default]
region = sgp1
output = json

The region should match your Spaces region (e.g., sgp1 for Singapore); output = json sets the default AWS CLI output format.

Now, you’ll pull the access logs from your Spaces bucket down to your Droplet.

Set variables for your log bucket and prefix, and endpoint:

LOG_BUCKET="your-log-bucket-name"
LOG_PREFIX="spaces-logs-prefix/" # Prefix path where logs are written
ENDPOINT="https://sgp1.digitaloceanspaces.com"

Change LOG_BUCKET and LOG_PREFIX to match your setup.

Create a local directory for logs and sync them:

mkdir -p /var/log/spaces-logs
cd /var/log/spaces-logs

# Copy all logs in the prefix to local storage
aws --endpoint-url "$ENDPOINT" s3 cp "s3://$LOG_BUCKET/$LOG_PREFIX" ./ --recursive

• The mkdir and cd commands set up a working directory for your logs.
• The aws s3 cp command recursively copies log files from Spaces to your local directory.

Example variable values:

LOG_BUCKET="akshit-logs"
LOG_PREFIX="logs/"
ENDPOINT="https://sgp1.digitaloceanspaces.com"

Use these as a template for your own values.

Access log files from Spaces are typically compressed with gzip to save space.

To decompress all .gz files to plain .log files:

find . -type f -name "*.gz" -exec gunzip -kf {} \;

• This command searches for all .gz files and decompresses them in place.
• The -k option keeps the original .gz files, and -f forces overwrite if needed.
• After running this, you’ll have uncompressed log files ready to process.

To simplify analysis, combine all the decompressed log files into a single file:

find . -type f -name "*.log" -print0 | xargs -0 cat -- > combined.log

• This finds every .log file and concatenates them into combined.log.
• Using one combined file makes it easier for GoAccess to generate a unified report.

Sanity check:

wc -l combined.log

• This shows the number of lines in combined.log, so you can confirm that data has been collected.

Now, use GoAccess to analyze the combined log file and create your dashboard.

Recommended GoAccess command:

goaccess /var/log/spaces-logs/combined.log \
  --log-format='%^ %^ [%d:%t %^] %h %^ %^ %^ %^ "%r" %s %^ %b %^ %^ %^ "%R" "%u" %^ %^ %^ %^ %^ %^' \
  --date-format=%d/%b/%Y \
  --time-format=%T \
  -a -o /var/www/html/spaces_report.html

• Specifies the expected log format, which matches the S3 log structure.
• -a enables analytics for all log entries.
• -o defines the output path for the HTML dashboard.

If you are working with CDN logs in CloudFront format, use instead:

goaccess /var/log/spaces-logs/combined.log \
  --log-format=CLOUDFRONT \
  -a -o /var/www/html/spaces_report.html

• The built-in CLOUDFRONT format covers standard CDN logs.

Why this format works

The log format string tells GoAccess how to parse each field in the S3 log entry:

• %d/%b/%Y and %T match date and time formatting, e.g., [04/May/2016:12:11:43 +0000].
• %^ skips fields you don’t want to process.
• %h grabs the IP address.
• "%r" captures the HTTP request line in quotes.
• %s, %b, "%R", and "%u" capture response code, bytes sent, referrer, and user agent, respectively.

Quick validation for report generation: Before deploying, you can quickly generate a preview report to ensure parsing works:

goaccess /var/log/spaces-logs/combined.log \
  --log-format='%^ %^ [%d:%t %^] %h %^ %^ %^ %^ "%r" %s %^ %b %^ %^ %^ "%R" "%u" %^ %^ %^ %^ %^ %^' \
  --date-format=%d/%b/%Y \
  --time-format=%T \
  --no-global-config \
  -o /tmp/spaces_report_test.html

This creates a test dashboard in /tmp. Check that data displays as expected before publishing live.

Nginx is a lightweight web server used to serve your GoAccess HTML dashboard.

Install Nginx:

sudo apt update
sudo apt install -y nginx

• Updates the package list and installs Nginx.

Set permissions so the web server can read your report:

sudo chown -R www-data:www-data /var/www/html
sudo chmod -R 755 /var/www/html

• Changes ownership and permissions for the web root so Nginx can serve the dashboard.

Start and enable Nginx:

sudo systemctl enable --now nginx

• Starts Nginx now and configures it to start automatically on reboot.

Now, open http://YOUR_DROPLET_IP/spaces_report.html in your web browser.

You will see an interactive dashboard visualizing your Spaces access logs—enabling quick insights into storage activity, requests, and usage patterns.

• Schedule sync and report generation with cron to keep dashboards current.
• Keep Spaces keys scoped to least privilege and rotate them on a regular schedule.
• Store generated reports behind access controls if logs contain sensitive metadata.
• Track parsing failures by testing each new log format variant before automation.

1. How do I check whether Spaces access logging is enabled?

Use the bucket Settings tab in the control panel or run aws s3api get-bucket-logging against the Spaces endpoint. The response includes LoggingEnabled when logging is active.

2. Where are Spaces access logs stored?

Spaces stores logs in the destination bucket and prefix you set in the logging configuration. Keep destination and source buckets separate, and use the same region for best performance.

3. Can GoAccess parse S3-compatible logs in real time?

Yes, GoAccess can process S3-compatible logs when you provide a matching --log-format, --date-format, and --time-format. For continuously updated dashboards, run sync and generation on a schedule.

4. Why is my GoAccess report empty or missing requests?

The most common causes are mismatched log format tokens, an empty combined.log, or delays in asynchronous log delivery. Validate with wc -l combined.log and test parsing to /tmp/spaces_report_test.html.

5. How long does it take for Spaces access logs to appear?

Access logs are delivered asynchronously. In most cases logs appear within about an hour, but delays can be longer during peak periods.

In this tutorial, you configured DigitalOcean Spaces access logging, synced logs locally using the S3-compatible AWS CLI, and transformed raw log data into an interactive HTML dashboard using GoAccess. This workflow provides a lightweight and cost-effective way to gain visibility into your storage usage without relying on heavy observability stacks.

By combining Spaces’ built-in logging with open-source tooling, you now have a practical foundation for monitoring traffic patterns, debugging issues, and understanding how your data is being accessed.

Next steps

To extend this setup, review the Spaces documentation and test DigitalOcean Spaces with your production logging workflow. You can also connect your broader infrastructure with DigitalOcean Kubernetes when you are ready to scale analytics pipelines.

Related DigitalOcean resources

• How to Configure Spaces Access Logs
• How to Install Nginx on Ubuntu 22.04
• How to Use the AWS Java SDK with DigitalOcean Spaces
• How to Use Doctl, the Official DigitalOcean Command-Line Client