Apply and Verify Log Rotation on an Existing Compose Service

Changing Compose logging options does not update an existing container when you restart it. This lab checks that behavior, recreates one disposable service with a bounded json-file policy, and verifies which recent messages remain.

1. Inspect the scope before changing retention

Use Docker Desktop with Linux containers and Compose. These PowerShell steps were checked with Docker CLI/Engine 29.6.2 and Compose v5.3.1. They target a generated practice project, publish no ports and use no application data volume. Keep the same shell and empty practice directory throughout. If your local context has another name, verify its local endpoint before adapting the variable below.

For an existing production service, first confirm its context, project, effective driver/options, persistent storage and log collection requirements. Recreation interrupts that service and discards the old container's local log history. Arrange log export and a rollout window before applying the equivalent change there. The broader driver choices are in the course lesson.

2. Create an uncapped practice service

Save compose.yaml. The explicit driver selects json-file. This lab's uncapped starting case assumes the engine has no inherited json-file rotation options; check the effective configuration below before continuing. BusyBox is only a small lab image; a real deployment should use its own controlled image reference.

services:
  logger:
    image: busybox:latest
    network_mode: none
    command: ["sh", "-c", "while :; do sleep 3600; done"]
    labels:
      com.xtensyon.practice: "${LAB_ID:?Set LAB_ID first}"
    logging:
      driver: json-file

$logContext = 'desktop-linux'
$logEndpoint = docker context inspect $logContext --format '{{.Endpoints.docker.Host}}'
if ($LASTEXITCODE -ne 0 -or $logEndpoint -notlike 'npipe:////./pipe/*') {
    throw 'Use the verified local Windows Docker context for this lab.'
}
$logOs = docker --context $logContext info --format '{{.OSType}}'
if ($LASTEXITCODE -ne 0 -or $logOs -ne 'linux') { throw 'A Linux engine is required.' }
$savedLabId = $env:LAB_ID
$env:LAB_ID = [guid]::NewGuid().ToString('N').Substring(0,12)
$logProject = 'xtdemo-logs-' + $env:LAB_ID
docker --context $logContext compose -p $logProject -f compose.yaml up -d logger
if ($LASTEXITCODE -ne 0) { throw 'Practice service did not start.' }
$firstLogId = docker --context $logContext compose -p $logProject -f compose.yaml ps -q logger
docker --context $logContext inspect $firstLogId --format '{{json .HostConfig.LogConfig}}'
docker --context $logContext exec $firstLogId sh -c 'echo BEFORE_RECREATE > /proc/1/fd/1'

The initial inspection should show Type: json-file and an empty Config. If options are already present, the engine supplied a different baseline: record those options and compare against them after restart, without removing the engine's policy. The marker is written through PID 1's stdout; ordinary output from docker exec does not by itself become the service's container log.

3. Add a cap, then observe what restart preserves

Replace compose.yaml with this complete version. Tiny limits make rotation visible in a short lab; choose real retention sizes from your workload and incident needs.

services:
  logger:
    image: busybox:latest
    network_mode: none
    command: ["sh", "-c", "while :; do sleep 3600; done"]
    labels:
      com.xtensyon.practice: "${LAB_ID:?Set LAB_ID first}"
    logging:
      driver: json-file
      options:
        max-size: "10k"
        max-file: "3"

docker --context $logContext compose -p $logProject -f compose.yaml config --quiet
if ($LASTEXITCODE -ne 0) { throw 'Invalid Compose file.' }
docker --context $logContext compose -p $logProject -f compose.yaml restart logger
if ($LASTEXITCODE -ne 0) { throw 'Restart failed.' }
$restartLogId = docker --context $logContext compose -p $logProject -f compose.yaml ps -q logger
if ($restartLogId -ne $firstLogId) { throw 'Expected the same container after restart.' }
docker --context $logContext inspect $restartLogId --format '{{json .HostConfig.LogConfig}}'

The ID is unchanged and the effective Config is still empty on this lab's engine, or still matches any inherited baseline options you recorded. The file on disk expresses the desired policy; inspection shows the policy on the actual container. Restarting the process does not recreate its container configuration.

4. Recreate only the target service and inspect again

docker --context $logContext compose -p $logProject -f compose.yaml up -d --no-deps --force-recreate logger
if ($LASTEXITCODE -ne 0) { throw 'Recreation failed.' }
$newLogId = docker --context $logContext compose -p $logProject -f compose.yaml ps -q logger
if ($newLogId -eq $firstLogId) { throw 'Expected a new container.' }
docker --context $logContext inspect $newLogId --format '{{json .HostConfig.LogConfig}}'
docker --context $logContext logs $newLogId

Expected: a new ID and options max-size: 10k, max-file: 3 under driver json-file. The old BEFORE_RECREATE marker is gone because it belonged to the removed container. --no-deps limits this recreation to logger; it is not a general guarantee that a real application can tolerate a dependency being unavailable.

5. Generate enough messages to verify rotation

docker --context $logContext exec $newLogId sh -c 'i=1; while [ "$i" -le 5000 ]; do printf "line %05d retained-log-check\n" "$i"; i=$((i+1)); done > /proc/1/fd/1'
if ($LASTEXITCODE -ne 0) { throw 'Log generation failed.' }
$retainedLogs = @(docker --context $logContext logs $newLogId)
if ($LASTEXITCODE -ne 0) { throw 'Could not read retained logs.' }
$retainedLogs.Count
$retainedLogs | Select-Object -First 1 -Last 1
if ($retainedLogs -contains 'line 00001 retained-log-check') { throw 'Old line unexpectedly retained.' }
if ($retainedLogs -notcontains 'line 05000 retained-log-check') { throw 'Final line is missing.' }

Expect fewer than 5,000 retained lines, the final numbered line present and the first absent. The test run retained 268 lines; treat that as an observation, not an exact retention promise. JSON metadata and message sizes affect how many lines fit. These checks establish that older messages were removed while recent messages remained readable through Docker.

6. Understand what the cap covers and clean up

This policy rotates the driver's stdout/stderr log files for each configured container. It does not bound application-written files, backups, remote collectors or total host disk use. The size/file settings describe a rotation policy, not a hard reservation for every Docker-related byte. Monitor free space and retention separately. Do not edit or truncate Docker-managed log files directly.

Remove only this practice project's labeled service and restore the shell variable:

$ownedLogIds = @(docker --context $logContext compose -p $logProject -f compose.yaml ps -aq)
if ($LASTEXITCODE -ne 0 -or $ownedLogIds.Count -ne 1) { throw 'Unexpected practice containers.' }
foreach ($ownedLogId in $ownedLogIds) {
    $ownedLabel = docker --context $logContext inspect $ownedLogId --format '{{index .Config.Labels "com.xtensyon.practice"}}'
    if ($LASTEXITCODE -ne 0 -or $ownedLabel -ne $env:LAB_ID) { throw 'Practice label mismatch.' }
}
docker --context $logContext compose -p $logProject -f compose.yaml down
if ($LASTEXITCODE -ne 0) { throw 'Practice cleanup failed.' }
$env:LAB_ID = $savedLabId

Exercise: before cleanup, change max-size to 20k and run only restart. Inspection still shows 10k. Recreate logger, then inspect: it should show 20k. Return to 10k only if you want to repeat the original retention check. A configuration edit is complete when the running container and its observed behavior match it.

References: json-file rotation options, Compose restart behavior and Compose service logging.