How to View Docker Logs | A Complete Guide

Docker is one of the most innovative technologies developed to simplify application deployment, scaling, and management by creating lightweight, portable containers. As with many other software application technologies, Docker logs are essential to Docker containers.

This article provides a complete overview of Docker container logging. We explain how to view Docker container logs using the ‘docker logs’ command and highlight various logging strategies, including the most commonly used strategy, Docker log drivers, and configuring a different log driver for your containers.

#What are Docker container logs?

Docker container logs are recorded events, messages, and information generated by applications running inside a Docker container. These logs provide insights into the behavior and performance of the applications. Docker currently provides several logging methods. Docker container logging allows developers to identify the root cause quickly by inspecting error messages, exceptions, and other relevant information and fix problems effectively. Furthermore, Docker logs help optimize containerized applications by analyzing resource usage, response times, and other critical metrics.

There are several ways to configure logging in Docker containers. The most popular way is using a Docker log driver. By default, it is set to json-file, and you can use several log drivers according to your requirement.

#Where are Docker container logs stored?

You might wonder, "Where do the logs of my container actually go?" Well, Docker neatly places them on your host system. Go to /var/lib/docker/containers/ directory, and you'll spot individual log files named after their respective container IDs, looking something like [container-id]-json.log.

#How to view Docker logs?

Before we start, we will use some simple Docker containers to understand container logging and how to view Docker logs. First, clone the following sample git repository from the Docker GitHub:

git clone https://github.com/docker/welcome-to-docker

Build the container image using the following command.

docker build -t welcome-to-docker

Also, you can use any of your existing containers to understand how logging works. We will use the sample application react-nginx for this tutorial, which you can open in Docker dev environments.

#View Docker container logs with the docker logs command

Docker provides two types of log commands to view logs.

• docker logs
• docker service logs

The docker logs command shows the log messages generated inside the container, while the docker service logs command shows the logs generated by a service within the container.

The following is the complete command to view the Docker container logs.

docker logs <options> <containerID>

#Useful options you can use with the docker logs command

• Use the -f or --follow option to see the logs in real-time.

If you do not use the --follow option, the Docker displays only the logs available during command execution. The --follow option continues generating the log output from the container's stdout and stderr.

• View the last N number of log lines using the --tail option.

docker logs <containerID> --tail <last_number-of_lines>

• View errors using the error command

docker logs <containerID> | grep -i error

• If you want to see the specific logs, use the grep command:

docker logs <containerID> | grep <pattern>

• View logs within a specific time range using use --since and --until commands

If you want to see the logs within a specific time range, you can use the --since command. For example, suppose you need to get the logs of the last 10 hours; the log command will be as follows.

docker logs --since 10h <containerID> 

If you want to view logs produced within the last specified hours, use the –until option. For example, the following command outputs the complete log trace without the logs generated in the last two hours.

docker logs  <containerID> --until 2h 

Also, combining both --since and --until options allow you to view the logs within an exact time range. For example, the following command shows the logs generated within10 AM on August 03, 2023, and 09 AM on August 04, 2023

docker logs <containerID>  --since 2023-08-03T10:00:00 --until 2023-08-04T09:00:00 

The logs command in Docker only shows all the messages sent to standard output (stdout) and standard error (stderr) streams from a container. Nonetheless, if the services running within the container log to files or other places instead of these two output streams, it will not display the logs. Thus, these commands are not useful in some scenarios.

#Docker logging strategies

There are several logging strategies you can configure to use with Docker containers.

1. Logging drivers - Docker provides several logging drivers that manage and transport the logging in containers. We will discuss more about this strategy in the next section.
2. Application logging - Docker containers host applications and related dependencies. Applications can have their logging frameworks, such as Log4j logging library for Java applications, Winston logging library for Node.js applications, and Serilog logging library for .NET applications.
3. Data Volumes - Data volumes can retain the logs permanently in a specific directory on the Docker host. Logs can also be lost when a docker container is destroyed or terminated. Data volumes can be defined so that. You can accumulate logs and transport them to a permanent location.
4. Dedicated logging containers - Maintain a separate logging container to make collecting, managing, and analyzing the logs easier. All the logs from multiple containers will be in a centralized location. You can add new containers when the log volume increases without impacting the performance of the application containers.
5. Log Shippers - Log shippers like Filebeat or Logstash forward logs to external systems or databases.

#Docker container logging using logging drivers

One of the advanced container logging mechanisms is using logging drivers provided by Docker. Several types of logging drivers are available, and the default logging driver configured for the docker daemon is the json-file, which stores the logs as JSON. These drivers can help log to different locations, such as cloud storage, databases, and files.

#View configured logging driver

Use the following command to view the logging driver configured in your docker daemons.

docker info --format '{{.LoggingDriver}}'

Also, if you want to see the logging driver configured to your container, use the inspect command.

docker inspect -f '{{.HostConfig.LogConfig.Type}}' <container_id>

#Change the configured logging driver of existing containers

If you want to change the default logging driver, you can apply the log driver value you need to the log-driver key in the docker daemon.json configuration file. The following example shows how to specify it in a Windows machine's docker desktop. In this example, the default logging driver is overridden to use the ‘local’ logging driver.

If you run the 'docker info --format '{{.LoggingDriver}}' command again, you will see that it has been changed to the configured driver.

#Change configured logging drivers when creating a new container.

When creating a new container, you can configure the log driver using the --log-driver option to override the default logging driver. The following command shows how to create an httpd container setting the logging driver as the local log driver.

docker run --log-driver local httpd

It is important to remember that if you choose to keep the json-file or the default logging driver, you need to configure the log rotation to avoid facing out-of-disk space issues. The docker document shows the list of supported log drivers other than the json-file and local. However, if you use the ‘local’ log driver, it automatically rotates logs.

#Configuring the log delivery mode

The delivery mode of the logging driver defines how you want the log container to deliver log messages to your configured log driver. There are two delivery modes - blocking and non-blocking. You can configure this value in the log-opts option of the docker configuration file.

1. Blocking delivery mode

This is the default delivery mode set to any docker container. In this mode, when messages are delivered to the log driver from the container, all other tasks the container does will be blocked. This mode ensures the delivery of all log messages to the driver. However, since it blocks the other operations, it will degrade the container's performance.

2. Non-Blocking delivery mode

Log messages are not directly delivered to the driver in this mode. Instead, the log messages are first stored in a buffer. Once the log driver is available, the messages will be delivered to the log driver for processing, clearing the buffer. Therefore, this mode will not block other container operations, eliminating any performance impact. On the other hand, there is a possibility of losing the log messages because of buffer overload. Thus, it is important to set the buffer size to the maximum using the max-buffer-size option. The default buffer size is 1 MB.

Also, you can set the delivery mode when creating a container using the --log-opt option. Following is an example command.

docker run --log-opt mode=non-blocking httpd

#Docker container logging best practices

When configuring a log driver, the best practice is to define a log delivery method that states how to deliver the logs from the container to the log driver. Here are more best practices to implement for Docker container logging.

• Centralize logs using tools like ELK Stack or Splunk for unified analysis and visualization.
• Implement structured logging, preferably in JSON, for easier querying and analysis.
• When using Docker's json-file logging driver, use options like max-size and max-file to limit the size and the number of log files.
• Use Docker's built-in logging drivers and ensure log rotation to manage storage efficiently.
• For applications spanning multiple containers or services, use correlation IDs in logs to trace related log entries across them.

#Conclusion

Logging is essential to Docker containerized applications because it enables easier troubleshooting and optimizing the Docker containers. Docker gives users the ‘docker logs’ command to view an existing container's logs. You can use several options with it, such as viewing logs generated in a specific time range and viewing only the specific number of last generated messages.

Need a reliable server to test logging for Docker containers at scale? Cherry Servers provides high-performance workload-optimized open cloud infrastructure. Affordable hourly billing, global availability, and free 24/7 technical support ensure a smooth workflow and complete control.