Table of Contents
Introduction
Docker provides an API for interacting with the Docker daemon (called the Docker Engine API), as well as SDKs for Go and Python. The SDKs allow you to build and scale Docker apps and solutions quickly and easily. If Go or Python don’t work for you, you can use the Docker Engine API directly.
For information about Docker Engine SDKs, see Develop with Docker Engine SDKs.
The Docker Engine API is a RESTful API accessed by an HTTP client such as wget or curl, or the HTTP library which is part of most modern programming languages.
More information here.
Preparation
Install the Docker Engine on your system by following this guide. Select your Desktop or Server platform and follow the instructions.
I have installed the Docker Engine on Ubuntu, but it should install just fine on all platforms listed.
Access the Docker Engine API
There are several ways to access the Docker Engine API.
Use an SDK
There are SDKs for Go and Python available. This is the preferred method if you develop a Python or Go application.
In this guide we are going to use the Docker Engine API directly.
Use the Docker Engine API directly
We can access the Docker Engine API using:
The Docker daemon can listen for Docker Engine API requests via three different types of socket.
- Via a
unixdomain socket - Via a
tcpsocket - Using Systemd socket activation
By default, a unix domain socket (or IPC socket) is created at /var/run/docker.sock.
We can use the unix domain socket or, for convenience, create a tcp socket. If we enable a tcp socket, we can also communicate with the Docker daemon from another computer (using for example Postman).
Note that this is not safe unless you take the necessary precautions. More information here.
Below is an instruction on how to make sure that the Docker daemon also listens on a tcp socket on port 2375.
Create daemon.json file in /etc/docker.
{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}Code language: JSON / JSON with Comments (json)Add /etc/systemd/system/docker.service.d/override.conf.
[Service]
ExecStart=
ExecStart=/usr/bin/dockerdCode language: Markdown (markdown)Reload the systemd daemon.
systemctl daemon-reloadCode language: Markdown (markdown)Restart the Docker Engine.
systemctl restart docker.serviceCode language: Markdown (markdown)Using the API
The Docker Engine API reference can be found here.
On the left you see the different resources like Containers, Images and Volumes with the different endpoints like Create a container or Get container logs.
Choose an endpoint to see the details on how to access it and the different responses it can have.
Ping
Let’s have a look at a simple endpoint like Ping.
You can use the unix domain socket if you have not enabled the tcp socket. You can specify localhost but any hostname will do.
$ curl --unix-socket /var/run/docker.sock --location --request GET 'http://localhost:2375/_ping'
OK
$Code language: Markdown (markdown)If you have enabled the tcp socket the following will also work.
$ curl --location --request GET 'http://localhost:2375/_ping'
OK
$Code language: Markdown (markdown)Replace localhost with the IP-address of the Docker host if you run the command from another computer.
List containers
Now let’s look at the List containers endpoint.
$ curl --location --request GET 'http://localhost:2375/containers/json'
[{"Id":"9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b","Names":["/portainer"],"Image":"portainer/portainer-ce","ImageID":"sha256:96a1c6cc3d158fac0b5be75382b9b24d0a89ed5db7a59a3442d01556c139fff1","Command":"/portainer","Created":1618732791,"Ports":[{"IP":"0.0.0.0","PrivatePort":8000,"PublicPort":8000,"Type":"tcp"},{"IP":"::","PrivatePort":8000,"PublicPort":8000,"Type":"tcp"},{"IP":"0.0.0.0","PrivatePort":9000,"PublicPort":9000,"Type":"tcp"},{"IP":"::","PrivatePort":9000,"PublicPort":9000,"Type":"tcp"}],"Labels":{},"State":"running","Status":"Up 2 minutes","HostConfig":{"NetworkMode":"default"},"NetworkSettings":{"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"929e58ac78e8ca5f02acbc1d31d3efa842498cd6451f2178f8f90a4e856c33eb","EndpointID":"56a6728ac18632ab1030323f22cc827af6350eda4dbc51efdb48971a369df5e2","Gateway":"172.17.0.1","IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02","DriverOpts":null}}},"Mounts":[{"Type":"volume","Name":"portainer_data","Source":"/var/lib/docker/volumes/portainer_data/_data","Destination":"/data","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"bind","Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]}]
$
Code language: JSON / JSON with Comments (json)As you can see the curl output is on one single line which is not very readable. We can use jq to make it more readable.
I also added --silent to suppress the output of the curl command itself. To make it even more readable, I have broken up the curl command line into multiple lines using the character “\“.
Linux allows stdout of a command to be connected to stdin of another command. You can make it do so by using the pipe character “|“.
$ curl --silent --location \
--request GET 'http://localhost:2375/containers/json' \
| jq .
[
{
"Id": "9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b",
"Names": [
"/portainer"
],
"Image": "portainer/portainer-ce",
"ImageID": "sha256:96a1c6cc3d158fac0b5be75382b9b24d0a89ed5db7a59a3442d01556c139fff1",
"Command": "/portainer",
"Created": 1618732791,
"Ports": [
{
"IP": "0.0.0.0",
"PrivatePort": 8000,
"PublicPort": 8000,
"Type": "tcp"
},
{
"IP": "::",
"PrivatePort": 8000,
"PublicPort": 8000,
"Type": "tcp"
},
{
"IP": "0.0.0.0",
"PrivatePort": 9000,
"PublicPort": 9000,
"Type": "tcp"
},
{
"IP": "::",
"PrivatePort": 9000,
"PublicPort": 9000,
"Type": "tcp"
}
],
"Labels": {},
"State": "running",
"Status": "Up 17 seconds",
"HostConfig": {
"NetworkMode": "default"
},
"NetworkSettings": {
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "929e58ac78e8ca5f02acbc1d31d3efa842498cd6451f2178f8f90a4e856c33eb",
"EndpointID": "9086df9d99450a127884b3ba5f5667c7019f4f1cbf5821a717ad543d9403dfc1",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
}
}
},
"Mounts": [
{
"Type": "volume",
"Name": "portainer_data",
"Source": "/var/lib/docker/volumes/portainer_data/_data",
"Destination": "/data",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/var/run/docker.sock",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
}
]
}
]
$Code language: JSON / JSON with Comments (json)On this Docker host we only have one container running.
We can also use jq to filter some of the output by jq filters. More information on jq here.
$ curl --silent --location \
--request GET 'http://localhost:2375/containers/json' \
| jq .[].Names
[
"/portainer"
]
$Code language: JSON / JSON with Comments (json)Get container logs
Now let’s look at the Get container logs endpoint.
In this case the URL looks like /containers/{id}/logs so we have to add the container id.
First find the container id.
We can use a filter to find the container id and add the query parameter “name” followed by the name of the container.
$ curl --silent --location \
--request GET 'http://localhost:2375/containers/json?name=portainer' \
| jq .[].Id
"9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b"
$Code language: JSON / JSON with Comments (json)Now we know the container id, we can use it to retrieve the container log.
We can use a filter to output the stderr log by adding “stderr=true” and only display the last 5 lines by adding “tail=5”.
You do not have to specify the entire container id. The first few unique characters will do.
$ curl --location \
--request GET 'http://localhost:2375/containers/9fc95/logs?stderr=true&tail=5'
2021/04/19 11:52:01 Starting Portainer 2.1.1 on :9000
2021/04/19 11:52:01 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process]
2021/04/19 11:57:01 No administrator account was created after 5 min. Shutting down the Portainer instance for security reasons.
2021/04/19 11:57:02 Starting Portainer 2.1.1 on :9000
2021/04/19 11:57:02 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process]
$Code language: Markdown (markdown)SDK versus API
The example from the previous section can also be written in Python using the Requests module.
import requests
url = "http://localhost:2375/containers/9fc95/logs?stdout=true&tail=5"
payload={}
headers = {}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)Code language: Python (python)Using the SDK this will look much cleaner and easier to manage and use.
import docker
client = docker.from_env()
container = client.containers.get("9fc95")
print(container.logs(stderr=True, tail=5).decode("ascii"))Code language: Python (python)If you want to try the code you can install Python.
apt update -y && apt install python3 python3-venv python3-pipCode language: Bash (bash)Create a new Python virtual environment and activate it.
mkdir test
cd test
python3 -m venv venv
. ./venv/bin/activateCode language: Bash (bash)Install the Docker SDK for Python.
pip install dockerCode language: Bash (bash)Create a file get_container_logs.py.
import docker
client = docker.from_env()
container = client.containers.get("9fc95")
print(container.logs(stderr=True, tail=5).decode("ascii"))Code language: Python (python)Run the Python code.
(venv) $ python3 get_container_logs.py
2021/04/19 13:37:37 server: Reverse tunnelling enabled
2021/04/19 13:37:37 server: Fingerprint 07:d0:c6:e5:ba:9b:30:0a:03:ab:bd:5c:dc:03:95:fa
2021/04/19 13:37:37 server: Listening on 0.0.0.0:8000...
2021/04/19 13:37:37 Starting Portainer 2.1.1 on :9000
2021/04/19 13:37:37 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process]
(venv) $Code language: Markdown (markdown)Deactivate the Python virtual enviroment
deactivateCode language: Bash (bash)If you want, you can now remove the test directory and its content.
More examples can be found here.