Wall socket with moby and text

Introduction to Docker Engine API

Introduction

Docker provides an API for interacting with the Docker daemon (called the Docker Engine API), as well as SDKs for Go and Python. The SDKs allow you to build and scale Docker apps and solutions quickly and easily. If Go or Python don’t work for you, you can use the Docker Engine API directly.

For information about Docker Engine SDKs, see Develop with Docker Engine SDKs.

The Docker Engine API is a RESTful API accessed by an HTTP client such as wget or curl, or the HTTP library which is part of most modern programming languages.

More information here.

Preparation

Install the Docker Engine on your system by following this guide. Select your Desktop or Server platform and follow the instructions.

I have installed the Docker Engine on Ubuntu, but it should install just fine on all platforms listed.

Access the Docker Engine API

There are several ways to access the Docker Engine API.

Use an SDK

There are SDKs for Go and Python available. This is the preferred method if you develop a Python or Go application.

In this guide we are going to use the Docker Engine API directly.

Use the Docker Engine API directly

We can access the Docker Engine API using:

  • wget or curl
  • HTTPie
  • Postman
  • …or any other HTTP client or HTTP library

The Docker daemon can listen for Docker Engine API requests via three different types of socket.

By default, a unix domain socket (or IPC socket) is created at /var/run/docker.sock.

We can use the unix domain socket or, for convenience, create a tcp socket. If we enable a tcp socket, we can also communicate with the Docker daemon from another computer (using for example Postman).

Note that this is not safe unless you take the necessary precautions. More information here.

Below is an instruction on how to make sure that the Docker daemon also listens on a tcp socket on port 2375.

Create daemon.json file in /etc/docker.

{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}
Code language: JSON / JSON with Comments (json)

Add /etc/systemd/system/docker.service.d/override.conf.

[Service] ExecStart= ExecStart=/usr/bin/dockerd
Code language: Markdown (markdown)

Reload the systemd daemon.

systemctl daemon-reload
Code language: Markdown (markdown)

Restart the Docker Engine.

systemctl restart docker.service
Code language: Markdown (markdown)

Using the API

The Docker Engine API reference can be found here.

On the left you see the different resources like Containers, Images and Volumes with the different endpoints like Create a container or Get container logs.

Choose an endpoint to see the details on how to access it and the different responses it can have.

Ping

Let’s have a look at a simple endpoint like Ping.

You can use the unix domain socket if you have not enabled the tcp socket. You can specify localhost but any hostname will do.

$ curl --unix-socket /var/run/docker.sock --location --request GET 'http://localhost:2375/_ping' OK $
Code language: Markdown (markdown)

If you have enabled the tcp socket the following will also work.

$ curl --location --request GET 'http://localhost:2375/_ping' OK $
Code language: Markdown (markdown)

Replace localhost with the IP-address of the Docker host if you run the command from another computer.

List containers

Now let’s look at the List containers endpoint.

$ curl --location --request GET 'http://localhost:2375/containers/json' [{"Id":"9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b","Names":["/portainer"],"Image":"portainer/portainer-ce","ImageID":"sha256:96a1c6cc3d158fac0b5be75382b9b24d0a89ed5db7a59a3442d01556c139fff1","Command":"/portainer","Created":1618732791,"Ports":[{"IP":"0.0.0.0","PrivatePort":8000,"PublicPort":8000,"Type":"tcp"},{"IP":"::","PrivatePort":8000,"PublicPort":8000,"Type":"tcp"},{"IP":"0.0.0.0","PrivatePort":9000,"PublicPort":9000,"Type":"tcp"},{"IP":"::","PrivatePort":9000,"PublicPort":9000,"Type":"tcp"}],"Labels":{},"State":"running","Status":"Up 2 minutes","HostConfig":{"NetworkMode":"default"},"NetworkSettings":{"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"929e58ac78e8ca5f02acbc1d31d3efa842498cd6451f2178f8f90a4e856c33eb","EndpointID":"56a6728ac18632ab1030323f22cc827af6350eda4dbc51efdb48971a369df5e2","Gateway":"172.17.0.1","IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02","DriverOpts":null}}},"Mounts":[{"Type":"volume","Name":"portainer_data","Source":"/var/lib/docker/volumes/portainer_data/_data","Destination":"/data","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"bind","Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]}] $
Code language: JSON / JSON with Comments (json)

As you can see the curl output is on one single line which is not very readable. We can use jq to make it more readable.

I also added --silent to suppress the output of the curl command itself. To make it even more readable, I have broken up the curl command line into multiple lines using the character “\“.

Linux allows stdout of a command to be connected to stdin of another command. You can make it do so by using the pipe character “|“.

$ curl --silent --location \ --request GET 'http://localhost:2375/containers/json' \ | jq . [ { "Id": "9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b", "Names": [ "/portainer" ], "Image": "portainer/portainer-ce", "ImageID": "sha256:96a1c6cc3d158fac0b5be75382b9b24d0a89ed5db7a59a3442d01556c139fff1", "Command": "/portainer", "Created": 1618732791, "Ports": [ { "IP": "0.0.0.0", "PrivatePort": 8000, "PublicPort": 8000, "Type": "tcp" }, { "IP": "::", "PrivatePort": 8000, "PublicPort": 8000, "Type": "tcp" }, { "IP": "0.0.0.0", "PrivatePort": 9000, "PublicPort": 9000, "Type": "tcp" }, { "IP": "::", "PrivatePort": 9000, "PublicPort": 9000, "Type": "tcp" } ], "Labels": {}, "State": "running", "Status": "Up 17 seconds", "HostConfig": { "NetworkMode": "default" }, "NetworkSettings": { "Networks": { "bridge": { "IPAMConfig": null, "Links": null, "Aliases": null, "NetworkID": "929e58ac78e8ca5f02acbc1d31d3efa842498cd6451f2178f8f90a4e856c33eb", "EndpointID": "9086df9d99450a127884b3ba5f5667c7019f4f1cbf5821a717ad543d9403dfc1", "Gateway": "172.17.0.1", "IPAddress": "172.17.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:11:00:02", "DriverOpts": null } } }, "Mounts": [ { "Type": "volume", "Name": "portainer_data", "Source": "/var/lib/docker/volumes/portainer_data/_data", "Destination": "/data", "Driver": "local", "Mode": "z", "RW": true, "Propagation": "" }, { "Type": "bind", "Source": "/var/run/docker.sock", "Destination": "/var/run/docker.sock", "Mode": "", "RW": true, "Propagation": "rprivate" } ] } ] $
Code language: JSON / JSON with Comments (json)

On this Docker host we only have one container running.

We can also use jq to filter some of the output by jq filters. More information on jq here.

$ curl --silent --location \ --request GET 'http://localhost:2375/containers/json' \ | jq .[].Names [ "/portainer" ] $
Code language: JSON / JSON with Comments (json)

Get container logs

Now let’s look at the Get container logs endpoint.

In this case the URL looks like /containers/{id}/logs so we have to add the container id.

First find the container id.

We can use a filter to find the container id and add the query parameter “name” followed by the name of the container.

$ curl --silent --location \ --request GET 'http://localhost:2375/containers/json?name=portainer' \ | jq .[].Id "9fc959395baa3ab31ef0bb501a5400379cd5dbd9e2fbbc1d50b28b5a3d63914b" $
Code language: JSON / JSON with Comments (json)

Now we know the container id, we can use it to retrieve the container log.

We can use a filter to output the stderr log by adding “stderr=true” and only display the last 5 lines by adding “tail=5”.

You do not have to specify the entire container id. The first few unique characters will do.

$ curl --location \ --request GET 'http://localhost:2375/containers/9fc95/logs?stderr=true&tail=5' 2021/04/19 11:52:01 Starting Portainer 2.1.1 on :9000 2021/04/19 11:52:01 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process] 2021/04/19 11:57:01 No administrator account was created after 5 min. Shutting down the Portainer instance for security reasons. 2021/04/19 11:57:02 Starting Portainer 2.1.1 on :9000 2021/04/19 11:57:02 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process] $
Code language: Markdown (markdown)

SDK versus API

The example from the previous section can also be written in Python using the Requests module.

import requests url = "http://localhost:2375/containers/9fc95/logs?stdout=true&tail=5" payload={} headers = {} response = requests.request("GET", url, headers=headers, data=payload) print(response.text)
Code language: Python (python)

Using the SDK this will look much cleaner and easier to manage and use.

import docker client = docker.from_env() container = client.containers.get("9fc95") print(container.logs(stderr=True, tail=5).decode("ascii"))
Code language: Python (python)

If you want to try the code you can install Python.

apt update -y && apt install python3 python3-venv python3-pip
Code language: Bash (bash)

Create a new Python virtual environment and activate it.

mkdir test cd test python3 -m venv venv . ./venv/bin/activate
Code language: Bash (bash)

Install the Docker SDK for Python.

pip install docker
Code language: Bash (bash)

Create a file get_container_logs.py.

import docker client = docker.from_env() container = client.containers.get("9fc95") print(container.logs(stderr=True, tail=5).decode("ascii"))
Code language: Python (python)

Run the Python code.

(venv) $ python3 get_container_logs.py 2021/04/19 13:37:37 server: Reverse tunnelling enabled 2021/04/19 13:37:37 server: Fingerprint 07:d0:c6:e5:ba:9b:30:0a:03:ab:bd:5c:dc:03:95:fa 2021/04/19 13:37:37 server: Listening on 0.0.0.0:8000... 2021/04/19 13:37:37 Starting Portainer 2.1.1 on :9000 2021/04/19 13:37:37 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process] (venv) $
Code language: Markdown (markdown)

Deactivate the Python virtual enviroment

deactivate
Code language: Bash (bash)

If you want, you can now remove the test directory and its content.

More examples can be found here.