Originally posted by: Jinming Lv @ Symphony
Docker technology offers two integration methods: Docker-py and CLI. This blog article takes IBM Spectrum Symphony as examples to illustrate effectively using Docker-py to integrate existing products with Docker containers.
Compared to KVM and VMWare virtual machines, Docker provides a lightweight virtualization technology with excellent performance and security. You can implement Caas (Containers as a Service) micro-service architecture based on Docker. Additionally, an application can provide multiple services in general, including internal services and external services; all services will be individually packaged by container, and will be deployed and run separately. This container methodology simplifies the complexity of maintaining the entire application, and also decouples modules, making it easier to develop individually and maintaining upgrades for different modules.
Build, ship, and run Docker solutions
Docker solutions are composed of three aspects: Build, ship, and run. Using Docker, you only need to install Docker on one physical machine; the applications reside on a separate Docker image. The Docker image can then be stored and distributed by a private Docker registry, which can sync Docker images from the Docker hub, so that the Docker host can download Docker images directly from the private Docker registry. This way, underlying data services, such as the database, do not need to run inside the Docker container. Moreover, a Dockerized build gives applications a clear structure, so that upgrading, managing, and maintaining the application is very simple. You simply need to update the Docker images in the Docker registry.
Docker integration solutions
Docker uses a server-client structure: a Docker engine (or daemon) on the server side, builds, publishes, distributes, and runs Docker containers; and a Docker CLI or remote API on the client side, controls and manages processes of a remote server by communicating with the Docker daemon.
As shown in the following Docker architecture diagram, the Docker client and server run on the same system. They connect to a remote Docker daemon using the Docker client. They mainly communicate using sockets and RESTful API:
There are two methods to integrating Docker with your product:
- Docker-py: Docker-py is an open source project. It uses Python to implement the package for Docker REST API access, and provides the same functionality as the Docker CLI (such as downloading and installing Docker images, and starting and stopping Docker containers). Docker-py is an API client for Docker written in Python.
- CLI: The Docker CLI is responsible for communicating with the Docker engine, and then to build, publish, and manage Docker containers.
Docker integration with IBM Spectrum Symphony using Docker-py
IBM Spectrum Symphony is a powerful enterprise-class management software, used to run a variety of distributed applications and big data analytics on scalable shared grids. It can accelerate dozens of applications running simultaneously, to efficiently use available resources. This section uses IBM Spectrum Symphony as an example to illustrate how Docker integrates with products using Docker-py.
For load scheduling and management, the client side sends the compute node requests using IBM Spectrum Symphony API, and waits to receive results. IBM Spectrum Symphony initializes service and run requests from the client, and the results obtained from the service is sent back to the client.
The service session manager (SSM) is responsible for connecting input and output tasks between the client application and compute nodes, receive resources needed by the session from EGO (resource manager), records status and progress of sessions and tasks, manages the life cycle of service instance manager (SIM), and interacts with the resource manager.
The following diagrams show how IBM Spectrum Symphony components interact:
IBM Spectrum Symphony communicates with the Docker daemon using the Docker-py API. Each service runs inside the Docker container, and the container runs with different operating systems and users. For multiple applications, run services inside the Docker container. You can also take advantage of isolating Docker containers so that the services run more securely.
As shown in the above diagram, multiple service instances can run on a host; these services need to be started directly by the SIM. With Docker, SIM directly starts the Docker container, ad services run in the container.
The following diagram show IBM Spectrum Symphony service operation mode, using Docker:
Each API within Docker has a version number. For example, here, we can see the API version that Docker 1.9.1 uses is 1.21, and the corresponding Docker-py version is 1.7.2:
$ docker version
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:25:01 UTC 2015
OS/Arch: linux/amd64
Server:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:25:01 UTC 2015
OS/Arch: linux/amd64
To use Docker-py, first download and install it (for example, to download Docker-py 1.7.2, refer to https://github.com/docker/docker-py/archive/1.7.2.zip). Installation requires Python version 3.5 or higher:
$ ../Python-3.5.0/python ./setup.py install
Docker-py establishes a connection with the Docker daemon using a socket. The opened socket is unix: ///var/run/docker.sock. With the Docker daemon started, you can use this socket connection to the Docker daemon to list the Docker images for the local host:
$ ../Python-3.5.0/python
Python 3.5.0 (default, May 11 2016, 10:42:46)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from docker import Client
>>> client = Client(base_url='unix://var/run/docker.sock')
>>> for img in client.images() :
... print("%s" % img['RepoTags'])
...
['myimage:latest']
['docker.io/weaveworks/weave:1.5.0']
['docker.io/weaveworks/plugin:1.5.0']
['docker.io/weaveworks/weaveexec:1.5.0']
['docker.io/weaveworks/weavedb:latest']
['docker.io/busybox:latest']
You can find the Docker image in the Docker hub using the Docker-py interface:
$ ../Python-3.5.0/python
Python 3.5.0 (default, May 11 2016, 10:42:46)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from docker import Client
>>> client = Client(base_url='unix://var/run/docker.sock')
>>> for img in client.search('ibm-java'):
... print("%s (%s)" % (img['name'], img['star_count']))
...
stackinabox/ibm-java (0)
apexits/ubuntu-was-liberty-8.5.5-ibm-jdk8 (0)
>>>
The most important aspect about integrating Docker using Docker-py is to manage (start and stop) the container. To start a container with the Docker-py interface, simply use a Docker run command (while a start container needs two interfaces: client.create_container and client.start through a remote API). Therefore, if some containers do not need to start, you can just create it and save space. Just as some containers are only used to store data, you can create it, but not start it:
>>> imgs = client.images(cfilters={'RepoTags':'r`ubuntu'}) # Retrieve image reference
>>> len(imgs)
1
>>> iid = imgs[0]['Id'] # Obtain Image ID
>>> container = client.create_container(iid) # Create Container
>>> container
{u'Id': u'd965a507...', u'Warnings': None}
>>> client.start(container) # Start Container
# Or you can run container this way
>>> container = client.run(image=iid, name='my-container')
>>> container
{u'Id': u'169ac0d341a5a...', 'name': 'my-container', u'Warnings': None}
Consider several elements when running a program inside the container:
- image
- container user
- container network (using host mode network allows the service instance to have the same network environment as before)
- file directory where the container needs to mount
To ensure that this feature is intuitive, IBM Spectrum Symphony provides this functionality within the cluster management console (Docker Container Definition page):
#SpectrumComputingGroup