PowerVC

In a previous post on Understanding PowerVC APIs, we dove into the nitty, gritty details of using the REST APIs provided by PowerVC. In today's post, we'll look at some python modules you might be able to use to greatly simplify things. If you need to use another language, skip down to the end where you can find pointers to SDKs for Java, Ruby, etc.

Getting an authenticated session

The first hurdle is authentication. In the previous post, we talked in detail about the keystone API used to request a token. The python example that we gave was roughly 40 lines of code, didn't do certificate validation, used a hardcoded password, leaves you responsible for re-authentication when a token expires, etc. We can do better than that.

The keystoneauth1 module provides a Session class that can be used to make multiple requests, using TCP session pooling and reuse to further improve performance and scaling. It will automatically request a new token when the old one expires. In general, it abstracts many of the details of authentication and session management away so that you don't have to worry about them.

So how can you use this? There are too many options to cover them all, but let's look at a few of them.

#1: Using OS_* environment variables or –os-* CLI arguments:

If you've ever used the OpenStack CLIs, you're probably already familiar with some of OpenStack's environment variables. If not, Setting environment variables in the PowerVC Knowledge Center can help get you started. Those OpenStack CLIs will let you specify many parameters via either a command line argument or an environment variable, and you can use the same utilities that facilitate this ability in your own scripting. Here's an example:

import argparse
import sys

from keystoneauth1 import loading as ks_loading

# support OS_* environment variables and/or --os-* arguments
parser = argparse.ArgumentParser()
ks_loading.register_auth_argparse_arguments(parser, sys.argv[1:],
                                            default='v3password')
ks_loading.register_session_argparse_arguments(parser)
args = parser.parse_args()

# get a connection based on those args/vars
auth = ks_loading.load_auth_from_argparse_arguments(args)
sess = ks_loading.load_session_from_argparse_arguments(args, auth=auth)

This method is highly recommended for standalone scripts. If you don't already have the necessary environment variables set, you'll likely soon find that you need to start setting them for other things anyway. And if you want to use a different user's credentials, or talk to a different PowerVC instance, you won't have to edit your scripts because that information isn't stored in the scripts themselves.

#2: Explicit parameter coding

Maybe relying on environment variables or user input isn't the right fit for your use case. You can instead directly specify the information necessary to get a token:

from keystoneauth1.identity import v3 as auth_v3
from keystoneauth1 import session

AUTH_URL = 'https://myhostname:5000/v3/'
USER_DOMAIN_NAME = 'Default'
USERNAME = 'myusername'
PASSWORD = 'mypassword'
PROJECT_DOMAIN_NAME = 'Default'
PROJECT_NAME = 'ibm-default'
CACERT = '/etc/pki/tls/certs/powervc.crt'

auth = auth_v3.Password(AUTH_URL,
                        user_domain_name=USER_DOMAIN_NAME,
                        username=USERNAME,
                        password=PASSWORD,
                        project_domain_name=PROJECT_DOMAIN_NAME,
                        project_name=PROJECT_NAME)
sess = session.Session(auth=auth, verify=CACERT)

#3: Using an existing token

You can also create a Session based off an existing token instead of user credentials:

from keystoneauth1.identity import v3 as auth_v3
from keystoneauth1 import session

AUTH_URL = 'https://myhostname:5000/v3/'
TOKEN = 'gAAAAABYRs3ku5HheueDqRzQH1dEDQlmKa3-tnwYuMyR2V0vHLbJvmKwduwsI9IlJUL' \
        'b_lOs9vZLMst24dqzSnNSCcYse0pa_AKsr90Asy0tiUejZOmlyr8i6rVTpQVZcwIKum' \
        'SrWOhH9BPB9LytP_2J-5-pWWTJIXqgidy82BlcEvY_Zl1Lj6aznTCpLjcpVo2x5b66K' \
        '2F9ylCWYYFF2pyPSdesHD_9QpSbcg3NFH0MTomdJju9p9U'
PROJECT_DOMAIN_NAME = 'Default'
PROJECT_NAME = 'ibm-default'
CACERT = '/etc/pki/tls/certs/powervc.crt'

auth = auth_v3.Token(AUTH_URL,
                     token=TOKEN,
                     project_domain_name=PROJECT_DOMAIN_NAME,
                     project_name=PROJECT_NAME)
sess = session.Session(auth=auth, verify=CACERT)

Note that if you're using an existing token like this, the Session will not be able to automatically re-authenticate when the token expires, because you haven't given it the user credentials necessary to do that.

Doing what you came to do

Obviously getting an authenticated session is just the first step. Now that you've worked that out, you can get down to doing whatever it is that you wanted to do in the first place. And once again, there are a variety of different ways you can do that...

#1: Using project-specific clients

Each of the OpenStack services present in PowerVC has its own python module for client access via that service's REST APIs. These modules (keystoneclient, novaclient, etc.) are already installed with PowerVC. A simple usage example would be the following code to list compute templates (which OpenStack calls "flavors"):

import os

from novaclient import client as nova_client

nova = nova_client.Client(version=os.getenv('OS_COMPUTE_API_VERSION', '2.1'),
                          session=sess)
for flavor in nova.flavors.list():
    print flavor.name

Note that there is a slight twist on the above when using the cinder APIs, as shown in this example listing storage templates (which OpenStack calls "volume types"):

from cinderclient import client as cinder_client

# have to specify service_type due to bug
# https://bugs.launchpad.net/python-cinderclient/+bug/1621126
cinder = cinder_client.Client(version=os.getenv('OS_VOLUME_API_VERSION', '2'),
                              session=sess, service_type='volume')
for types in cinder.volume_types.list():
    print types.name

To learn how to use the various clients, check their online documentation (e.g. http://docs.openstack.org/developer/python-novaclient/). You can also find the code itself under /usr/lib/python2.7/site-packages.

While these clients support a wide range of capabilities, you may find gaps, especially if you're working with an API that PowerVC has extended. With a little more work it is still possible to use the clients where those gaps exist, as demonstrated in the following example listing placement policies:

for p in nova.client.get('/ego/policy/placement')[1]['placement_policies']:
    print p['name']

#2: Using the session directly

PowerVC provides a few services which are not available from the OpenStack community and do not have their own client implementations. The validation service is one example. You can use the Session instance directly to call such APIs. For example:

import json

resp = sess.get('/v1/validate/result',
                endpoint_filter={'service_type': 'ttv',
                                 'interface': 'public'})
print json.loads(resp.text)

#3: Using the openstack module

The openstack python module is also called “the OpenStack SDK”. You'll find a lot more information in its documentation, but here's a simple example:

import argparse
import os
import sys

from openstack import connection
import os_client_config

# hack to work around bug
# https://bugs.launchpad.net/python-openstacksdk/+bug/1629359
del os.environ['OS_COMPUTE_API_VERSION']

# support OS_* environment variables and/or --os-* arguments
parser = argparse.ArgumentParser()
cloud_config = os_client_config.OpenStackConfig()
cloud_config.register_argparse_arguments(parser, sys.argv[1:])
opts = parser.parse_args()

# get a connection based on those args/vars
conn = connection.from_config(options=opts)

# do something
for flavor in conn.compute.flavors():
    print flavor.name

Notice that this is actually using a different method of pulling authentication settings from the OS_* environment variables than what was detailed above. There are ways to create a Connection where you pass in a Session, but that is expected to be a openstack.session.Session instance, not a keystoneauth1.session.Session, so be careful there.

This option presents some nice features but is still going through some growing pains in my opinion. While this is the strategic option, you may find that it is not ready to do everything you want to do today. If you find missing or broken function, you can view and open bugs for the OpenStack community to address here. Update: This may not actually be strategic anymore. I would probably wait to see how things shake out before investing in this option.

Logging

The python modules discussed here include logging statements in their code. Enabling this logging can be very helpful for debugging issues as you are developing and testing your scripts. Here's an example of how you might do that at the beginning of your script:

import logging

log_path = 'myscript.log'
formatter = logging.Formatter('%(asctime)s %(levelname)s: %(name)s %(message)s')
file_handler = logging.FileHandler(log_path)
file_handler.setFormatter(formatter)
modules_to_log = ['keystoneauth', 'novaclient', 'cinderclient']
for m in modules_to_log:
    logger = logging.getLogger(m)
    logger.addHandler(file_handler)
    logger.setLevel(logging.DEBUG)

For more information, see https://docs.python.org/2/library/logging.html.

Closing thoughts

While the modules that are discussed here are already installed by PowerVC, you are not limited to running scripts from that system. If you want to run scripts somewhere else, you can pip install the necessary modules on other systems.

We've focused on using Python modules, but there are resources for other languages as well. And even with Python we have only scratched the surface. See Development resource for OpenStack clouds and Known SDKs for more information.

We hope this has been helpful. Feel free to leave questions or requests in the comments!