In a previous post on Understanding PowerVC APIs, we dove into the nitty, gritty details of using the REST APIs provided by PowerVC. In today's post, we'll look at some python modules you might be able to use to greatly simplify things. If you need to use another language, skip down to the end where you can find pointers to SDKs for Java, Ruby, etc.
Getting an authenticated session
The first hurdle is authentication. In the previous post, we talked in detail about the keystone API used to request a token. The python example that we gave was roughly 40 lines of code, didn't do certificate validation, used a hardcoded password, leaves you responsible for re-authentication when a token expires, etc. We can do better than that.
The keystoneauth1 module provides a Session
class that can be used to make multiple requests, using TCP session pooling and reuse to further improve performance and scaling. It will automatically request a new token when the old one expires. In general, it abstracts many of the details of authentication and session management away so that you don't have to worry about them.
So how can you use this? There are too many options to cover them all, but let's look at a few of them.
#1: Using OS_* environment variables or –os-* CLI arguments:
If you've ever used the OpenStack CLIs, you're probably already familiar with some of OpenStack's environment variables. If not, Setting environment variables in the PowerVC Knowledge Center can help get you started. Those OpenStack CLIs will let you specify many parameters via either a command line argument or an environment variable, and you can use the same utilities that facilitate this ability in your own scripting. Here's an example:
import argparse
import sys
from keystoneauth1 import loading as ks_loading
# support OS_* environment variables and/or --os-* arguments
parser = argparse.ArgumentParser()
ks_loading.register_auth_argparse_arguments(parser, sys.argv[1:],
default='v3password')
ks_loading.register_session_argparse_arguments(parser)
args = parser.parse_args()
# get a connection based on those args/vars
auth = ks_loading.load_auth_from_argparse_arguments(args)
sess = ks_loading.load_session_from_argparse_arguments(args, auth=auth)
This method is highly recommended for standalone scripts. If you don't already have the necessary environment variables set, you'll likely soon find that you need to start setting them for other things anyway. And if you want to use a different user's credentials, or talk to a different PowerVC instance, you won't have to edit your scripts because that information isn't stored in the scripts themselves.
#2: Explicit parameter coding
Maybe relying on environment variables or user input isn't the right fit for your use case. You can instead directly specify the information necessary to get a token:
from keystoneauth1.identity import v3 as auth_v3
from keystoneauth1 import session
AUTH_URL = 'https://myhostname:5000/v3/'
USER_DOMAIN_NAME = 'Default'
USERNAME = 'myusername'
PASSWORD = 'mypassword'
PROJECT_DOMAIN_NAME = 'Default'
PROJECT_NAME = 'ibm-default'
CACERT = '/etc/pki/tls/certs/powervc.crt'
auth = auth_v3.Password(AUTH_URL,
user_domain_name=USER_DOMAIN_NAME,
username=USERNAME,
password=PASSWORD,
project_domain_name=PROJECT_DOMAIN_NAME,
project_name=PROJECT_NAME)
sess = session.Session(auth=auth, verify=CACERT)
#3: Using an existing token
You can also create a Session
based off an existing token instead of user credentials:
from keystoneauth1.identity import v3 as auth_v3
from keystoneauth1 import session
AUTH_URL = 'https://myhostname:5000/v3/'
TOKEN = 'gAAAAABYRs3ku5HheueDqRzQH1dEDQlmKa3-tnwYuMyR2V0vHLbJvmKwduwsI9IlJUL' \
'b_lOs9vZLMst24dqzSnNSCcYse0pa_AKsr90Asy0tiUejZOmlyr8i6rVTpQVZcwIKum' \
'SrWOhH9BPB9LytP_2J-5-pWWTJIXqgidy82BlcEvY_Zl1Lj6aznTCpLjcpVo2x5b66K' \
'2F9ylCWYYFF2pyPSdesHD_9QpSbcg3NFH0MTomdJju9p9U'
PROJECT_DOMAIN_NAME = 'Default'
PROJECT_NAME = 'ibm-default'
CACERT = '/etc/pki/tls/certs/powervc.crt'
auth = auth_v3.Token(AUTH_URL,
token=TOKEN,
project_domain_name=PROJECT_DOMAIN_NAME,
project_name=PROJECT_NAME)
sess = session.Session(auth=auth, verify=CACERT)
Note that if you're using an existing token like this, the Session
will not be able to automatically re-authenticate when the token expires, because you haven't given it the user credentials necessary to do that.
Doing what you came to do
Obviously getting an authenticated session is just the first step. Now that you've worked that out, you can get down to doing whatever it is that you wanted to do in the first place. And once again, there are a variety of different ways you can do that...
#1: Using project-specific clients
Each of the OpenStack services present in PowerVC has its own python module for client access via that service's REST APIs. These modules (keystoneclient, novaclient, etc.) are already installed with PowerVC. A simple usage example would be the following code to list compute templates (which OpenStack calls "flavors"):
import os
from novaclient import client as nova_client
nova = nova_client.Client(version=os.getenv('OS_COMPUTE_API_VERSION', '2.1'),
session=sess)
for flavor in nova.flavors.list():
print flavor.name
Note that there is a slight twist on the above when using the cinder APIs, as shown in this example listing storage templates (which OpenStack calls "volume types"):
from cinderclient import client as cinder_client
# have to specify service_type due to bug
# https://bugs.launchpad.net/python-cinderclient/+bug/1621126
cinder = cinder_client.Client(version=os.getenv('OS_VOLUME_API_VERSION', '2'),
session=sess, service_type='volume')
for types in cinder.volume_types.list():
print types.name
To learn how to use the various clients, check their online documentation (e.g. http://docs.openstack.org/developer/python-novaclient/). You can also find the code itself under /usr/lib/python2.7/site-packages.
While these clients support a wide range of capabilities, you may find gaps, especially if you're working with an API that PowerVC has extended. With a little more work it is still possible to use the clients where those gaps exist, as demonstrated in the following example listing placement policies:
for p in nova.client.get('/ego/policy/placement')[1]['placement_policies']:
print p['name']
#2: Using the session directly
PowerVC provides a few services which are not available from the OpenStack community and do not have their own client implementations. The validation service is one example. You can use the Session
instance directly to call such APIs. For example:
import json
resp = sess.get('/v1/validate/result',
endpoint_filter={'service_type': 'ttv',
'interface': 'public'})
print json.loads(resp.text)
#3: Using the openstack module
The openstack python module is also called “the OpenStack SDK”. You'll find a lot more information in its documentation, but here's a simple example:
import argparse
import os
import sys
from openstack import connection
import os_client_config
# hack to work around bug
# https://bugs.launchpad.net/python-openstacksdk/+bug/1629359
del os.environ['OS_COMPUTE_API_VERSION']
# support OS_* environment variables and/or --os-* arguments
parser = argparse.ArgumentParser()
cloud_config = os_client_config.OpenStackConfig()
cloud_config.register_argparse_arguments(parser, sys.argv[1:])
opts = parser.parse_args()
# get a connection based on those args/vars
conn = connection.from_config(options=opts)
# do something
for flavor in conn.compute.flavors():
print flavor.name
Notice that this is actually using a different method of pulling authentication settings from the OS_* environment variables than what was detailed above. There are ways to create a Connection
where you pass in a Session
, but that is expected to be a openstack.session.Session
instance, not a keystoneauth1.session.Session
, so be careful there.
This option presents some nice features but is still going through some growing pains in my opinion. While this is the strategic option, you may find that it is not ready to do everything you want to do today. If you find missing or broken function, you can view and open bugs for the OpenStack community to address here. Update: This may not actually be strategic anymore. I would probably wait to see how things shake out before investing in this option.
Logging
The python modules discussed here include logging statements in their code. Enabling this logging can be very helpful for debugging issues as you are developing and testing your scripts. Here's an example of how you might do that at the beginning of your script:
import logging
log_path = 'myscript.log'
formatter = logging.Formatter('%(asctime)s %(levelname)s: %(name)s %(message)s')
file_handler = logging.FileHandler(log_path)
file_handler.setFormatter(formatter)
modules_to_log = ['keystoneauth', 'novaclient', 'cinderclient']
for m in modules_to_log:
logger = logging.getLogger(m)
logger.addHandler(file_handler)
logger.setLevel(logging.DEBUG)
For more information, see https://docs.python.org/2/library/logging.html.
Closing thoughts
While the modules that are discussed here are already installed by PowerVC, you are not limited to running scripts from that system. If you want to run scripts somewhere else, you can pip install the necessary modules on other systems.
We've focused on using Python modules, but there are resources for other languages as well. And even with Python we have only scratched the surface. See Development resource for OpenStack clouds and Known SDKs for more information.
We hope this has been helpful. Feel free to leave questions or requests in the comments!
#Infra#apis