This article is the first of a four-part series addressing Db2 for z/OS and modern development utilizing a Agile methodology and DevOps processes. In this article we define traditional versus modern development and how Db2 for z/OS, while remaining the premier database for high volume system of record applications, might be left behind when it comes to best practices involving innovation. Did you notice that “Db2” follows “Traditional” in the title of this article, but “Database” follows “Modern”? It is my opinion that Db2 for z/OS is not the first database that comes to mind when considering a DevOps development methodology. However, there is no reason why it shouldn’t be!
The first thing we need to do is to identify what we mean by traditional versus modern development. In this series of articles, we will be speaking specifically of the waterfall methodology versus an Agile methodology and DevOps practices.
This is the traditional methodology that has been employed in application development for many years. This involves multiple stages of design and development in a linear flow with each stage to be completed prior to the next stage commencing, thus the waterfall designation. Some familiar stages might be design, development, testing, user acceptance, and implementation. The advantages of this methodology are extensive documentation, a well understood plan with easily laid out milestones, and a concrete set of objectives. One of the issues with this methodology was the length of time it took from design to implementation. During this time if developers would leave and get replaced there was a large amount of ramp-up time for new personnel. There was also an issue of “scope creep” where new issues would come up during development and the project could subsequently get stuck in a never-ending development cycle. Then there was the issue upon implementation where the users would find that they forgot some issues or some features that they wanted never came to fruition. For these reasons, among others, waterfall development is being replaced by DevOps and Agile development.
I don’t believe we can talk about DevOps without talking about Agile since many of the principals of DevOps expand upon the Agile methodology. The Agile development methodology abandons the strict waterfall development for a faster and more flexible development process that focuses on collaboration over documentation and self-organization over strict rules and procedures for software development. This solves some of the problems associated with large scale software development in that with Agile the process is broken up into steps in which delivery of smaller pieces of a project can be developed, tested, approved, and integrated. The issues with scope creep, staff turnover, and user expectations are addressed with this piece-wise development process. The software development process is broken up into small teams of individuals working to very quickly deliver specific pieces of software in very small intervals of time. The result is that software can be delivered faster, at least for specific desired functionality. The downside is that the development process can be viewed as disjointed, wasteful, and chaotic.
DevOps is a conglomeration of development and operations in which the goal is to offer continuous software development and monitoring. It builds on top of Agile principals and expands on it with continuous deployment, testing, operations, and collaboration; utilizing automation whenever possible. This leads to a more robust and stable development process offering continuous deployment and continuous integration of software components. Automation is a key component of the DevOps process and enables fast deployment of stable software components, and there are many tools that support this automation. A DevOps Engineer is the person responsible for implementing and maintaining these various automation tools. The deployment cycles of new software components can be delivered in a matter of weeks, versus the months or years utilizing a waterfall methodology.
Db2 for z/OS Practices Can and Must Evolve
Competition is fierce and the enterprises that can adapt and innovate the fastest are going to win. You cannot think that your enterprise is immune from competition and keep your traditional methodologies. At this point in time, IT is part of what drives your enterprise and the product it delivers to your customers, regardless of the product being delivered. Thus, innovation is the key to success, and we need to start adapting how we manage Db2 for z/OS in order to keep pace with this level of innovation.
For many years, and for the most part still today, the traditional tools for database development and maintenance in db2 for z/OS has been TSO/ISPF or possibly a set of tools based upon TSO/ISPF. Database Administrators are using either SPUFI and the Db2 system catalog, or perhaps a vendor supplied product, to create and maintain the various database objects in support of application development. IBM has, over the years, introduced PC based tools in an attempt to provide a modern GUI based interface to Db2, including Control Center, Optimization Service Center, Data Studio, and Data Server Manager. However, a collection of flat files stored as member of a PDS file served as the repository of the DDL used to define or alter database objects, with separate libraries for the development, test, user acceptance, and production environments. Maintaining these libraries of DDL was a laborious manual process requiring a significant amount of “eyeballing” the code and checking the Db2 system catalog for any inconsistencies. Creating a new environment often meant copying a significant amount of DDL, database objects, and data, which often required the submission of many batch jobs or in the very least a few massive batch jobs. Most shops had a significant number of Database Administrators with responsibility for a handful of databases. Most of these Database Administrators knew little or nothing about the applications that used the databases, as this was primarily in the hands of Data Administrator and Application Developers. While Database Administrators may want to understand what the applications using Db2 for z/OS are doing, many are so overworked that they just don’t have the time to support application development at that level.
Things now have changed dramatically for the vast majority of Db2 for z/OS Database Administrators. The increase in available and relatively inexpensive storage has shifted the focus of the job away from storage management. Db2’s ability to self-manage allocations and the internal organization of objects has dramatically reduced the need for the REORG utilities. The Db2 engine’s ability to self-tune has increased, and along with improvements in CPU and DASD performance the focus on Db2 performance issues has shifted to outliers and regressions. All this has reduced the workload on the individual Database Administrator, but has it? Whereas a Database Administrator formerly managed only a few databases in a few environments, the reduction in responsibility of managing storage and data organization has resulted in a reduction in staff. In addition, attrition due to retirements and other cost savings measures have further place responsibilities on a dwindling number of Database Administrators. Place on top of this the fact that vast majority of new hires are not acquiring z-based skills at the university level. So, the modern Db2 Database Administrator is now responsible for dozens, if not hundreds, of databases and hundreds, if not thousands, of database objects.
So, now enter the prospect of the adaptation of an Agile methodology and DevOps practices on top of the demands placed upon the average Db2 for z/OS Database Administrator. Supporting many different development efforts that focus on pieces of software, and the database objects that support that software, while keeping any changes tracked and in sync can be an overwhelming prospect. However, this is not the time for idle management of Db2 for z/OS, and enterprises are going to have to support the database administration staff with the tools and personnel necessary to facilitate a transformation. Remember that we are discussing the potential to improve innovation by utilizing and expanding existing Db2 for z/OS resources to support this effort. The enterprise Db2 for z/OS administration team will need the ability to create, manage, and destroy multiple schemas in support of fast-moving database development efforts. In addition, developer authorities, DDL version tracking, and approvals of changes will have to happen at light speed. Integration with modern application development tools, such as Git, Jenkins, Maven, and Urban Code Deploy (and potentially many others) will be required in order to keep pace with development and deployment, with intervals measured in weeks versus months. Database Administrators, as well as Data Administrators and System Administrators, will have to adapt and collaborate with newer team members in the development community, such as DevOps Engineers. We are talking about a real shift in the way we manage our Db2 for z/OS databases and we need to catch up with the innovations that are taking place within application development so that we can take advantage of the true nature of Db2 for z/OS and our system of record data store!
The next article of this four-part series will address some of the definitions and terminology used in modern application development and DevOps.
#Db2 #DBA #Db2 #Db212 #Db2z #Analytics #DevOps