IBM Destination Z - Group home

Best System Programmer Attributes

By Destination Z posted Mon December 23, 2019 03:30 PM


Since you're reading this you're likely a z Systems system programmer, somewhere in the broad spectrum spanning apprentice, novice, journeyman and grizzled veteran. And you may wrangle just one mainframe OS or multiples including z/TPF, z/VSE, z/VM, z/OS and Linux on z Systems.

But no matter; across both dimensions—experience depth and systems supported—it takes more than "snips and snails and puppy dog tails" or "sugar and spice and everything nice" to make a talented, competent, productive, personable system programmer. But what goes into the mix?

For this article, I queried mainframers:

"What attributes do system programmers absolutely need? Which are nice to have? Are there kiss-of-death traits for system programmers? Attributes can be skills, instincts, knowledge, etc. What's critical, what works, what doesn't matter, what's best never shown or seen? Perhaps at least as important, which attributes are innate and which can be learned (or, when necessary, unlearned)? Do these vary across different systems supported/maintained/developed?"

In return, I received an inbox-full listing the good, necessary, funny and toxic traits in today’s workforce. The first (!) response was:

"A systems programmer must be: Single, have no life outside of the data center, be addicted to working with inanimate objects that seemingly are alive with a mind of their own, drink endless amounts of coffee or fizzy drinks and like working in the middle of the night."

It came with a smiley and probably wasn't autobiographical, but the responder and I— and likely you—have known people like that, for better or worse.

Don't Do Mainframes Without These Traits

More seriously, there's agreement on critical traits such as curiosity, tenacity, imagination, and a healthy fear of and respect for Murphy's Law. One colleague noted a Murphy's Law subtlety:

"If it can go wrong, it already has,
but you haven't noticed it yet."

He described warding off Mr. Murphy to make his site's first disaster recovery exercise a success, with full MVS and VM recovery. The key was spending three hours a week for six weeks identifying everything Murphy could do. What they ended up with wasn’t a list of what could go wrong, because they were all variations on themes.

So they didn't prepare for each variation, but developed generalized fallbacks for various situations such as multiple core volume backups using DASD Dump Restore and DFHSM. That proved valuable when one DFHSM tape was unreadable, so they used a DDR copy to restore it. Other glitches and failures were remedied by the "belt and suspenders" preparation approach. They even had DVDs with images of core backup tapes to upload and write to tape.

Common sense, instinct, flexibility and perspective are hard to measure and gauge during job interviews. With too much work for too few people without enough time, distinguishing between critical tasks and the rest makes the difference between perpetual crisis and stable operation. Being open-minded to new solutions instead of locking into one way of doing something is important.

While knowledge is generally gained over time, it's often better working with a neophyte versus someone who's simply been around a long time. In fact, a youngster with curiosity and the ability to quickly research credible information and learn/draw from growing experience is always a valuable team member. But at any age, skills can be taught, given willingness and eagerness to learn.

Related to learning is flexibility and cooperation with others. The time for religious battles—VM vs. MVS, CMS vs. TSO, Xedit vs. ISPF, mainframe vs. everything else—is past; they waste time and make combatants look foolish to bystanders and managers. Some organizations use ITIL, described by Wikipedia as, "a set of practices for IT Service Management (ITSM) that focuses on aligning IT services with the needs of business."

Similarly, honesty and respect for company values—from initial resume and application, through owning decisions and mistakes, to forming external relationships—aren't negotiable.

Curiosity leads to introspection and pondering such questions as:

  1. How could this happen?
  2. Are there other (better!) ways to solve this?
  3. What happens if I do this?
Tenacity leads to confidence that, having analyzed a situation, the solution is there and there’s a willingness to persevere through frustration.

Willingly documenting code, interfaces, operation, configurations and everything else is essential. You won't remember how and why you did something a year later, you don't want the 3 a.m. calls and you might not be promoted if you're the only reference to critical technology. Communicate what you're thinking, what you're planning, what you're doing, what you've done, why and where you'll be if needed.

Debugging—a mandatory talent because Murphy DOES visit—requires tenacity, curiosity, openmindedness, and the ability to find and apply information. Even better than debugging skills is arguing for and being steadfast about anticipating serviceability needs and building in redundancy and fault tolerance. Instinct is a powerful problem-solving tool. I've seen good system programmers help users debug programs in languages they didn't know by asking questions until the users found the answer.

Not last among musts is something that system programmers haven't always needed since many were strictly back-room employees: a good customer service attitude. Whether they're internal or external, users can't be taken for granted. In these days of shifting computing platforms, outsourcing, offshoring and "cloudy weather," keeping them happy and loyal can be essential to not just job ratings, but continued employment.

Nice to Have

Beyond technical aptitude and an ability to avoid catastrophes, a few pleasant personal traits set apart the people we most enjoy working with. Collaboration and willingness to mentor helps team performance and morale and addresses the need to maintain mainframe staffing as people retire. Open-mindedness lets everyone contribute rather than simply deciding things based on loudness of arguments. And recognizing others' contributions shares credit for successes and enhances team credibility throughout an organization.

Bonus points are scored for aggressively bringing order out of chaos—finding areas to organize, clean up and document—such as piles of software distribution media and rats-nest network cable tangles.

Having and using personal networks amplifies one's skills and knowledge. It's worthwhile being connected on LinkedIn to relevant communities for both keeping up with the industry and getting questions answered; the same goes for participating on industry discussion lists (e.g., IBM-MAIN, IBMVM, etc.) and when possible, attending local and national conferences (e.g., CMG, VM Workshop, SHARE).

Kiss-of-Death Traits

System programmers sometimes fall into the mindset that they're the elite echelon, as expressed in the old joke that "users exist to test system programmers’ system programs." Of course that's backward; system programmers are hired to enable business functions. Arrogance and conceit at best irritate colleagues and at worst are career-limiting attitudes. Dismissing questions with RTFM—you can look it up—drives people away, which may lead to your being told to go away. Variations on grandstanding such as "I do it all, and all by myself" or "Nobody understands what I do so nobody needs to know" don't convey skill and confidence. And "I don't like how the vendor does it so we do it my way" puts out the welcome mat for Mr. Murphy.

Beware being unable to change course when theories and choices don't pan out. Technology, the industry, business requirements, economics and local priorities all change. Be sure that you're solving today's and tomorrow's problems based on all knowable factors. And never follow a change path you can't retrace to put things back the way they were when you started. That is, blend courage and fear in your activities.

Finally, it's best avoiding controversial topics at work (e.g., politics and religion) or at least not being confrontational about them.

Innate, Learned and Learnable

There's no indication that these attributes value varies across systems supported—it's all ones and zeros, as one person commented. The best people I've known and worked with have evolved through their careers, picking up not just technical skills and knowledge but also habits and instincts making them more perceptive, productive and pleasant professionals.

Picking one last ability, focus on seeing forest and trees: To ward off the unpleasant Mr. Murphy don't let a slick GUI distract you from understanding what lies beneath,

Gabe Goldberg has developed, worked with and written about technology for decades. Email him at