Blogs

Originally posted by: xlbowler

There's been a great deal of evolution of the XL C/C++ compilers in recent years and I'm sometimes asked by customers for the inside scoop.  I've decided to share as best I can a bit of what we've been up to and where we're headed.  If you'll indulge me, I'd like to tell the story from my personal perspective because my IBM career happens to align well with the past, present and future of the XL C/C++ compilers.  I'll share with you a few of my personal opinions, not necessarily representative of IBM, so please take them for what they are. Hired in 2002 as a developer for the XL C++ frontend, I quickly became enamored with the design of programming languages and front-end architecture.   As C++ evolved over the years, I had a front row seat to a spectacular and incredibly complex evolution.  C++ went from something you could almost wrap your head around to something you distinctly knew you didn't fully understand.  Some great features like lambdas and constexpr got in.  Some great ideas like garbage collection and concepts got left behind (at least for now.)   I recall a day when the paper copy of the standard that occasionally sat on my lap felt somewhere around twice as heavy than when I started in 2002.  The process of implementing C++11 was incredibly onerous.  

As a C++ developer, one comes to realize that writing C++ standard compliant code is not sufficient to ensure portability.  Try as you may to prevent it, real code becomes dependent on vendor extensions and implementation defined behaviour.  This is especially so in a world where gcc has become a de facto standard.  The XL implementation of std::unordered_map on AIX may have an awesome rehashing algorithm but you probably don't care if the application you're porting to AIX encounters a performance problem (because it was tuned to another rehashing algorithm.)  It doesn't matter if non-plain-old-data variable length arrays are non-compliant C++: you have them in your code (possibly by accident) because it works on other platforms that you’re targeting.  

Decades ago, C++ copy-on-write (reference counted) string may have been a great implementation, but it took many years to see that it was a bad idea for multithreaded applications.  When the C++11 standard was updated to forbid reference counted strings, C++ implementations that were affected were faced with a difficult decision: either force a recompile of all C++ code or run the risk that users may encounter runtime failures when an updated string is passed to previously compiled code or vice versa.  The change is fortunate for the industry to move forward, but perhaps unfortunate for C++ users that didn't want a compatibility breakage between releases, and especially unfortunate for compiler developers who needed to navigate the ramifications. 

Over time, I have become painfully aware that different is almost always bad in C++ implementations.  The conundrum is: while a single C++ implementation doesn't benefit from competition, a myriad of implementations risks vendor locking the community.  I believe there needs to be competition amongst C++ implementations, however, such implementations need to be a collaborative effort across our industry.  Fourteen years into my compiler career, as the technical lead of the XL C/C++ front-end, I've had an opportunity to help drive XL compilers further down such a path. Many years ago, little did I know there was a pivotal evolution in C++ implementations underway.  When I first learned of Clang, an open source frontend that supports C and C++, I didn't give it much thought. As things would have it, investigation into Clang turned into quite an epiphany.  Experimentation confirmed that Clang could handle Boost libraries and complex template metaprogramming well.  The Clang diagnostic messages were surprisingly good.  Namelookup and exception handling were in good shape.  Compile time was especially good.  Everything examined showed Clang to be the real deal.  Equally appealing was the permissive licensing of Clang and the LLVM back-end, and modularity of the design.  Clang was not only an opportunity to adopt a better language implementation, it was also an opportunity to dispose of some of XL’s technical debt.  When IBM Power 8 servers and little-endian Linux on Power distributions were on the way, there was an excellent staging point for introducing the XL C/C++ products on a new architecture.  Discussion quickly moved from "if" Clang to "how" and "when".   The "how", as it turned out, was not as simple as you may think.  Typically, compiler development shops, including IBM’s, invest heavily in backend optimization technology that can be reused across many languages.  The backend technology in most compilers have been developed over many years and most compilers have their own internal intermediate representation (IR.)  While clang targets LLVM IR and provides an open-source LLVM optimizer, offering a solution that uses the Clang frontend without XL’s back-end optimization, would, at least in the short term, be untenable as it would regress performance significantly.  Different approaches to enabling proprietary XL optimization behind Clang have been the subject of intense discussion in the XL compilers team, but fortunately for you it's an implementation detail you don't need to worry about.

The first products that incorporated Clang frontend technology were the XL C/C++ 13.1.1 Linux compilers, released in December 2014.  The future direction is hopefully convergence and modernization of compiler technology across more platforms than only Linux.  It's hard to keep all stakeholders happy, but it's clear that XL C/C++ compilers have to forge ahead and keep more timely pace with the complex and rapidly evolving C++ standard.  I'm confident we've laid the ground work to do just that and I'm excited to continue bringing future XL compiler products to you.