C/C++ and Fortran

 View Only

Security: Stack Protection in XL Compilers

By Archive User posted Fri May 13, 2011 12:02 PM


Originally posted by: DavidNichols

Are you concerned about program security? The XL Fortran v13.1 and XL C/C++ v11.1 compilers have a stack protection feature. It is not an active system that protects or denies write access to the stack. Instead, it is a defensive mechanism that can detect a corrupted or overwritten stack and take the appropriate action. Regardless of how the corruption occurred, either due to programming error or an attack on your system, stack protection always terminates the executing program.

The compilers have two new command line options dealing with stack protection: -qstackprotect and -qinfo=stp.

-qstackprotect, documented here for XL Fortran and here for XL C/C++, provides you with a method to specify what procedures to protect. -qstackprotect=all informs the compiler that you would like to protect every procedure, including the main program. -qstackprotect=size=N allows you to protect procedures with vulnerable objects of a specified number of bytes or larger.

The use of these command line options raises a few questions, such as what is a vulnerable object? A vulnerable object can be thought of as an object that may be used to access data outside of its defined memory allocation. In Fortran terms, this would include dummy arguments or local variables that are allocatables, pointers or arrays.

Another question is why not protect all procedures instead of providing a mechanism to only protect certain procedures? The simple answer here is overhead. The overhead caused by stack protection is completely dependent on how the program was written, for example, imagine a program that makes extensive use of getters and setters. For every protected procedure, extra code is inserted by the compiler to construct a stack frame that inserts a canary word onto the stack between the return address and the dummy arguments and local variables, as seen below. The compiler also inserts extra code immediately before the procedure returns to verify the value of the canary. Given this overhead, you should make use of the -qstackprotect=size=N suboption to protect your software as appropriate.

/* C */
void blank ( char* cvar )
  *cvar = ' ';

! Fortran
subroutine blank ( cvar )
  character(4) :: cvar(:)
  cvar = ' '
end subroutine

The stack frame for blank looks like this:

      Return Address
      Stack Pointer
       Canary Word

The second command line option is -qinfo=stp, documented here for XL Fortran and here for XL C/C++. With this option, the compiler will provide you with warning messages that state which procedures have not been protected. By employing this option, you can fine tune the optimal -qstackprotect=size=N value for your program. We can put it all together in the following example:

! Fortran
SUBROUTINE set_broken (this, key) 
  CHARACTER(*), INTENT(in)    :: key 
  CLASS(keyboard)             :: this 
  this%number_broken = this%number_broken + 1 
  this%broken_keys(this%number_broken) = key 
END SUBROUTINE set_broken <

FUNCTION get_broken_keys (this) 
  CHARACTER(10)               :: get_broken_keys(2) 
  CLASS(keyboard), INTENT(in) :: this 
  get_broken_keys = this%broken_keys 
END FUNCTION get_broken_keys 

/* C */
void set_broken (char key) 
  keyboard.broken_keys[keyboard.number_broken] = key; 
char* get_broken_keys () 
  char local_array[20]; 
  return keyboard.broken_keys; 

Here we have two procedures: get_broken_keys which has an array of 20 bytes and set_broken which does not have a vulnerable object. If we did not want to protect either of these the appropriate command line options would be -qstackprotect=size=20 -qinfo=stp and would result in the following output:

1500-072 (W) WARNING: set_broken is not protected against stack smashing.
1500-072 (W) WARNING: get_broken_keys is not protected against stack smashing.

In this way, stack protection provides the programmer with enough transparency to fine tune the program's security without ignoring efficiency. Which also means the end user will ultimately have a product that is better protected from malicious attacks and programming error.

1 comment



Sun June 26, 2011 06:43 AM

Originally posted by: SiyuanZhang

A Chinese version of this blog can be found at developerWorks China: https://www.ibm.com/developerworks/mydeveloperworks/blogs/12bb75c9-dfec-42f5-8b55-b669cc56ad76/entry/_e5_ae_89_e5_85_a8_e6_80_a7_xl_e7_bc_96_e8_af_91_e7_9a_84_e6_a0_88_e4_bf_9d_e6_8a_a4_e6_9c_ba_e5_88_b618?lang=en thanks!