Hello,
I find memory capacity is the culprit of performance problem much more often than its computing power as well.
This may be dued to the fact that the growth of memory consumption is not as fast as the development of CPU speed (double each year according to Moore's law).
I find the cpu-bound system mostly dued to the large number of records processed which occurs in batch applications. Moreover, many computing intensive applications such as bioinformatic, scientific, cryptography use a different variation of application serving pattern such as grid computing as well.
Naturally, during the performance tuning session, you will encounter OutOfMemoryError issue first. This is mostly dued to bad programming practice (e.g. large object, unmanaged java collection). When this issue is overcomed and you ramp up the number of concurrent session, you may encounter locking and concurrency issue which can occur either internal to JVM (sychronized) or external to JVM such as database deadlocks.
If you drive up concurrency high enough, then you are likely to see CPU capacity as your next bottleneck.
This is my experience. Others' mileage may differ. I just think of it as the legacy of Moore's law. :-)