I've never seen this issue before, but this is how I'd start trying to figure out what's going on.
Is this reporting or dashboards? The next steps are the same either way, but there are more possible points of JS failure in dashboards.
First I would open the developer toolbar and use the element selector to find the checkboxes.

This should tell you if there's something
over the checkboxes somehow. I don't think it's likely, but if there's something blocking the element then it would explain why nothing happens.
Next, look at the console. Are there any JavaScript errors before or when the user clicks? The checkboxes Cognos uses are not actually HTML checkboxes, they're images with onclick events. Any JavaScript error will stop the execution, explaining why it's failing.
Are you using any extensions or custom controls? It may be possible something is conflicting in them.
------------------------------
Paul Mendelson
------------------------------