The most common problem with Windows EXCLUDE.DIR is too many backslashes, or specifying an incorrect (non-matching) drive or host. For example, a very effective EXCLUDE.DIR would be something like:
EXCLUDE.DIR Microsoft\Crypto\RSA
You can confirm whether the EXCLUDE.DIR statement is working the way you want, including the effects of the OS, TSM Client options, and the Server "Client Option Set" (CLOPSET) using the TSM Client command "PREVIEW BACKUP <directory>" -- which also explains *why* something is excluded, making it great for debugging.
In addition, Windows keeps its own list of files and directories not to backup, even using SystemState (VSS writers), by adding the "FilesNottoBackup" Registry entry. Here is the URL:
https://msdn.microsoft.com/en-us/library/windows/desktop/bb891959(v=vs.85).aspx
So even if the directory and contents are included by Windows (not TSM) you should be able to prevent their backup by SystemState.
Last word of advice -- any time there is a problem with a TSM Client, just ask for this one output file:
dsmc query systeminfo >>hostname-systeminfo-2017-02-03.txt
That contains the DSM.OPT file, output from "Q OPTION", the versions of all the TSM Client code, SCHEDLOG, ERRORLOG, and the registry entries (amongst other goodies). Once and done! :-) Good luck.
Best Regards,
Robert Luehrs