SPSS Modeler: Validation, Cross-Validation, and the Auto Classifier Node

Hi all,

I have a couple of questions about validation and cross-validation. To help you out, I've included screenshots of my stream and all my settings, and I'll reference them in my questions. You can see my overall stream in stream.png. Basically, what I'm trying to do is classify your reaction time as fast, fast-medium, slow-medium or slow based on your brain response before you actually make the response (cool stuff!). Chance performance is 25%.

---

(1) Is the partition node pointless if you choose "cross-validate" as the option on the auto classifier node (rather than "use partitioned data) [screenshot: autoclassifier.png]?

(2) What is the difference between using "cross validate" and "use partitioned data" in the auto classifier node? [screenshot: autoclassifier.png]?

(2.5) Even though I checked "cross-validate" and not "use partitioned data" in the auto classifier node menu [screenshot: autoclassifier.png], the model output still indicated that partitioned data was used. This isn't really a question so much as a "Why bother asking permission to use partitioned data if you're just going to use it anyway?" [screenshot: model output.png]

(3) Does the auto classifier algorithm go like this: train --> validate --> test... or is it train --> test --> validate [screenshot: partition.png]?

(3.5) When evaluating overall model performance, should one pay more attention to the performance in the "test" cases or in the "validation" cases? [screenshot: analysis.png]

Thanks so much for your help!
KW
#SPSSStatistics