Dear SPSS Statistics community,

I have been having problems with analysing interactions with covariates when using Generalized estimating equations. Briefly, here is the outline of our main issues:

While analyzing a dataset in a backward masking psychophysics study using Generalized estimating equations (GEEs) I have encountered a few problems. In the behavioural study, participants were presented with a series of body images showing different emotions, while the stimuli were masked with white noise. 

These are our variables:

• DV – Response (0- unseen, 1 – seen) – data has been averaged across 2 runs and blocks, so the values are in decimals now

• IV1 – Emotion (threat, fear, sadness)

• IV2 – Stimulus orientation (upright/inverted)

• IV3 – Stimulus gender (male, female)

• IV4 – Participant gender (male, female)

• Covariate – White noise centered (the mean was removed)

DV = dependent variable, IV = independent variable

The covariate was centered and two additional variables have been created, e.g. White noise + 1SD and White noise – 1SD, to be able to assess effects for different levels of the covariate.

We used a full factorial model with a normal distribution and log link function, AR1 was used for the covariance structure.

While running the GEE model we have encountered a few problems:

Problem 1:

On certain occasions when we try out several different models for the random (e.g., Unstructured, Independent and AR1) or fixed part (full factorial, certain interactions removed etc.), we get numerically identical Goodness of fit metrics (e.g. QIC or QICC). This seems quite unusual, considering that Unstructured and Independent, for example, are very different covariance structures and I would personally expect large differences is goodness of fit. Do you perhaps know why this is happening and whether something is wrong?

Problem 2:

When re-running the full factorial model with White noise (average), the intercept cannot be computed and an error message saying "Unable to compute due to numerical errors" appears. Why is that?

Problem 3:

When re-running the same full factorial model with White noise + 1SD (low amount of white noise), the values of the interactions with the covariate White noise are numerically identical (i.e., F-values, p-values). 

This is also the case for simple effects (the estimated marginal means and corresponding pairwise comparisons were the same). Interestingly, the F and p-values for interactions with the covariate are different if we run the model with White noise - 1SD (high amount of white noise), but the EMMs and simple effects are still numerically identical. 

Problem 4:

With the goal of avoiding re-running the GEEs for the 3 different levels of the covariate, we tried to implement changes to the syntax to obtain interaction effects for the 3 levels of the covariate within the same model right away. We tried to achieve this by adding the subcommand OTHER to the EMMEANS command, according to these instructions from the SPSS documentation (https://www.ibm.com/docs/en/SSLVMB_28.0.0/pdf/IBM_SPSS_Statistics_Command_Syntax_Reference.pdf):

However, this does not work with our version of SPSS for GEEs (SPSS version 27 and 28).

Our problems are further elaborated (including SPSS tables) in the word document in the attachment. 

We would be very grateful if you could help us with these issues, since we are really stuck.

Thank you in advance!

Attachment(s)

SPSS problem.docx   313 KB 1 version

Hi Ema,

Problem 1: The phenomenon of getting numerically identical goodness-of-fit metrics (e.g., QIC or QICC) for different models can occur due to various reasons.

Convergence to the Same Solution: It is possible that the optimization algorithms used to fit the models converged to the same solution, regardless of the initial starting values or covariance structures. This might happen if the data and models have specific characteristics that lead to a unique optimal solution.

Model Equivalence: Sometimes, different covariance structures or certain interactions can be mathematically equivalent in terms of model representation and fitting. This can lead to the same goodness-of-fit metrics even though the models look different on the surface.

Small Sample Size: In some cases, with a relatively small sample size, the statistical power to detect differences in goodness-of-fit between complex models may be limited.

Data Characteristics: The specific data being used may not vary enough in a way that would differentiate the models' goodness-of-fit. If the data patterns are consistent across different conditions, it might lead to similar results.

Problem 2: The error message "Unable to compute due to numerical errors" when trying to compute the intercept in the full factorial model with White noise (average) suggests that there might be numerical stability issues when fitting the model. This can occur due to various reasons:

Perfect Separation: If the data or model structure leads to perfect separation of the outcome variable (DV) based on the predictor variables, it can cause convergence issues. For example, if there are specific combinations of the independent variables that always result in a response of 0 or 1, the model might have difficulty estimating the intercept.

Multicollinearity: High multicollinearity among the predictor variables can lead to unstable parameter estimates, including the intercept.

Data Scaling: If the data has not been appropriately scaled or centered, it might cause numerical instability during model fitting.

Model Specification: There could be issues with the specific model specification that affect the convergence.

To address this issue, you can try the following:

Check for perfect separation and multicollinearity in your data. If detected, consider removing or combining variables to resolve these issues.

Ensure proper scaling and centering of the covariates and predictor variables.

Experiment with different optimization algorithms and starting values during model fitting.

Double-check the code  for potential errors.

Problem 3: The fact that interactions with the covariate White noise are numerically identical (i.e., F-values, p-values) when running the model with White noise + 1SD (low amount of white noise) suggests that the White noise variable might not have a significant effect in this condition. The F-values and p-values for interactions are different when running the model with White noise - 1SD (high amount of white noise), indicating a potential effect in this case.

It's important to note that a lack of significant effects for the White noise variable in one condition does not necessarily mean that it has no effect in other conditions. The presence of an interaction suggests that the relationship between the independent variables and the dependent variable is moderated by the level of the White noise variable.

To investigate further, 

Check the distribution and variability of the White noise variable in both conditions. Ensure that it varies sufficiently to detect potential effects.

Examine the scatterplots or descriptive statistics to see if there are any visible patterns that might explain the differences in effects.

Consider running additional post hoc analyses or conducting power analyses to determine if the study has enough statistical power to detect effects.

Overall, it's crucial to thoroughly examine the data, the model specifications, and the results to gain a comprehensive understanding of the patterns and potential issues in the analysis.

Problem 4:
Please try the link below, because the EMMEANS + OTHER you have tried is for Complex Samples GLM procedure 
https://www.ibm.com/docs/en/spss-statistics/28.0.0?topic=reference-genlin