SPSS Statistics

 View Only
  • 1.  Calculating IQR Fence Values without Generating a Table

    Posted 17 days ago
    Edited by Mark Sloan 17 days ago

    I am automating SPSS analysis through development of an Excel-based syntax-generator and SPSS results-processor.  I need to automatically calculate IQR-based (interquartile) upper and lower fences in SPSS.  I currently use Excel to easily calculate fence values copying over SPSS-generated Frequencies-Statistics tables with quartiles (25, 50, 75), from syntax pasted from 'Analyze > Descriptive Statistics > Explore...'.  However, this is proving very inefficient for processing 2,000 multi-stage MIXED models.

    I request help to identify how to calculate single value variables in SPSS for the 25th and 75th quartiles (and then to calculate IQR and upper and lower fence values (IQR x 3.0)), without manually pulling the results from SPSS Statistics and Descriptives tables.  Everything online points to using 'Analyze > Descriptive Statistics > Explore...", but this does not efficiently work for my situation.  Instead, is there a way to use 'Transform > Compute Variable...' to generate syntax that calculates the quartiles and IQR fences for a variable as new single value variables in the dataset, without taking a detour to Excel?



    ------------------------------
    Mark Sloan
    ------------------------------



  • 2.  RE: Calculating IQR Fence Values without Generating a Table

    IBM Champion
    Posted 17 days ago
    When you say you are manually pulling these values, have you considered using OMS to capture a table of what you need?  That could be directly exported to Excel or merged in another way with the main dataset.  Constructing the syntax for this dynamically could be done with the macro facility or,  better, a short bit of Python code.  I can help with that if you can provide more details..

    --





  • 3.  RE: Calculating IQR Fence Values without Generating a Table

    Posted 16 days ago
    Edited by Mark Sloan 16 days ago

    Thank you for your quick response! Here are some more details, I hope this isn't too much:  I've developed an Excel spreadsheet that when I fill in variable names and parameters from the dataset, Excel produces the SPSS syntax for a series of MIXED models that I just copy and paste into SPSS, and viola! I have all the information I need, including MLM assumptions assessment, finding/removing extreme outliers (beyond IQR x 3.0), and getting all the fixed/random effects and their coefficients and significance for the variable relationships.  It works.  The issue is now streamlining due to a need to process up to 2,000 MIXED models in an iterative process as recommended by the multilevel modelling (MLM) literature.  I currently have the SPSS syntax in four sections, going back and forth with Excel to do some of the processing before proceeding to the next section, and I have OMS code in the syntax to create new variables for the Level-2 random effects (Intercepts and slopes). 

    The current syntax processor works well, but I now need to take the 4 sections and combine them into one.  My main issue at the moment is the desire to calculate the upper and lower IQR fence values in specific variables (at the IQR x 1.5 and IQR x 3.0 fences), to identify and FILTER out the extreme outliers. I currently take the IQR, 25, 50, 75 quartile values from SPSS, copy and paste into Excel, and Excel produces the SPSS syntax to correctly filter out the extreme outliers, which appears to work perfectly.  I would like to calculate these values within the syntax in SPSS automatically, and not in Excel.  This would allow me to combine sections and further automate the overall process.  I don't have any Python experience, and I am already wrestling code between SPSS syntax, Excel VBA, and MATLAB.  I'd prefer to keep programming directly with SPSS syntax, if possible.  Thank you for your help!



    ------------------------------
    Mark Sloan
    ------------------------------



  • 4.  RE: Calculating IQR Fence Values without Generating a Table

    Posted 16 days ago
    Edited by Mark Sloan 16 days ago

    [repeat] deleted.



  • 5.  RE: Calculating IQR Fence Values without Generating a Table

    IBM Champion
    Posted 16 days ago
    It's hard to be more specific based on your description, but perhaps it would work to capture the quartile values from FREQUENCIES via an OMS export to an Excel file and then within Excel pull those numbers into your syntax sheet.

    I do have to wonder, though, whether it is really a good idea to be excluding purported outliers en masse all the time unless there is something specific that tells you that these values do not belong in the data.

    --





  • 6.  RE: Calculating IQR Fence Values without Generating a Table

    Posted 16 days ago

    Thank you again for your quick responses.

    The process you describe is similar to what I am already doing.  I currently have Excel take the Frequencies-Statistics table quartile values, and it then calculates and pulls IQR fence values into my syntax sheet.  What I would like to do is forego Excel and have SPSS do this from COMPUTE syntax or something similar.

    I have an Excel-based "Scoreboard" that displays SPSS' results both BEFORE and AFTER extreme outlier removal, to allow me to compare the impacts of the SPSS-Excel-identified outliers on the analysis.  My data collection process included physiological measures of subjects in various VR urban environments. Abnormal movements of arms, fingers, body could cause extreme outliers.  By fencing for extreme outliers beyond IQR x 3.0 (and keeping those at IQR x 1.5), this provides a mechanism to address potential "contaminants" from the data collection process, while enabling the ability to still see results with these "outliers" included as well.  

    I know nothing of Python, but if this is something I could learn enough of to implement in a day or two in SPSS, I am open to it.  I will review your other responses as well.  Thank you for your help!



    ------------------------------
    Mark Sloan
    ------------------------------



  • 7.  RE: Calculating IQR Fence Values without Generating a Table

    Posted 16 days ago

    In my other post, I just laid out exactly what I am trying to do.  Thank you for your help and patience!



    ------------------------------
    Mark Sloan
    ------------------------------