SPSS Statistics

 View Only
Expand all | Collapse all

SPSS v 29: Lasso regression not working because of Python?

  • 1.  SPSS v 29: Lasso regression not working because of Python?

    Posted Fri January 20, 2023 09:42 AM
    Hello!
    I purchased SPSS v 29 yesterday to run lasso regressions, but when I attempt them, I get the following error message:
    Warnings
    Python is not installed or not configured properly.
    Execution of this command stops.


    What do I need to do?
    Best,
    John


    ------------------------------
    John Hill
    ------------------------------


  • 2.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Fri January 20, 2023 10:03 AM
    Hi. It's hard to know exactly what the problem is without more information.

    Start SPSS.
    Open a new syntax window.
    Copy and paste this line into that window, then run it:


    SHOW EXTPATHS PLUGINS.

    and let me know the results.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 3.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Sat January 21, 2023 12:36 PM
    This is what it gives me:



    ------------------------------
    John Hill
    ------------------------------



  • 4.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Sat January 21, 2023 01:47 PM



    (The same as above, but in plain text)

                                       

    System Settings                                 

    Keyword

     

    Description

    Setting

    EXTPATHS CDIALOGS                         

    1

    Locations for custom dialogs

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\CustomDialogs

     

    2

     

    C:\PROGRA~1\IBM\SPSS Statistics\ext\lib

    EXTPATHS EXTENSIONS

    1

    Locations for extension commands

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\extensions

     

    2

     

    C:\PROGRA~1\IBM\SPSS Statistics\extensions

    EXTPATHS RPACKAGES

    1

    Locations for R packages

    R_HOME\library

    PLUGINS

     

    Programmability plug-ins installed

    Python2:Yes--

     

     

     

    Python3:Yes--C:\PROGRA~1\IBM\SPSS Statistics\Python3

     

     

     

    R:Yes--C:\PROGRA~1\IBM\SPSS Statistics\R

     

     

     

    .NET:No

     

     

     

    JAVA:Yes

     

     

     








  • 5.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 10:38 AM
    OK. In a command window, run these commands:

    cd '%appdata%\IBM\SPSS Statistics\one\xtensions'
    dir

    and let's see what's there.



    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 6.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 11:59 AM

    C:\Users\John's Dell 2020>cd '%appdata%\IBM\SPSS Statistics\one\xtensions'
    The filename, directory name, or volume label syntax is incorrect.






  • 7.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 12:07 PM
    Well, that's a problem, if not "THE" problem.

    On the menu system, please go to Extensions > Extension Hub... and install the Lasso procedure. Search "Lasso" to make it easier to find.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 8.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 02:49 PM

    According to the Extension Hub, I have already installed the latest version (1.0.4) of Linear Lasso Regression. Could I have entered the cmd code incorrectly with what I sent you?






  • 9.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 02:54 PM
    Hm. Then let's try this:

    cd "%appdata%\IBM\SPSS Statistics"
    dir

    Let's see what you have for SPSS in general in this directory.

    If that fails:

    cd %appdata%
    cd IBM
    cd "SPSS Statistics"

    and we'll see which one of those gives an error.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 10.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 04:31 PM
    Do I need to include the quote marks in the cmd line?

    Sent from my iPhone




  • 11.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 04:33 PM
    Yes. Whenever there is a space in a directory name, we use quotation marks so the operating system will find it.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 12.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 04:59 PM

    C:\Users\John's Dell 2020>cd "%appdata%\IBM\SPSS Statistics"

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics>dir
     Volume in drive C is OS
     Volume Serial Number is 0212-F506

     Directory of C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics

    10/11/2022  07:23 AM    <DIR>          .
    09/08/2021  06:17 PM    <DIR>          ..
    08/29/2022  06:56 PM    <DIR>          28
    01/23/2023  03:58 PM    <DIR>          one
                   0 File(s)              0 bytes
                   4 Dir(s)   9,700,335,616 bytes free

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics>






  • 13.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:05 PM
    OK, then CD into the "one" directory from there:

    CD one
    dir

    Is there an xtensions directory there? Should be. Then:

    cd xtensions
    dir

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 14.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:07 PM

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics>cd one

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one>dir
     Volume in drive C is OS
     Volume Serial Number is 0212-F506

     Directory of C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one

    01/23/2023  03:58 PM    <DIR>          .
    10/11/2022  07:23 AM    <DIR>          ..
    01/23/2023  03:58 PM                61 .lastCheck.properties
    10/11/2022  07:25 AM                38 4Qresd5.xmf
    01/23/2023  03:58 PM    <DIR>          autoRecovery
    01/23/2023  01:44 PM    <DIR>          CrashReport
    01/18/2023  03:30 PM    <DIR>          CustomDialogs
    01/18/2023  03:30 PM    <DIR>          extensions
    01/18/2023  01:52 PM    <DIR>          Graphboard
    10/11/2022  07:25 AM    <DIR>          Python310
    01/23/2023  01:46 PM            35,997 statistics.jnl
    01/18/2023  01:52 PM    <DIR>          WelcomeDialog
    01/18/2023  03:30 PM    <DIR>          xtensions
                   3 File(s)         36,096 bytes
                  10 Dir(s)   9,567,612,928 bytes free

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one>






  • 15.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:11 PM
    Right. Then:

    cd xtensionsdir

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 16.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:12 PM

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one>cd xtensions

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions>dir
     Volume in drive C is OS
     Volume Serial Number is 0212-F506

     Directory of C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions

    01/18/2023  03:30 PM    <DIR>          .
    01/23/2023  03:58 PM    <DIR>          ..
    01/18/2023  03:30 PM    <DIR>          krr
    01/18/2023  03:04 PM    <DIR>          lenr
    01/18/2023  02:57 PM    <DIR>          llr
    01/18/2023  03:03 PM    <DIR>          lrr
                   0 File(s)              0 bytes
                   6 Dir(s)   9,563,668,480 bytes free

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions>






  • 17.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:13 PM
    cd llr
    dir

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 18.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:14 PM

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions>cd llr

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions\llr>dir
     Volume in drive C is OS
     Volume Serial Number is 0212-F506

     Directory of C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions\llr

    01/18/2023  02:57 PM    <DIR>          .
    01/18/2023  03:30 PM    <DIR>          ..
    01/18/2023  02:57 PM    <DIR>          intl
    01/18/2023  02:57 PM            11,358 LICENSE
    01/18/2023  02:57 PM             9,685 LLR-properties.json
    01/18/2023  02:57 PM           365,773 llr.spxt
    01/18/2023  02:57 PM            37,355 llr_wrapper.py
                   4 File(s)        424,171 bytes
                   3 Dir(s)   9,563,062,272 bytes free

    C:\Users\John's Dell 2020\AppData\Roaming\IBM\SPSS Statistics\one\xtensions\llr>







  • 19.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:24 PM
    OK. All of that looks right. Have you tried any other extensions? Could you open a syntax window, copy and paste this, then run it?

    input program.
    vector v(3).
    loop #i=1 to 50.
    loop #j=1 to 3.
    compute v(#j)=rv.normal(0,1).
    end loop.
    end case.
    end loop.
    end file.
    end input program.

    KRR v1 WITH v2 v3 /KERNEL LINEAR(ALPHA=1.0) /CRITERIA TIMER=5.


    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 20.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:26 PM

    >Error # 105.  Command name: DATASET ACTIVATE
    >This command is not valid before a working file has been defined.
    >Execution of this command stops.

    >Error # 101.  Command name: input program
    >This command is not allowed in the set of file definition commands.  Either
    >complete the definition of the file, or use the NEW FILE command to discard
    >the partial file definition.
    >Execution of this command stops.

    Kernel Ridge Regression
    Warnings
    Python is not installed or not configured properly.
    Execution of this command stops.









  • 21.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:28 PM
    OK, let's see if Python works outside of an extension call.

    dataset close all.
    new file.

    begin program Python3.
    import spss
    spss.Submit("SHOW VER.")
    end program.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 22.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Mon January 23, 2023 05:29 PM

    SHOW

    Version
    Component Version Build Date
    STATS.EXE 29.00.00.00 241 22-SEP-2022
    SPSSWCTL.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWCOM.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWLIB.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWDAT.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWDST.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWKRR.DLL 29.00.00.00 241 25-AUG-2022
    SPSSWFIL.DLL 29.00.00.00 241 25-AUG-2022









  • 23.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Wed February 22, 2023 09:37 PM
    Edited by Todd Armstrong Wed February 22, 2023 11:43 PM

    Hey there. Bumping this up. I am getting a similar error message when trying to run Lasso. Generated the following:

    Was there a solution found?

    Thanks!





  • 24.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 09:22 AM

    Hi Todd,

    The error above shows that the "numpy" cannot be located by Python. That library should be installed with the Lasso XTension and loaded into:
    "C:\Users\<username>\AppData\Roaming\IBM\SPSS Statistics\one\Python310\Lib\site-packages\numpy", where "<username>" in this path is the user name on your machine that you have edited out of the EXTPATHs above. Can you look there please and let us know if a "numpy" directory exists at the above location?

    One possible explanation for the problem could be an apostrophe in your username. That was the case in the original report in this thread made by John Hill. The basic issue there is that Python (and R) sometimes has problems when there is an apostrophe in file system paths. 

    Regards,



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 25.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 10:18 AM

    Hi Curtis.

    I actually don't see the path you are referring to:

    The full path for this folder is: C:\Users\taaa\AppData\Roaming\IBM\SPSS Statistics\one\Python310\Lib\site-packages .. I rules out the apostrophe anyhow. 

    I'm wondering if this needs to be installed as an administrator. I'm using a machine on which I am not an administrator. When I tried to save the Lasso .spxt file to the SPSS Statistics folder, I wasn't able. I saved to my desktop and installed that way instead. Would this make a difference? I am still able to open the dialog box through my .sav file, it just seems to be unable to execute... And if it does require administrator installation, how would I uninstall the current version?

    Thanks!

    Todd

     



    ------------------------------
    Todd Armstrong
    ------------------------------



  • 26.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 10:44 AM
    Edited by Curtis Browning Thu February 23, 2023 10:46 AM

    Hi Todd, that is certainly a clue. You should have write access to the folders under C:\Users\taaa\AppData\Roaming\IBM\SPSS Statistics\one. Those folders exist for each user account to write application data into. You do not need to be an administrator to write there.

    So the first thing to test is to open a Windows Explorer window and go to the `C:\Users\taaa\AppData\Roaming\IBM\SPSS Statistics\one` folder as shown in the screen shot below. There you should right-click on the "one" folder circled in red below, and choose "Properties". From the Properties window that comes up, choose the "Security" tab from that window, and take a screen shot and send it here. That will tell us if your 'taaa' account has normal access to your user account folders.
    This is what it looks like for my account, which happens to be an Administrator account. Your account can be just a normal user account and you should still have access to your account's application folders.
    This shows the folders that the taaa account should own and have write access to



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 27.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 11:17 AM

    Hey Curtis.

    Thanks for the info. Though it looks like I don't even have the security tab....

    Any inkling on where to go from here?....

    Todd



    ------------------------------
    Todd Armstrong
    ------------------------------



  • 28.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 11:58 AM

    Okay, lets take the next steps then: 

    Is this a normal Windows 11 user account, or was it setup with special profile settings by an admin?

    Lets try these tests:

    1. From a Windows Explorer window and logged in as the same user, are you able to create a new folder under "C:\Users\taaa\AppData\Roaming\IBM\SPSS Statistics\one"?
    2. Open a command window and type the following, press enter, and report the results: ECHO %APPDATA%
    3. From the same command window enter the following and send the results here: icacls "%APPDATA%\IBM\SPSS Statistics\one"

    The results above will inform our next tests.



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 29.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 12:21 PM

    It was setup with special profile settings by an admin. I'm not sure what those were, but I can say that it is a corporate account. I can also say it is Windows 10 Enterprise.

    1. I can create a new folder: 

    2. Here are results from command:

    3. Here are results from command:



    ------------------------------
    Todd Armstrong
    ------------------------------



  • 30.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 01:56 PM

    Okay, I understand why you removed the username in the screen shots above. Can you confirm that they are all the same, and the same as the login account you are using? This is especially true of the last line of the last screen shot above.

    If all the names match above, you may need help from the admin who setup the account in order to understand why the application cannot write to the normal user account folder. If they do not match, then again I think the special profile that was setup on your machine may be related.

    One last test to try so we can either confirm or refute the theory that your login account cannot create files in the user's writable APPDATA folder.

    Note that this will delete all private Statistics information such as installed extensions, custom dialogs, crash reports, and auto recovery information. Under normal circumstances it should all be set back to default values the next time you run Statistics from the same account:

    From a command prompt:

    DEL /S /Q "%APPDATA%\IBM\SPSS Statistics\one"



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 31.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 02:17 PM

    Thanks Curtis. I can confirm all are the same and they are the same account I am using.

    I ran your script and got a long output. It was essentially deleting a bunch of files. Is this what is expected? And what does it tell us?



    ------------------------------
    Todd Armstrong
    ------------------------------



  • 32.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 02:32 PM

    When you restart Statistics, look back in the "%APPDATA%\IBM\SPSS Statistics\one" folder to see if the sub-folders were recreated or not. 

    On a standard Windows 11 VM, I tried the test I asked you to perform, looking for the Security tab to see your permissions. I was able to see the tab using a non-admin account, which further confirms there is something different going on with the profile that was setup for you on that machine. Next step is to ask the administrator about that and see if they can relax permissions to allow your installed applications to write to the %APPDATA% folder.



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 33.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 03:46 PM

    Thanks Curtis. Yes, they were all recreated.

    After our last message, I reached out to our ITS folks and they were able to spot the issue and modify permissions for me right away. It fixed the issue :)

    Thank you Curtis for your prompt help today.



    ------------------------------
    Todd Armstrong
    ------------------------------



  • 34.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu February 23, 2023 05:01 PM

    Awesome! Glad you were able to get it working (with a little help from your ITS people :-) ). 

    Have nice afternoon,



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 35.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Fri July 28, 2023 10:54 AM

    I am now facing the same issue, I did all the trials above, I checked the permissions for AppData folder all granted, I am using a personal laptop so I have the admin privileges

    Any recommendations ?



    ------------------------------
    maher mousa
    ------------------------------



  • 36.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted Thu September 07, 2023 09:43 AM

    Hi, 
    I am facing the same issue and was not able to resolve it with this thread, yet. Has anyone found a solution? 



    ------------------------------
    Luisa Haberl
    ------------------------------



  • 37.  RE: SPSS v 29: Lasso regression not working because of Python?

    Posted 5 days ago

    Thank you for this @Curtis Browning.
    As you mentioned numpy and XTension, I went ahead and chose "Extensions" from the menu bar and there is "Install Python and R modules".
    Then I did "insert new field" and wrote "numpy" (just numpy without the "") into that field and hit "run".
    Now Linear Lasso Regression works.
    Also a thank you to @Rick Marcantonio for the trouble shooting help.

    I am on Mac ARM / Silicon , if it matters how the menu looks.

    All the Best



    ------------------------------
    Hans-Jascha Sibbel Becker
    ------------------------------