SPSS Statistics

 View Only
Expand all | Collapse all

Issue with SPSS Duplicating and Replacing Lines of Data

  • 1.  Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed August 05, 2020 09:51 AM
    Hello, I'm using SPSS on my Windows PC at work. I've had this issue come up twice where some lines of data will be duplicated and will replace other lines of data. For example, we have 107 study subjects, and subject #1's data will duplicate and replace the data for 3-4 subjects. This is a difficult error to catch, as the database will show that it still has 107 lines of data correlating with our 107 subjects. This happened once when I was using SPSS 26 and happened again recently with SPSS 27. I believe it's happened after sorting cases and saving. I was not analyzing any data in both instances, only editing certain cells. Has anyone had this happen before? Could I be doing something wrong that is causing this? Any insight as to how to prevent/fix this is greatly appreciated.

    ------------------------------
    Vida Sadeghi
    ------------------------------

    #SPSSStatistics


  • 2.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    IBM Champion
    Posted Wed August 05, 2020 12:48 PM
    I infer that you were editing data using the Data Editor Data View.  The only thing I can think of is that you might have accidentally copied and pasted a case in the process.
    If the IDs are sequential numbers or some other regular pattern or you can count on uniqueness, you could run a check after an editing session to see if the IDs violate the pattern.

    ------------------------------
    Jon Peck
    ------------------------------



  • 3.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Tue July 06, 2021 09:44 AM
    I have exactly the same problem. ​When editing data in the SPSS 27 editor, SPSS randomly duplicates lines of data and overwrites existing data. So far, this has only occurred with one particular dataset. Could there be a problem with the file? I have not copy/pasted complete lines of data myself, so I definitely did not create these duplicates myself.  
    Thanks in advance for any advice!

    ------------------------------
    Gerhard Rocholl
    ------------------------------



  • 4.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed July 07, 2021 08:25 AM
    Hi Gerhard,

    If you are able to send me a copy of the data file that is resulting in duplicated cases, along with the steps you are taking when cases are duplicated, I will be happy to take a look at it and get back to you with my findings.

    Best,
    Curtis Browning

    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 5.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed October 20, 2021 05:31 AM
    Curtis, this thread looks alarming. Users may want to know what was going there, what versions of SPSS can be affected, and how one can check themselves and to try to reproduce the "case duplicating" error. I've not encountered the error myself so far but I am worried. Do you have any news? Is there a troubleshooting case opened?

    ------------------------------
    Kirill Orlov
    ------------------------------



  • 6.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed October 20, 2021 10:29 AM
    Hi Kirill. I responded 2 days ago here in this thread (see above). We have extensively tested a dataset provided by Dr. Thimna Klatt and been unable to reproduce the issue using Statistics 28.0.0. As noted in the earlier message, we encourage anyone seeing problems of this nature with v28 to please contact us.

    Regards,
    Curtis Browning

    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 7.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Tue July 06, 2021 01:27 PM
    Hello!
    I have exactly the same problem. I'm using SPSS 27 and when editing data in the SPSS editor, SPSS quite regularly replaces random lines of data with duplicates of other cases. And I did not copy/paste whole lines of data, so this is definitely not a "manual mistake".
    So far, this has only occurred when I'm working on a specific dataset. Could there be a problem with the file?
    Thanks in advance!

    ------------------------------
    Gerhard Rocholl
    ------------------------------



  • 8.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Mon September 20, 2021 09:15 AM
    Hello there

    Please may I ask if you managed to find out what was causing this problem? I am experiencing the same issue working with a large dataset in SPSS 27 and it is causing lots of stress and frustration!

    Any advice you are able to offer would be hugely appreicated!

    ------------------------------
    Keeley Dobinson
    ------------------------------



  • 9.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Mon October 18, 2021 09:41 AM
    Edited by System Fri January 20, 2023 04:08 PM
    We believe this has been fixed in version 28.0.0 as it cannot be reproduced in-house with that version of Statistics. Before closing this issue we would like to see if any users have seen this problem with version 28? Please reply here or privately with any experiences with this issue using v28.

    @Vida Sadeghi, @Gerhard Rocholl, @Keeley Dobinson

    Thank you,
    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 10.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Fri May 13, 2022 09:23 AM
    Hello,

    I am having the same trouble with SPSS version 28.0.0.
    It is duplicating the number of cases in my sample. When I ask for an analysis it considers the duplicated data and not the original one.

    Last month I didn't have any problem. It started a few weeks ago...

    Can someone help me with this?

    ------------------------------
    Margarida Santos
    ------------------------------



  • 11.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Fri May 13, 2022 09:40 AM
    Hi Margarida,

    Do you have replication steps for this problem. So far no one has been able to provide steps that will enable us to see this in-house. We have tried numerous scenarios and thus far been unable to make it happen.

    Can you share the exact steps you followed when this problem occurred please? Are there steps one can take to make the problem manifest? Was it specific to a particular dataset?

    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 12.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Mon August 22, 2022 12:29 PM
    Hi. I am having exactly the same problem as you describe, using SPSS version 28.0.1.0. While editing data in the data view, suddenly one line of data duplicates and overwrites another line. The total number of lines keep the same. I was just trying some things, so I used only a part of my original dataset with 5 datalines and actually saw it happen in front of me. As I was just editing manually, I cannot provide any reproducible paths to let it happen. The only way I can think of is videorecording my screen as I am editing and "hoping" for it to happen, so others can see it too. I'm afraid I don't trust using SPSS for my data anymore, as this now happened a few times, using different datasets.

    ------------------------------
    Sanne van Dijk
    ------------------------------



  • 13.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Mon August 22, 2022 01:02 PM
    Hi @Sanne van Dijk. Sorry to hear that you're experiencing problems with the Data Editor. Can you share a bit more information please?

    1. Are you running on MacOS or Windows?
    2. What actions are you doing in the Data Editor when the problem occurs? Did you use the clipboard to copy rows of data, were you editing a single cell, etc.
    3. Had you recently sorted the data before the problem appeared?

    Thanks - again apologies for the problem, but this is a tricky one and we're still dedicated to discovering and resolving the problem.

    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 14.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Tue August 23, 2022 09:04 AM

    Dear Curtis,

     

    Thank you for your response. I agree with you: it is tricky. I tried to answer your questions as completely as possible:

     

    1. Windows 10 Pro.

    2. I manually added variables for which I filled in the values in single cells A) manually and B) by copy-pasting via the clipboard, indeed. I did not copy complete rows of data.

    3. I did not sort the data while I was working on cleaning the data.

     

    I hope that these answers are helpful.

     

    Kind regards,

    Sanne van Dijk

     

    Sanne van Dijk MSc BA

    PhD candidate Health Technology & Services Research group

    University of Twente ï Faculty of Behavioural, Management & Social Sciences (BMS) ï www.utwente.nl/bms/htsr

    Technical Medical Centre (Technohal), room 3106 ï visiting address: Hallenweg 5 ï 7522 NH Enschede

    Telephone: +31 (0)53 489 8866 ï E-mail: s.h.b.vandijk@utwente.nl  

    Post office box 217 ï 7500 AE Enschede ï Office hours: Monday - Friday 08h30 -17h15

     

    Picture1

     






  • 15.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Tue August 23, 2022 04:00 PM
    That does help Sanne, thanks for the above information. I have added it to the defect report that we have on this issue.

    Best,

    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 16.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed February 22, 2023 09:28 AM

    Hi

    We have started to experience the same issue where we end up with duplicate rows overwriting existing rows. It seems to have happened after copying and pasting and also after editing a cell. We are only working on one data file (or copies of it) at the moment so can't say if it is happening with other files.

    Before I spend a lot of time trying to troubleshoot and reproduce, I thought I would check to see if there is any update on this issue?

    Many thanks
    Ryan



    ------------------------------
    Ryan Bentham
    ------------------------------



  • 17.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed February 22, 2023 10:26 AM

    Hi. No one here is doubting that it happens. The problem is that we have not yet been able to find a replication scenario.



    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 18.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed February 22, 2023 11:16 PM

    Hi Rick,

    Thanks for the update.

    After a lot of mucking about, I have been able to reproduce the issue by copying and pasting (using the keyboard) when a .sav file is encrypted with a password. I am using version 29.0.0.0 (241). Another user mentioned the had the same issue occur when they were typing into the cell (not copy and paste) but I have not been able to reproduce that issue reliably. If I remove the password from the file, I cannot get the issue to occur.

    I can reproduce the issue using these steps, and I have attached a recording showing the issue happening just in case.

    1. Open the attached data file (Test Data.sav) using the password password
    2. Click in to the cell in the third row of the VAR00003 column
    3. NOTE: row three VAR00001 = 10003; row four VAR00001 = 10004;
    4. Type in anything (e.g. Blah) and push Enter
    5. Push the up arrow to select the previous cell
    6. Copy the text using CTRL + C
    7. Push the down arrow to select the next cell
    8. Paste the text using CTRL + P

    ERROR:
    Row four now contains the data from row 1 (e.g., 10001 and 60).

    Hopefully, following these steps will allow you to reproduce the error. We have removed SPSS encryption from this file in the meantime and will monitor if we continue to have this issue.



    ------------------------------
    Ryan Bentham
    ------------------------------

    Attachment(s)

    sav
    Test Data.sav   930 KB 1 version


  • 19.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Thu February 23, 2023 09:47 AM

    Thanks, Ryan.

    We're looking at it right now.



    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 20.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed March 15, 2023 09:56 AM

    Hello, 

    We have been dealing with this frustrating issue since 2020 and came across this thread today. Following the instructions outlined above by Ryan Bentham, we were able to replicate the problem using SPSS v28.0.1.1 on Windows 7 enterprise, 64 bit OS.  We also replicated this with numeric variables.   

    Rick Marcantonio, we appreciate that IBM is looking into this critical problem. 

    Warm regards,



    ------------------------------
    Heather Brittain
    ------------------------------



  • 21.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed March 15, 2023 10:10 AM

    Thank you, Heather.

    One clue is that the data files showing this problem are password-protected. At least, we think that is a necessary condition. Can you tell me, has that been your experience as well?



    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------



  • 22.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed May 10, 2023 10:49 AM

    Hello,

    I have also had this issue for a while. I am using the 26.0.0.0 version on Windows at work. My document is password protected.
    Today the issue occured again:
    1. I created two new variants in between existing variants in variable view, named them and filled the columns by hand (not copy-paste) with data in data view.
    2. I saved my document and noticed the issue had occured. It seemed like the overwritten lines had nothing in common, except the new copies were from the first 6 lines. There may have been 8 overwritten lines in my table of 36 rows, but I'm not sure. I noticed that the new variants I had added had not all been overwritten on the otherwise overwritten rows.
    3. Luckily I already had had this issue for a few times and had backup versions of my document. I was able to copy-paste information (ctrl+c ctrl+v), but in two parts: columns before the new variants and after the new variants.
    4. As I pasted the information before/to the left of the new variant columns, the new data on these new columns was also overwritten by the same rows, that SPSS had copied (eg. I was correcting a row that had been overwritten by row 2 and the new corrrect data in the new columns was then overwritten by the new data on row 2)
    5. When I pasted the latter part, no new issues occured.

    I have also had this issue but the rows SPSS copies move a column or two to the right. As I have columns that are defined strings, dates or numbers, you see how this messes up the table completely and I end up with cells filled with mixed symbols, such as @, ?, &,  and letters or numbers.

    Has a solution already been found? If not, I hope this helps in the process.



    ------------------------------
    Elina Oksanen
    ------------------------------



  • 23.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted Wed May 10, 2023 12:35 PM

    Hi Elina,

    Unfortunately this is a subtle timing-related issue and a solution has not been found yet. The information you supplied here is helpful however and confirms that thus far we have only seen this problem with password-protected files.

    Therefore a work-around for this issue until a fix has been delivered: If you need to perform extensive editing in the data editor with a password-protected .sav file, first save it into a non-protected file, perform the edits, then save back into the original password-encrypted file, making sure to delete the temporary .sav file when finished. I know this is a sub-optimal solution, but hopefully it will enable you to make the edits you need to move ahead with your work.

    Best regards,



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 24.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted 16 days ago

    We also recently started having this issue in SPSS 20.0 Version 29.0.2.0 (20) I had hoped the patch would fix the issue. I do think I got to recreate the issue in a non-password protected file. Before it would only replace the cases with a duplicate random case when we were sorting ascending and descending, but we were adjusting value labels and then noticed that it has created a duplicate. It always starts with duplicating the first case in our data but will randomly pick a case from the others to duplicate.



    ------------------------------
    Katherine Brown
    ------------------------------



  • 25.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted 16 days ago

    @Katherine Brown Sorry you're seeing this problem.

    • May I ask please if you are seeing it with a password-protected file?  
    • Are you using Windows or MacOS?

    I have re-emphasized this defect for examination and fixing in the next release.

    Thanks and regards,



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------



  • 26.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted 11 days ago
    Hello,
     
    I am experiencing the same issue with password-protected data. The cases duplicate randomly when I enter a new variable or sometimes it changes the values of the other variables (such as dates). So far, this occurs with sorted data (based on ID numbers or dates) when I enter values either manually or copy-paste into a new variable. 
     
    I am using Windows 11, 64-bit, and SPSS 29.0. 
     
    I will try to work on non-Encrypthope data as you suggested above. I hope a solution is found soon because this is so frustrating and I don't even want to contemplate the scenario of not realizing this mistake:(
     
    Thank you so much, 


    ------------------------------
    Zeynep Ertekin
    ------------------------------



  • 27.  RE: Issue with SPSS Duplicating and Replacing Lines of Data

    Posted 10 days ago

    Hi @Zeynep Ertekin, sorry you are seeing this issue too. We are working hard to track down the cause and fix this issue, which appears to be happening only when a .sav file is password protected.

    Regards,



    ------------------------------
    Curtis Browning
    SPSS Statistics Architect
    ------------------------------