SPSS Statistics

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

I can't replicate that behavior, Art. In SPSS, I can open the files I copied in the BEGIN END PROGRAM.

There must be something different about the text and XLSX files I'm using, although I've tried several.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Fri February 03, 2023 01:51 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

Can you paste a copy of the error message(s) you're getting?

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Fri February 03, 2023 02:09 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

I can't replicate that behavior, Art. In SPSS, I can open the files I copied in the BEGIN END PROGRAM.

There must be something different about the text and XLSX files I'm using, although I've tried several.

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:51 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

looks like this.  i'm using the paste function to try and open it.  but when i open the document close it, without editing at all it will work.



------------------------------
Art Jack
------------------------------

Original Message:
Sent: Fri February 03, 2023 02:10 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Can you paste a copy of the error message(s) you're getting?

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 02:09 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

I can't replicate that behavior, Art. In SPSS, I can open the files I copied in the BEGIN END PROGRAM.

There must be something different about the text and XLSX files I'm using, although I've tried several.

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:51 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

Original Message:
Sent: 2/3/2023 1:52:00 PM
From: Art Jack
Subject: RE: Python Copies Not Opening in SPSS

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------

This worked thanks.  Sent you an email, everyone have a good weekend. - Art

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Fri February 03, 2023 03:12 PM
From: Jon Peck
Subject: Python Copies Not Opening in SPSS

I don't know what might be causing these intermittent failures, but it might be that the file copying has not completed by the time the SPSS attempt to open the file takes place.  You could test this idea by adding a sleep call of a few seconds following the copy call.  

The shutil.copy doc does not say whether the copying is asynchronous, but there is this note.

Changed in version 3.8: Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently.

It would be easy to believe that the syscalls timing is different from what would have happened in older versions.

On a somewhat related note, I have just finished an overhaul of the SPSSINC PROCESS FILES extension command that makes it much easier to use.  This hasn't been posted to the Extension Hub yet, but if you want to try it out, I can put it on my OneDrive site.  Send me an email (jkpeck@gmail.com) if you want to explore this.  It includes an expository article on usage, which I can also send you before posting it if you are interested.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 2/3/2023 1:52:00 PM
From: Art Jack
Subject: RE: Python Copies Not Opening in SPSS

I'm not even trying to submit open in spss.  Just the copy part block in python which works.  But when i try to open it in SPSS it won't read the CSV and gives me an error if it's an xlsx.  But when i open & shut the flat file run the same code in spss to get the file it will open. 

*make copy to one drive Python block.
BEGIN PROGRAM python3.
import os
import shutil
import glob
import spss
source_folder = r"C:\path\folder//"
destination_folder = r"C:\Data\e23\Copies//"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
end program.
set unicode=off.

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 01:30 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

OK, so you're doing something like this to copy the files:

begin program python.
import os
import shutil
import spss

in_dir="/Users/SPSS/tmp/in"
out_dir="/Users/SPSS/tmp/out"

for file_name in os.listdir(in_dir):
  source = in_dir + "/" + file_name
  destination = out_dir + "/" + file_name
  if os.path.isfile(source):
    shutil.copy(source, destination)
    print('copied', file_name)
##spss.Submit("GET DATA etc") ??
end program.

Except that, after the copy, you are submitting an spss.Submit(cmd) that opens each file with GET DATA, and some of those are failing?​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 01:00 PM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Hi Rick, I have a set of csv's that I'm copying over from one directory to another with Python.  But when i try to open them using the standard get data for xlsx or csv it's giving me an error in spss.  When i open the document manually & close it, not editing it at all, I'm then able to open it in SPSS?

------------------------------
Art Jack

Original Message:
Sent: Fri February 03, 2023 12:08 PM
From: Rick Marcantonio
Subject: Python Copies Not Opening in SPSS

Hi. I'm not sure what you mean.

You are attempting to open a series of CSV files (using a GET DATA command I presume), and some of them are failing?

Where is the error coming from - Python or SPSS?


------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: Fri February 03, 2023 10:52 AM
From: Art Jack
Subject: Python Copies Not Opening in SPSS

Morning, I have a split file set of csv's that i'm copying into another directory with Python.  I'm trying to open some of those files within spss and it's giving me an error.  When i open the copy manually & close it, run the same code it's opening.  Is this an issue &/or fix? - Art

------------------------------
Art Jack
------------------------------