SPSS Statistics

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

Jon Peck probably has an extension that allows that, I haven't looked but will not be at all surprised if he does.

Apart from that, you would write a Python program to get all the files in the target directory (with the appropriate extension), filter them according to your filename requirement(s), then either construct an ADD FILES command (if they are SAV files) or concatenate them (if they are syntax files) in Python.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

Original Message:
Sent: 12/7/2022 4:09:00 PM
From: Art Jack
Subject: RE: Combining Files

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------

#SPSSStatistics

I couldn't get this to work. 

I found this online & changed it a little, John i think you had posted it originally.  The records # total matches & seems to work.  Thanks for your help everyone. - Arthur

dataset close all.
NEW FILE.
DATASET NAME DataSet1 WINDOW=FRONT.
BEGIN PROGRAM python.
import spss, glob
cmd = ["ADD FILES"]
for f in glob.glob(r"c:/et/test/A/*Demographics*.sav"):
cmd.append("""/FILE="%s" """ % f)
spss.Submit(cmd)
end program.
execute.

​

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 04:10 PM
From: Jon Peck
Subject: Combining Files

This ignores the current active file.  Try a few experiments.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 4:09:00 PM
From: Art Jack
Subject: RE: Combining Files

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------
#SPSSStatistics

Original Message:
Sent: 12/8/2022 9:31:00 AM
From: Art Jack
Subject: RE: Combining Files

I couldn't get this to work. 

I found this online & changed it a little, John i think you had posted it originally.  The records # total matches & seems to work.  Thanks for your help everyone. - Arthur

dataset close all.
NEW FILE.
DATASET NAME DataSet1 WINDOW=FRONT.
BEGIN PROGRAM python.
import spss, glob
cmd = ["ADD FILES"]
for f in glob.glob(r"c:/et/test/A/*Demographics*.sav"):
cmd.append("""/FILE="%s" """ % f)
spss.Submit(cmd)
end program.
execute.

​

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Wed December 07, 2022 04:10 PM
From: Jon Peck
Subject: Combining Files

This ignores the current active file.  Try a few experiments.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 4:09:00 PM
From: Art Jack
Subject: RE: Combining Files

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------
#SPSSStatistics

I need to iterate over all of the files within a directory open them create a variable based off of the file name/or path & then combine them all.  I will use the python block above to combine them.  But I'm having issues getting SPSS Process files to work.  Usually on the Syntax= portion of the process files script I'm only opening a sav & saving as a csv or xls.  Is the sequence correct in the macro, get file, save below?  - Thanks Art

SPSSINC Script
*process sav to sav.
SPSSINC PROCESS FILES INPUTDATA='C:\et\test\A\Demographics*.sav'
SYNTAX="C:\General Syntax 22-23\ConvertSavToSavFileName.sps" CONTINUEONERROR=YES
VIEWERFILE= "C:\General Syntax 22-23\Split Convert\convert.spv" CLOSEDATA=YES.

*here's the sav to sav sps file.
define !out () !quote(!concat(!unquote(!eval(!JOB_DATADIR)),
"/",
!unquote(!eval(!JOB_DATAFILEROOT)))) !enddefine.
GET FILE=JOB_INPUTFILE.
begin program python.
import spss, spssaux
# if the active file does not have a name, the generated variable will be blank
filename = spssaux.GetDatasetInfo()
spss.Submit("""string file(a64).
compute file="%s".""" % filename)
end program.
SAVE OUTFILE=!out
/COMPRESSED.

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Thu December 08, 2022 10:08 AM
From: Jon Peck
Subject: Combining Files

That would be equivalent, but both should work.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/8/2022 9:31:00 AM
From: Art Jack
Subject: RE: Combining Files

I couldn't get this to work. 

I found this online & changed it a little, John i think you had posted it originally.  The records # total matches & seems to work.  Thanks for your help everyone. - Arthur

dataset close all.
NEW FILE.
DATASET NAME DataSet1 WINDOW=FRONT.
BEGIN PROGRAM python.
import spss, glob
cmd = ["ADD FILES"]
for f in glob.glob(r"c:/et/test/A/*Demographics*.sav"):
cmd.append("""/FILE="%s" """ % f)
spss.Submit(cmd)
end program.
execute.

​

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 04:10 PM
From: Jon Peck
Subject: Combining Files

This ignores the current active file.  Try a few experiments.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 4:09:00 PM
From: Art Jack
Subject: RE: Combining Files

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------
#SPSSStatistics

Original Message:
Sent: 12/9/2022 8:52:00 AM
From: Art Jack
Subject: RE: Combining Files

I need to iterate over all of the files within a directory open them create a variable based off of the file name/or path & then combine them all.  I will use the python block above to combine them.  But I'm having issues getting SPSS Process files to work.  Usually on the Syntax= portion of the process files script I'm only opening a sav & saving as a csv or xls.  Is the sequence correct in the macro, get file, save below?  - Thanks Art

SPSSINC Script
*process sav to sav.
SPSSINC PROCESS FILES INPUTDATA='C:\et\test\A\Demographics*.sav'
SYNTAX="C:\General Syntax 22-23\ConvertSavToSavFileName.sps" CONTINUEONERROR=YES
VIEWERFILE= "C:\General Syntax 22-23\Split Convert\convert.spv" CLOSEDATA=YES.

*here's the sav to sav sps file.
define !out () !quote(!concat(!unquote(!eval(!JOB_DATADIR)),
"/",
!unquote(!eval(!JOB_DATAFILEROOT)))) !enddefine.
GET FILE=JOB_INPUTFILE.
begin program python.
import spss, spssaux
# if the active file does not have a name, the generated variable will be blank
filename = spssaux.GetDatasetInfo()
spss.Submit("""string file(a64).
compute file="%s".""" % filename)
end program.
SAVE OUTFILE=!out
/COMPRESSED.

------------------------------
Art Jack
------------------------------

Original Message:
Sent: Thu December 08, 2022 10:08 AM
From: Jon Peck
Subject: Combining Files

That would be equivalent, but both should work.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/8/2022 9:31:00 AM
From: Art Jack
Subject: RE: Combining Files

I couldn't get this to work. 

I found this online & changed it a little, John i think you had posted it originally.  The records # total matches & seems to work.  Thanks for your help everyone. - Arthur

dataset close all.
NEW FILE.
DATASET NAME DataSet1 WINDOW=FRONT.
BEGIN PROGRAM python.
import spss, glob
cmd = ["ADD FILES"]
for f in glob.glob(r"c:/et/test/A/*Demographics*.sav"):
cmd.append("""/FILE="%s" """ % f)
spss.Submit(cmd)
end program.
execute.

​

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 04:10 PM
From: Jon Peck
Subject: Combining Files

This ignores the current active file.  Try a few experiments.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 4:09:00 PM
From: Art Jack
Subject: RE: Combining Files

Does this need to be just like this, or should i alter aspects of it?

cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n""" for fi in thefiles)

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:46 PM
From: Jon Peck
Subject: Combining Files

Here's an example.

begin program python3.
import glob, spss
thefiles = glob.glob(r"c:/temp/*demographics*.sav")
cmd = "ADD FILES FILE=" + "\n/FILE=".join(f"""/FILE="fi"\n"""  for fi in thefiles)
spss.Submit(cmd)
end program

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 12/7/2022 3:31:00 PM
From: Art Jack
Subject: RE: Combining Files

Thanks all.  Could you send me a short example? - Art

------------------------------
Art Jack

Original Message:
Sent: Wed December 07, 2022 03:21 PM
From: Jon Peck
Subject: Combining Files

The most general way to do that would be with SPSSINC PROCESS FILES, where you can specify the file names to process using a regular expression.  But you could, alternatively, just use the Python glob.glob function to get all those names and concatenate them in an ADD FILES command.

------------------------------
Jon Peck

Original Message:
Sent: Wed December 07, 2022 03:11 PM
From: Art Jack
Subject: Combining Files

Using SPSS is there a way to append all files in a directory with common names.  For example I need to append all files that have demographics in the file title? - Thanks Arthur

------------------------------
Art Jack
------------------------------
#SPSSStatistics