SPSS Statistics

SPSS Statistics

Your hub for statistical analysis, data management, and data documentation. Connect, learn, and share with your peers! 

 View Only

  • 1.  Combining Files

    Posted Mon August 22, 2022 03:18 PM
    I need to combine a weekly archived data set & incorporate it in a SPSS production script that runs daily.  Is there a way to combine files with different names without having to change the production script everyday?  I'm working with SPSS & flat files with no database connection. - Arthur

    ------------------------------
    Art Jack
    ------------------------------

    #SPSSStatistics


  • 2.  RE: Combining Files

    Posted Mon August 22, 2022 03:24 PM
    You could use environment variables. In the OS, define the updated file something like this:

    set UPDATED_FILE=<path/updated file name>

    In your syntax file, define a FILE HANDLE:

    FILE HANDLE updated /NAME="%UPDATED_FILE%".

    Then, in your ADD/MATCH/GET/whatever...

    ADD FILES /FILE=* /FILE=updated /etc.

    If you're on Mac, in the OS you use export, then in your syntax file, it is $UPDATED_FILE.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------

    Original Message


  • 3.  RE: Combining Files

    Posted Mon August 22, 2022 03:36 PM
    I would be a little nervous about the environment variable approach, although it would work, since if you forgot to update the name sometime, the script would still run but mess up the result.

    If you are using production mode, it support a special macro mechanism that would let you specify the file name as a parameter of the production job.  As long as you don't set a default for that, it would be safer.

    --



    Original Message


  • 4.  RE: Combining Files

    Posted Mon August 22, 2022 03:47 PM
    I'm not sure I understand.  I have a script set up in the production facility & it runs a regular spss syntax script I have.  Right now as one of those productions scripts is a python block that copies over the most recent flat file I have for whatever files are used.  But I need to 'add files' a weekly archived data set within it.  This is the format of the archive file, the string date is changed weekly.

    Active 8 15 2022

    ------------------------------
    Art Jack
    ------------------------------

    Original Message


  • 5.  RE: Combining Files

    Posted Mon August 22, 2022 03:57 PM
    In the Production Facility, you can have it prompt you for a file name that will be substituted via macro, so you could update the dated file name without modifying the job syntax.

    If you wanted to use the filename with date approach via the spssaux2 functions, you would have to set it up so that the date/time format matched what is in that code or make your own version that uses the existing date format.



    Runtime values

    Last Updated: 2021-11-09

    Runtime values defined in a production job file and used in a command syntax file simplify tasks such as running the same analysis for different data files or running the same set of commands for different sets of variables. For example, you could define the runtime value @datafile to prompt you for a data file name each time you run a production job that uses the string @datafile in place of a file name in the command syntax file.

    • Runtime value substitution uses the macro facility (DEFINE-!ENDDEFINE) to create string substitution values.
    • Runtime values in command syntax files are ignored if they are enclosed in quotation marks. If the runtime value needs to be quoted, select Quote Value. If the runtime value is only part of a quoted string, you can include the runtime value in a macro with the !UNQUOTE and !EVAL parameters.

    Symbol. The string in the command syntax file that triggers the production job to prompt the user for a value. The symbol name must begin with an @ sign and must conform to variable naming rules.


    --



    Original Message


  • 6.  RE: Combining Files

    Posted Mon August 22, 2022 03:37 PM
    This file has a text string date.  Should i put this as the most recent file or latest file in the updated file name?  
     
    This is the name of the file 'active 8 15 2022.sav'

    set UPDATED_FILE=<path/updated file name>

    ------------------------------
    Art Jack
    ------------------------------

    Original Message


  • 7.  RE: Combining Files

    Posted Mon August 22, 2022 03:42 PM
    I don't think I'm understanding your question, Art. As long as the file name is legal, this should work. As for where and how you refer to the file in syntax, that's entirely up to you.

    ------------------------------
    Rick Marcantonio
    Quality Assurance
    IBM
    ------------------------------

    Original Message


  • 8.  RE: Combining Files

    Posted Mon August 22, 2022 03:43 PM
    There is a function in the spssaux2.py module installed with Statistics that creates a filename with a date stamp.

    def CreateFileNameWDate(basename=None):
        """Create a filename, including path, of the form base_datetime.ext
        where datetime is the current date and time in a file system safe format of
        YYYY-MM-DD_HH-MM.
       
        If the basename is not specified, the filename of the active dataset is used.
        If there is none, ValueError is raised.
        If basename already contains a datetime stamp at the end (but before the extension),
        it is removed."""

    There is a companion function that the find most recent file.
    def FindMostRecentFile(basename):
        """Return the full filename, including path if included in basename,
        of the most recent file matching basename or None if there is no match.
       
        Matching means the filename equals the basename with or without a timestamp.
        A timestamp has the format
        _YYYY-MM-DD_HH-MM
        just before the extension.  

    These two  functions could be incorporated into the production job.  They could be used up front in the job with a little Python block to define the appropriate file handle.

    --



    Original Message


Related Content