Variables created as temporary before MATRIX do not automatically dissapear
if the MATRIX creates a new dataset. SPSS Statistics 28.0.1.1 and earlier (e.g., 26).

.
dataset name Data.
temporary.
compute x= 1. /*Some temporary variable
matrix.
save 0 /outfile= *. /*MATRIX creates a new active dataset
end matrix.
dataset name Data2.

*Variable X is still visible in the Data dataset (albeit it is without values, a phantom).
*And it is the error, or at least a major, annoying inconvenience.

*One has to return once again to the old dataset to run "execute"
*so that X at last an in fact disappears:.
dataset activate Data.
execute.

(I've reported the issue from inside the SPSS (the "Report an Issues" menu item). Don't know if it was accepted and registered.



------------------------------
Kirill Orlov
------------------------------
#SPSSStatistics

Jon, I understand it. I am against the thing that the temporary variable is still seen, as a phantom, in the initial dataset. A user comes to observe it and waht they see? They see the column X that must not be found anymore in the dataset. Imagine also that that temporary variable X had been created not consciously by the user, but by some syntax they are using, and so they face the "unexpected" extraneous column X. It may scare.

I think that the best solution or work-around from the side of the Statistics would be to hide any temporarily created variables from the GUI whenever the processor is about to enter MATRIX session. The MATRIX session can use (read) the temporary variable data if needed, but the column should not be visible anymore as long as MATRIX begins to act.

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 11:30 AM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

Strictly speaking, TEMPORARY is working as documented.  According to the CSR,

"Temporary transformations apply to the next command that reads the data. Once the data are read, the temporary transformations are no longer in effect."

Your MATRIX code does not pass the data.  But why does this matter?  The temporary variable would disappear when the main dataset data are passed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 11:47:00 AM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Jon, I understand it. I am against the thing that the temporary variable is still seen, as a phantom, in the initial dataset. A user comes to observe it and waht they see? They see the column X that must not be found anymore in the dataset. Imagine also that that temporary variable X had been created not consciously by the user, but by some syntax they are using, and so they face the "unexpected" extraneous column X. It may scare.

I think that the best solution or work-around from the side of the Statistics would be to hide any temporarily created variables from the GUI whenever the processor is about to enter MATRIX session. The MATRIX session can use (read) the temporary variable data if needed, but the column should not be visible anymore as long as MATRIX begins to act.

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: 3/2/2022 11:47:00 AM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Jon, I understand it. I am against the thing that the temporary variable is still seen, as a phantom, in the initial dataset. A user comes to observe it and waht they see? They see the column X that must not be found anymore in the dataset. Imagine also that that temporary variable X had been created not consciously by the user, but by some syntax they are using, and so they face the "unexpected" extraneous column X. It may scare.

I think that the best solution or work-around from the side of the Statistics would be to hide any temporarily created variables from the GUI whenever the processor is about to enter MATRIX session. The MATRIX session can use (read) the temporary variable data if needed, but the column should not be visible anymore as long as MATRIX begins to act.

------------------------------
Kirill Orlov
------------------------------

//Your MATRIX code does not pass the data//
Jon, MATRIX runs pending transforms.



------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 11:30 AM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

Strictly speaking, TEMPORARY is working as documented.  According to the CSR,

"Temporary transformations apply to the next command that reads the data. Once the data are read, the temporary transformations are no longer in effect."

Your MATRIX code does not pass the data.  But why does this matter?  The temporary variable would disappear when the main dataset data are passed.

--

Jon K Peck
jkpeck@gmail.com

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: 3/2/2022 12:18:00 PM
From: Rick Marcantonio
Subject: RE: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov
------------------------------

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Rick, you have not so far convinced me that the being discussed behaviour is all right. You say:
// MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect. //

If TEMPORARY were intended to remain in effect, then this

dataset name Data.
temporary.
compute x= 1.
matrix.
end matrix.
frequencies x.

... or this

dataset name Data.
dataset declare Data2.
temporary.
compute x= 1.
matrix.
save 0 /outfile= 'Data2'.
end matrix.
dataset activate Data.
frequencies x.

... would work without saying. But it will not. The X variable is not available after MATRIX (and should not be available, I presume). And I conclude - MATRIX session (whatever it is doing, saving or not saving any data) should clear any remnant of the temporary X variable. Like a procedure reading data will do it. TEMPORARY should not "remain in effect" anyhow.

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 03:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------


Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: 3/2/2022 7:16:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, you have not so far convinced me that the being discussed behaviour is all right. You say:
// MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect. //

If TEMPORARY were intended to remain in effect, then this

dataset name Data.
temporary.
compute x= 1.
matrix.
end matrix.
frequencies x.

... or this

dataset name Data.
dataset declare Data2.
temporary.
compute x= 1.
matrix.
save 0 /outfile= 'Data2'.
end matrix.
dataset activate Data.
frequencies x.

... would work without saying. But it will not. The X variable is not available after MATRIX (and should not be available, I presume). And I conclude - MATRIX session (whatever it is doing, saving or not saving any data) should clear any remnant of the temporary X variable. Like a procedure reading data will do it. TEMPORARY should not "remain in effect" anyhow.

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 03:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------


Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov

// The bug here is that the empty MATRIX procedure did pass the data //
Jon, but this...

compute x= 1.
matrix.
end matrix.

executes the compute, and does so on all SPSS versions.
Do you want to say it is a bug? or that this is unjustified and should be otherwise? Or am I misunderstanding you?

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 07:52 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

The bug here is that the empty MATRIX procedure did pass the data.  It should not have triggered a data pass.  The procedure does not determine the data pass.  The central system controls that but procedures that request data would cause the central system to handle that.  MATRIX would not even know that a variable is temporary.

But there is another rule that applies here and is confusing things.  You cannot switch datasets when there are pending transformations, so the system has to force a data pass before switching.  That means that the second example, which uses a different dataset, does require a data pass (unknown to MATRIX), although the first dataset is active after MATRIX completes.

There may also be an issue with datasets vs files, because for a long time MATRIX did not support datasets, much to everyone's annoyance.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 7:16:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, you have not so far convinced me that the being discussed behaviour is all right. You say:
// MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect. //

If TEMPORARY were intended to remain in effect, then this

dataset name Data.
temporary.
compute x= 1.
matrix.
end matrix.
frequencies x.

... or this

dataset name Data.
dataset declare Data2.
temporary.
compute x= 1.
matrix.
save 0 /outfile= 'Data2'.
end matrix.
dataset activate Data.
frequencies x.

... would work without saying. But it will not. The X variable is not available after MATRIX (and should not be available, I presume). And I conclude - MATRIX session (whatever it is doing, saving or not saving any data) should clear any remnant of the temporary X variable. Like a procedure reading data will do it. TEMPORARY should not "remain in effect" anyhow.

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 03:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM


Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Original Message:
Sent: 3/2/2022 9:03:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

// The bug here is that the empty MATRIX procedure did pass the data //
Jon, but this...

compute x= 1.
matrix.
end matrix.

executes the compute, and does so on all SPSS versions.
Do you want to say it is a bug? or that this is unjustified and should be otherwise? Or am I misunderstanding you?

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 07:52 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

The bug here is that the empty MATRIX procedure did pass the data.  It should not have triggered a data pass.  The procedure does not determine the data pass.  The central system controls that but procedures that request data would cause the central system to handle that.  MATRIX would not even know that a variable is temporary.

But there is another rule that applies here and is confusing things.  You cannot switch datasets when there are pending transformations, so the system has to force a data pass before switching.  That means that the second example, which uses a different dataset, does require a data pass (unknown to MATRIX), although the first dataset is active after MATRIX completes.

There may also be an issue with datasets vs files, because for a long time MATRIX did not support datasets, much to everyone's annoyance.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 7:16:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, you have not so far convinced me that the being discussed behaviour is all right. You say:
// MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect. //

If TEMPORARY were intended to remain in effect, then this

dataset name Data.
temporary.
compute x= 1.
matrix.
end matrix.
frequencies x.

... or this

dataset name Data.
dataset declare Data2.
temporary.
compute x= 1.
matrix.
save 0 /outfile= 'Data2'.
end matrix.
dataset activate Data.
frequencies x.

... would work without saying. But it will not. The X variable is not available after MATRIX (and should not be available, I presume). And I conclude - MATRIX session (whatever it is doing, saving or not saving any data) should clear any remnant of the temporary X variable. Like a procedure reading data will do it. TEMPORARY should not "remain in effect" anyhow.

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 03:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM


Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov

Jon, no. I think TEMPORARY can be useful and be used before MATRIX only when the latter somehow capitalizes on it, for example, reads the temporary variable created or temporary filter erected:

temporary.
compute x= 1.
filter by y.
matrix.
get x /var= x.
....
end matrix.

Clearly, TEMPORARY is applied prior MATRIX somehow for the sake of the matrix session.

My earlier example with empty MATRIX, such as

temporary.
compute x= 1.
matrix.
end matrix.

is just a toy example. I never meant that the compute command in this instance could be expected (or wanted) to be carried out after the matrix session by some subsequent procedure or by EXECUTE. No, not that. The compute should be carried out, or launched, by the MATRIX. (And it is, and always was so, in SPSS.)

------------------------------
Kirill Orlov
------------------------------

Original Message:
Sent: Wed March 02, 2022 09:14 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

It seems to me that that, assuming that you meant to include a TEMPORARY, should not pass the data IMO.  It would, of course, do the compute, which will piggyback on the next data pass.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 9:03:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

// The bug here is that the empty MATRIX procedure did pass the data //
Jon, but this...

compute x= 1.
matrix.
end matrix.

executes the compute, and does so on all SPSS versions.
Do you want to say it is a bug? or that this is unjustified and should be otherwise? Or am I misunderstanding you?

------------------------------
Kirill Orlov
------------------------------


Original Message:
Sent: Wed March 02, 2022 07:52 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

The bug here is that the empty MATRIX procedure did pass the data.  It should not have triggered a data pass.  The procedure does not determine the data pass.  The central system controls that but procedures that request data would cause the central system to handle that.  MATRIX would not even know that a variable is temporary.

But there is another rule that applies here and is confusing things.  You cannot switch datasets when there are pending transformations, so the system has to force a data pass before switching.  That means that the second example, which uses a different dataset, does require a data pass (unknown to MATRIX), although the first dataset is active after MATRIX completes.

There may also be an issue with datasets vs files, because for a long time MATRIX did not support datasets, much to everyone's annoyance.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 7:16:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, you have not so far convinced me that the being discussed behaviour is all right. You say:
// MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect. //

If TEMPORARY were intended to remain in effect, then this

dataset name Data.
temporary.
compute x= 1.
matrix.
end matrix.
frequencies x.

... or this

dataset name Data.
dataset declare Data2.
temporary.
compute x= 1.
matrix.
save 0 /outfile= 'Data2'.
end matrix.
dataset activate Data.
frequencies x.

... would work without saying. But it will not. The X variable is not available after MATRIX (and should not be available, I presume). And I conclude - MATRIX session (whatever it is doing, saving or not saving any data) should clear any remnant of the temporary X variable. Like a procedure reading data will do it. TEMPORARY should not "remain in effect" anyhow.

------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 03:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

With all due respect, I disagree that this behavior is a bug. You may see a design issue here, but the program is working the way it is designed.

The MATRIX documentation is clear, as I posted earlier:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

MATRIX does not pass the data in the active dataset, so TEMPORARY remains in effect.

Perhaps the user should define the temporary variable when they're ready to use it (i.e., once they return to the first dataset), or  define it before MATRIX and use a scratch variable, as I mentioned before.

------------------------------
Rick Marcantonio
Quality Assurance
IBM


Original Message:
Sent: Wed March 02, 2022 03:08 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

// I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run. If you are really worried about this, you could put a variable label on the temporary variable pointing out that it will be removed. //

Sure, it will disappear after EXECUTE or some procedure. But, if one wants it to be proper and not frighten a user, it should disappear from the GUI earlier (when MATRIX starts or MATRIX ends). MATRIX itself acts as a procedure in the respect that it runs pending transformations; so SPSS should kill temporary variables automatically, leaving no signs of their presence after any MATRIX session.

What we observe is not any good or normal anyway.


------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 02:04 PM
From: Jon Peck
Subject: Temporary variables before a (saving data) MATRIX

I do see the temporary variable now in the gui, but it does disappear as soon as a data-passing procedure is run.  If you are really worried about this, you could put a variable label on the temporary variable  pointing out that it will be removed.

--

Jon K Peck
jkpeck@gmail.com

Original Message:
Sent: 3/2/2022 12:30:00 PM
From: Kirill Orlov
Subject: RE: Temporary variables before a (saving data) MATRIX

Rick, I am not discussing scratch variables. I am discussing temporary variables. It is clear to me that there is a problem. Actually, the problem has not been solved in all SPSS versions of the last, say 5 or 7 years.

Once again. The problem (it is a real inconvenience, perhaps should be considered a real bug) is that a column of a variable temporarily created before MATRIX is still present visually in the old dataset when the MATRIX has created a new dataset. MATRIX runs pending transforms (which is right), and so any temporary variables (or their visual phantoms or remnants) must not be observed anymore in the original dataset, even though MATRIX has changed the focus of activity onto the new dataset it has just created and so SPSS has been detached from the original dataset.

------------------------------
Kirill Orlov


Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Rick Marcantonio
Subject: Temporary variables before a (saving data) MATRIX

In the MATRIX documentation, I see:

"As discussed below, a matrix program is for the most part independent of the rest of the session, although it can read and write IBM SPSS Statistics data files, including the active dataset."

I may be missing in doc where it says that transformations from the active dataset are passed.

The user could simply use a temporary variable, right?
COMPUTE #x=1.

This would be "invisible" to the front-end but still operative in the backend until the data are passed.

Sorry if I have missed something in the discussion so far.​

------------------------------
Rick Marcantonio
Quality Assurance
IBM

Another and very bad thing, now seen only in version 28 (28.0.1.1) is that after the block

dataset name Data.
temporary.
compute x= 1. /*Some temporary variable
matrix.
save 0 /outfile= *. /*MATRIX creates a new active dataset
end matrix.
dataset name Data2.

Transform > Run Pending Transforms is not active when I click back on the Data dataset (in order to run Pending transformds, i.e., execute, and thus to get rid of the X column still hanging in the dataset's GUI). Run Pending Transforms menu item becomes inactive as I click on the Data dataset to make it active.

------------------------------
Kirill Orlov
------------------------------

This I am filing as a bug.

------------------------------
Rick Marcantonio
Quality Assurance
IBM
------------------------------

Original Message:
Sent: Wed March 02, 2022 12:18 PM
From: Kirill Orlov
Subject: Temporary variables before a (saving data) MATRIX

Another and very bad thing, now seen only in version 28 (28.0.1.1) is that after the block

dataset name Data.
temporary.
compute x= 1. /*Some temporary variable
matrix.
save 0 /outfile= *. /*MATRIX creates a new active dataset
end matrix.
dataset name Data2.

Transform > Run Pending Transforms is not active when I click back on the Data dataset (in order to run Pending transformds, i.e., execute, and thus to get rid of the X column still hanging in the dataset's GUI). Run Pending Transforms menu item becomes inactive as I click on the Data dataset to make it active.

------------------------------
Kirill Orlov
------------------------------