Maximo

 View Only
Expand all | Collapse all

Saved query gets deleted when user entitlement is removed from MAS core.

  • 1.  Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Thu October 03, 2024 08:38 AM

    Hello Community,

    We have noticed that when users entitlement in MAS 8.11 core is removed for a Manage user then all the saved queries (public/private) where this user is the owner, gets deleted from QUERY table. Have you noticed this in your MAS environment?

    For e.g. Mr. X has been working for past several years and he created so many good saved queries over the years, which are shared across teams. Now Mr. X is moving to a different role in the organization and he doesn't need access to Maximo anymore. We have to remove his entitlements of Manage using MAS Administration console. We removed his entitlement (set to No entitlement), user sync is done. 

    We notice that userid of Mr. X is set to DELETED but it is present in the MAXUSER table but all the saved queries are deleted from the QUERY table.

    In earlier versions (Maximo 7.X) saved queries were available even after users are deactivated.

    Can you think of any good reason behind this new design? 



    ------------------------------
    Chandan Singh
    ------------------------------


  • 2.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Fri October 04, 2024 12:42 AM
    Edited by Johann Rumpl Fri October 04, 2024 12:42 AM

    Hi Chandan,

    I can't think of a good reason for this design as well. Perhaps to reduce the number of available queries for a specific user to increase the overview. Something we've addressed with an Add-On (EAM QueryManagement - https://youtu.be/qYKhunpETaI?si=pK92LM03NNJ-gHxE).

    But we observed that Maximo 7.6 does show the same behaviour - if you delete a user the queries are deleted as well. A very bad thing if someone else used such queries in start centers..... But, it doesn't happen if you deactivate a user, which should be the preferred approach in this case.

    I'm wondering if someone else has a clue why it has been designed this way. To be honest I don't even get the idea to change the status to DELETED if you execute the Delete Action. That is different to any other Maximo Application, where records are deleted from the DB if thy pass several checks. Very confusing - at least for me.....

    cheers
    Johann



    ------------------------------
    Johann Rumpl
    CEO / Senior Consultant
    EAM Swiss GmbH
    Steinmaur
    Switzerland
    ------------------------------



  • 3.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Fri October 04, 2024 02:15 AM
    Edited by Andrzej Więcław Fri October 04, 2024 02:19 AM

    Hi Chandan and Johann,

    things seem to work a bit different, at least in MAS9. I know that it doesn't help for customers running 8.11 but I believe it's good to be what's already behind the corner.

    When user gets deactivated then hist entitlement is automatically removed but that doesn't trigger MAXUSER.STATUS to be set to DELETED any longer. User record in Manage is marked as DELETED only if you actually delete the account from MAS Core. 
    BTW: For auditing purposes that doesn't remove user data from MongoDB, just like user record is not actually removed from Manage DB. In both cases records are simply not available in the UI and through REST API calls.

    Having said, I would assume (I didn't check it though myself) that if user gets only deactivated in MAS9 when entitlement is revoked then saved queries are not being removed.

    For more details you can review my LinkedIn article Core vs Manage User Statuses in MAS 9.0.



    ------------------------------
    Andrzej Więcław
    Maximo Technical Consultant
    AFRY
    Wrocław, Poland
    ------------------------------



  • 4.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Mon October 07, 2024 07:59 AM

    While this behaviour is not ideal for your situation there is a good reason for it.

    If an organisation has a high turnover of people then the number of queries can grow quickly particularly if there aren't standard queries that are being used by multiple people.

    It is good to share queries but there are dangers particularly if the person building the query doesn't know how to optimise it. I have seen a lot of really inefficient user generated queries that tied up database resources and cause problems - particularly on start centres.

    IMO The best way forward is to identify the best queries and then change the owner to be one of the system administrators, they can then check the SQL and fine tune it to use the relevant indexes if necessary e.g. by removing wildcard references where appropriate. - http://www.linkedin.com/pulse/additional-information-maximos-wildcard-search-type-mark-robbins

    If you want to understand if a user generated piece of SQL is affecting your start centr then look at this article - https://www.linkedin.com/pulse/using-sql-tracing-start-centre-performance-problems-mark-robbins/



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 5.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 10:53 AM

    Team,

    All good points. Challenge I have is that since Maximo uses a non-normalized DB with all referential integrity governed by the MBOs, since when is it a good idea to delete records that have dependencies. (i.e., the public flag?) If perhaps @Steve Hauptman coudl shed some light on this it might help.  Also as pointed out by @Johann Rumpl and @Mark Robbins "change Owner to be one of the system administrators".  However Maximo has never had this feature (although it has been requested for years.) Perhaps an opportunity for BP's? 

    At any rate in my view if the 8.11 version deletes the user queries even when flagged as "public", the risk is too great to have this NOT be a bug.  Deletion of a public query which may well in fact be used is a huge risk for system crash. The easier check is not to delete it when the user is being deleted. And we need to add a way to manage better queries in general.



    ------------------------------
    Bradley K. Downing , MBA
    Senior Solution Architect
    IBM
    Bakersfield
    ------------------------------



  • 6.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 11:23 AM

    "However Maximo has never had this feature (although it has been requested for years.) Perhaps an opportunity for BP's? "

    There are some "opportunities" that should be resolved by IBM.

    The risk of leaving it to BPs is that every BP / customer combination could come up with a subtly different soluton.

    So potential differences could include:

    • Generating an audit record, or not, when the owner is changed - my preference is for a system property to control this - default no
    • Differences in the front-end e.g. can an administrator manually change this
    • Allowing restrictions on which queries could be modified e.g. one BP may allow public queries to be deleted and another may not - my preference is a system properties that allows non public queries to be deleted.  A separate property should control if public queries are changed to a designated user e.g. MAXADMIN. The designated userid should be controlled via a different system property
    • Automatically making the change when the user account is disabled - this could be controlled via a system property - default is yes

    I'm sure that others could think of other variations or argue for different settings.

    IBM are in the unique position of being able to provide a single solution for everyone.

    Experience, requests, have shown that this is a problem and it should be a relatively simple change to implement.



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 7.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 01:07 PM

    Mark,

    I do not disagree with you at all. Yet as you well know that when an idea is considered and not implemented there is usually a good reason for this.  In this case the lack of movement on "Query Management" (just a name for now...) indicates something yes?  I do not speak for IBM nor the development process.  All I can provide is an opinion.  The use of the AHA! portal is a tool that the community can use to lobby for change.  Please feel free to advocate on the portal for the "relatively simple change to implement".



    ------------------------------
    Bradley K. Downing , MBA
    Senior Solution Architect
    IBM
    Bakersfield
    ------------------------------



  • 8.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 11:33 AM

    Hi  @Bradley Downing

    Could you please expand on these comments:

    " the risk is too great to have this NOT be a bug.  

    Deletion of a public query which may well in fact be used is a huge risk for system crash"

    There isn't enough infomation here to raise a PMR. I have raised PMRs to fix memory leaks and I can't think of a situation where I could pinpoint it back to the loss of a public query.

    Could you please provide steps so that a PMR could be raised or pointers about what to look for?

    I'm happy to put the work in to raise a PMR to ensure that any system threatening bug is investigated.

    I'm happy to discuss this offline if required. Steve H / others can share my email address if you haven't got it



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 9.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 01:17 PM

    My view:

    Public queries (since they are public) will have an unknown consequence.  To simply delete them is an error in programming judgement in my view.  This is my opinion.  I have, in the course of my career, advocated for issues I and my teammates have found, to be be flagged as "defects" successfully on many occasions as well as you have I would imagine (since you are an IBM Champion.)

    When a start-center (relying on a public query) has no public query the start center will simply  fail to render yes? Perhaps my words "system crash" are a bit harsh and not correct.  OK. no big deal. This does not mean there is not data there to raise a Case Ticket.  To be able to pinpoint the lack of the query due to deletion of a user, and capabilities reliant upon base functionality now no longer work due to a programming issue, is certainly enough grounds to raise a Case (PMR?  That system ended about six years ago now I think?).  I would like to believe that you would be able to gather enough evidence to do so.



    ------------------------------
    Bradley K. Downing , MBA
    Senior Solution Architect
    IBM
    Bakersfield
    ------------------------------



  • 10.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Tue October 08, 2024 11:11 PM

    Thank you everyone, glad that this triggered a thoughtful discussion.

    I have a PMR logged for this issue but as always its a challenge to get through the first level and convince them this is a bug. I have been getting repeated response that not deleting the query will be a security issue, without explaining it further. I am puzzled how this could cause a security issue, can you think of how? Here is the latest response from support.

    In the Maximo Application Suite (MAS), the decision to prevent the saving of queries for deactivated users was primarily made for security reasons. In earlier versions, saved queries often remained in the database even after a user was deactivated, which posed potential security risks.

    Currently, if a user is deactivated, any queries they created(Public/Private) cannot be saved. Simply put, a saved query must have an active owner. If the creator is inactive, the query cannot be kept.

    I will try once again quoting the public query deletion case, let's see if they consider it a bug.



    ------------------------------
    Chandan Singh
    ------------------------------



  • 11.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Wed October 09, 2024 07:24 AM

    This is an interesting discussion with a lot of angles.  At a minimum, I would think the decision to delete queries should be controlled by a system property so the client can decide if they consider a security risk or not.  I do think there is some merit to it; we are always encouraging clients during upgrades to use that as an opportunity to clean up queries and QBR reports that were created by users that are no longer with the company.  I like the idea of having an opportunity to automate that process for them.

    If a more involved solution is put in place for query management, it should ultimately be developed by IBM.  I know we, and many other partners, have built solutions and recommended processes for query management, and they are likely all just a bit different.  The idea of a standard solution with some properties to manage automation is a good one.  Is there an idea on the portal already that we can all find and vote for?

    Amy



    ------------------------------
    Amy Tatum
    Technical Director
    Starboard Consulting
    Longwood FL
    407-392-3177
    ------------------------------



  • 12.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Wed October 09, 2024 01:46 PM

    Ideas already raised

    • Allow Updating of Owner Field For Queries (Link, 32 votes since June 2013)
    • To allow Maximo System Administrators the ability to modify the Owner for a Query (Link, 88 votes since Feb 2015)

    There are a bunch of ideas for Saved Queries, but these two are directly related to this bug / discussion.



    ------------------------------
    Blessings,
    Jason Uppenborn
    Sr. Technical Maximo Consultant
    Cohesive
    ------------------------------



  • 13.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Thu October 10, 2024 04:15 AM

    Going back to the original question,  it should be raised with IBM Support because that is our way of getting through to the Developers. But it is highly likely to come back as - as per design, please raise an Idea. In my opinion it is a defect if there has been a change between versions that is not switchable using a System Property, it should be a defect as it effects existing clients.

    Did anyone try this system property mxe.deletepersondata Delete person data on user delete. The QUERY.OWNER field is defined SameAs PERSON.PERSONID.

    Jason found the two of the several Ideas. But while you are in the Ideas portal please vote for this one. https://ideas.ibm.com/ideas/MASM-I-1198 Operational Dashboard - Administrator to manage/view/edit dashboards where they are not the owner. We have the same problem with Operational Dashboards.

    A user can also personalise a Work Queue and we will have the same problem there too when someone leaves, unless the automatic deletion covers work queues as well.

    Sorry to jump in on your response Amy but I think as a group of IBM Champions we should have some collective influence over what Ideas are prioritised by IBM so that some of their budget is spent on closing out issues like this which have existed for years. Hope to see you at TechXChange.



    ------------------------------
    Andrew Jeffery
    Maximo SME
    ZNAPZ b.v
    Barnstaple
    +44 (0)777 1847873
    ------------------------------



  • 14.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Wed October 09, 2024 04:26 AM

    In my view this is 100% a defect. Public queries should not be deleted without warning.

    They can be underpinning a range of Maximo configurations - most commonly start centers but also other areas of MAS such as Maximo Health scoring groups.

    This action wont take place on a test system under change control either - the user is likely to be deactivated immediatley in production as part of off-boarding and any consequences will have operational impact.

    If there is legitimate security concern in retaining queries (I can't think of one), then the system should warn of the existance of public queries that are linked to other configurations before allowing the deactivation.

    Regards

    Dom



    ------------------------------
    Dominic Bramley
    ------------------------------



  • 15.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Wed October 09, 2024 04:58 AM

    Thanks @Dominic Bramley for adding additional reasons.

    @Chandan Singh - Security tends to be a trump card in discussions like this, it is very difficult to overcome.

    IMO the best way forward would be to:

    • highlight the impact on other components
      • ideally providing actual logs / errors generated at the time
      • this may be slightly more pervasive because IBM are encouraging users to use features like Maximo Health
    • accept that this functionality isn't likely to be changed in the near future
    • raise an aha idea to highlight where these queries are being used elsewhere
      • there is probably a new feature in the queries code so that the warning is triggered whenever someone attempts to delete a public query that is still being used in a different component



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 16.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Thu October 10, 2024 08:04 AM

    I have discussed this with people in the development team and I have put together a response which includes a partial workaround... there are still several factors that would need to be considered.

    The community system has flagged the response for review probably because it is lengthy and contains several links to technotes.

    Hopefully the response will be published soon otherwise I wll publish it on my blog

    On  a related note, please consider voting for this idea - https://ideas.ibm.com/search?query=MASM-I-1233

    Implement public query validation and approval process

    This would:

    • Ease the management of queries by allowing administrators to check for problems before a query is made public e.g. looking for things that will make the query inefficient
    • Changing the owner of the query e.g. to maxadmin - this would remove the risk of an important query being lost when a user is deleted


    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 17.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Thu October 10, 2024 10:12 AM

    My response with the workaround seems to have been delayed in the review process.

    I have posted the same information on my blog. if you want to read it quickly then go to the link in my signature and follow the first link by this text:

    Looking for my post about the MAS 9 Deleting users? click on this link (link is on the linkedin page)



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 18.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Fri October 11, 2024 03:12 AM

    Thank you everyone, many interesting points. Hopefully IBM would consider this need and several solution ideas and someday provide a solution for this.

    My discussion with support has concluded and the conclusion is "This is not a bug, it is working as designed". I know even if I push further it won't really make a difference. For now our objective is to retain those queries and an easy way out I can think of is, create a parallel table CQUERY which will add and update records as and when it happens in the QUERY table, except delete; using automation script.



    ------------------------------
    Chandan Singh
    ------------------------------



  • 19.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Fri October 11, 2024 02:29 PM

    All,

    I have discussed this with IBM Development and there are several conclusions. I have mixed the IBM conclusions with some of my thoughts on how to progress this. Please take the time to read it all as it covers several different related issues.

    Deletion process
    ==
    How does the current delete process work?
    --
    The code to delete the queries associated with a user is associated with the MAXUSER object.
    The code uses the "QUERY" relationship to identify user records that are associated with QUERY records.
    the current clause is:
    "owner = :userid"

    Short term workaround - orphan records left in the database:
    --
    Redefine the QUERY relationship so that it doesn't delete these entries e.g. change the clause from "owner = :userid" to:"1=2"
    This change will prevent ALL deletions because the relationship will return no rows so it will leave the queries there.

    Medium term workaround - orphan records left in the database:
    --
    A more intelligent solution would be to only delete rows that we don't want to keep e.g. delete the private queries.

    in that case the change would be to something that checks the locations that the queies could be being used.
    "owner = :userid and  clausename not exists in all the tables where it is important that the query is kept e.g. start centres"

    Orphan records

    ==

    Either workaround would mean that there are references in the database to queries that don't have an owner

    Tables like LAYOUT & SCTEMPLATE.
    LAYOUT is easy to check because the values are stored in QUERYOWNER AND QUERYCLAUSENAME.
    SCTEMPLATE is harder because the values are stored as part of an XML block

    I suspect that there are other tables with similar references but I haven't made time to identify them. Technote ref1 hints at the work involved in checking for other references.

    If you do want to modify the relationship then these instructions from technote 256057 should be helpful:

    Gow to update the owner in the query table and the LAYOUT table that is used for start centres. I have reproduced the critical steps here but you should read the whole technote.

    2) If you want to change the owner of query to another user, please follow the steps below.

    - Stop your Maximo Server.
    - Run the queries below.

    XXX : your query condition
    YYY : your query clause name

    UPDATE QUERY SET OWNER = 'XXXX' WHERE CLAUSENAME = 'YYY';
    UPDATE LAYOUT SET QUERYOWNER = 'XXXX' WHERE QUERYCLAUSENAME = 'YYY';
    COMMIT;

    - Restart your Maximo server


    256057 - https://www.ibm.com/support/pages/cannot-edit-where-clause-saved-query

    ref1 - https://www.ibm.com/support/pages/updating-or-changing-ownership-existing-query-maximo

    ==

    Longer term future
    --
    The Development team are now definitely aware of this issue and its importance to the people in this thread and the potential impact on users looking to adopt components that rely on these queries.

    I expect that there to be additional discussions about a longer term solution and I hope to be involved in those discussions. Changes tend to be related back to ideas. IF changes are decided upon then I will ask for the related aha idea number so I/others can update it with any additional thoughts.
    In the meantime I am happy to be a conduit for thoughts / ideas while the discussion is at an informal stage.

    Whole public query development process

    ==

    There is a potential discussion about how queries should be developed / managed in general e.g. should users be allowed to create public queries at all?

    A better process may be to create queries that an administrator can then "adopt" , tune and deploy.

    I have seen lots of user developed public queries that are inefficient and where 5 minutes of an administrators time could have improved them. In an ideal world the administrator could use an automated tool to check the SQL and suggest changes.

    In the current situation administrators need to make do with more primitative solutions... If administrators are interested in understanding how to work more efficiently in this area then please reach out to me...  I have some useful tools that could help.


    Note about the deletion functionality
    ==
    I was gently reminded that the general policy of deleting the old queries has been in place for 10 years... presumably without people complaining...

    Personally I used to advise Maximo users to never delete a user unless the person had actually passed away... I saw several cases where people retired / left and then returned... deleting the user meant a nasty data fix... the change to involve deleting QUERY records came after I stopped administering user accounts

    The problem has achieved a higher profile because it is now implemented when a user is made inactive rather than being deleted.

    Thanks

    ==

    Thanks to the various IBMers who responded to my note and provided information that has been used here.

    Thanks to the non-IBMers who provided input publicly and privately.



    ------------------------------
    Mark Robbins
    Support Lead/Technical Design Authority / IBM Champion 2017 - 2023
    Cohesive (previously Vetasi Limited)
    https://www.linkedin.com/pulse/maximo-support-advice-from-non-ibm-engineer-article-mark-robbins/
    ------------------------------



  • 20.  RE: Saved query gets deleted when user entitlement is removed from MAS core.

    Posted Mon October 14, 2024 02:38 PM

    All,

    I'm late for this discussion, however, this may be a good chance to request IBM to overhaul treatment of queries in Maximo in general.

    We do have an application that we offer our customers that tries to go around several of the pain points that go with the original design of the QUERY object in Maximo.

    Especially the decision of making the OWNER attribute part of the business primary key, that was probably because they wanted different users to use the same CLAUSENAME without getting that the record (from another user) already existed even when they could not see it. It may be a decision taken before IBM took over Maximo, in MRO or even PSDI times.

    I have already created yet another AHA idea with the pain points and possible solutions and work arounds. Again, we have an application to alleviate most of them and some of the technical solutions are not that elegant or good (for example, the "Query Counter" and "Last Time Used" attributes of our application are difficult and kind of resource intensive to populate as we need to identify when a QUERY is going to be executed vs. just retrieving the record to do something else). For that, we are relying on Java stack traces to pinpoint known caller points where the query is being called to be executed. 

    See https://ideas.ibm.com/ideas/MASM-I-1235 and consider upvoting as well.

    So, putting the idea there will allow IBM to take a look at additional useful functionality they may be able to consider.

    I also have upvoted the ideas that were listed above.

    And especially in this subject, as most of us above; I do not concur on deleting records on the Maximo Database without warning, notice or admin recourse. I would even have preferred for the code to set the OWNER to something like 'DELETED' or 'MAXADMIN' as proposed above, which is not that good either, as the code may fail in case some query with a combination of APP and CLAUSENAME has been already "deleted" for another user.

    Thanks, and have a great day,

    Julio.



    ------------------------------
    Julio Perera
    Senior Maximo Technical Consultant
    Interloc Solutions Inc., US.
    ------------------------------