DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Python script to create a secure backup and download using XML management interface

  • 1.  Python script to create a secure backup and download using XML management interface

    Posted 05/07/26 04:06 PM

    I have a Python script that will create a secure backup and put the files in the temporary folder on my Datapower using SOMA and the XML management interface.  

    Now I want to retrieve those files using either the XM management interface or the REST interface and it seems like Datapower isn't allowing me to do that.  Is it possible?



    ------------------------------
    John Parker
    ------------------------------


  • 2.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/07/26 04:41 PM

    You can't retrieve them from the box, or put them back into it for a restore?

    You might want to check the default XML Manager.  It may not be allowing the XMI to export a document that large.



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 3.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/08/26 07:58 AM

    I don't think so.  The files are around 1,885 KB compressed.



    ------------------------------
    John Parker
    ------------------------------



  • 4.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/08/26 05:51 AM
    Edited by Andreas Brand 05/08/26 05:54 AM

    Yes, it is possible. I also run secure backup using Python, and it works perfectly.


    My approach in the script is as follows:

    1. Create a secure backup, wait for it to finish
    2. Download the backup files
    3. Delete the backup files (on the Datapower)

    However, I use the Raidstore as the storage location. I've never tried using the temporary directory.



    ------------------------------
    Andreas Brand
    ------------------------------



  • 5.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/08/26 08:02 AM

    I can create the backup, I just can't retrieve the files from Temporary.   I may try saving them in Local and see if I can pull from there



    ------------------------------
    John Parker
    ------------------------------



  • 6.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/08/26 10:41 AM
    Edited by Joseph Morgan 05/08/26 10:42 AM

    I use Java rather than Python, but I cannot imagine it has anything to do with the language.   I have no problems exporting anything from temporary.

    Have you turned on internal logging and examined the log to see if there is any error or indication related to the export attempt?



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 7.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 11:43 AM

    Yes, but nothing seems to be hitting the logs.

    What permissions do you give the credentials that connect to the Datapower?  I'm wondering if that is an issue.  I am also trying to do a full export, which I can create and download on some of my Datapowers, but not all of them, which is kind of weird



    ------------------------------
    John Parker
    ------------------------------



  • 8.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 11:51 AM

    I am using a privileged account, but not the "admin" account.  So you may be onto something.  What is the permission string for your user?



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 9.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 12:05 PM

    Ok, I also have a privileged account just for backups, but not the admin account.   Is there any other permissions that you give to it?  What version of firmware are you running?



    ------------------------------
    John Parker
    ------------------------------



  • 10.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 02:46 PM

    I'm doing it with KumbaSoft's DPAA, which uses the XMI with the account credentials.  I don't do anything additional than the account is privileged.  It works with FW from 10.5.x through 11.  FWIW, the V11 is docker only.



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 11.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 02:54 PM

    I used to be able to do this in WAMC, but the server does use the latest TLS, and non of the browsers will connect.



    ------------------------------
    John Parker
    ------------------------------



  • 12.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/18/26 03:20 PM

    What FW are you on.  Unless it is really old, I can at least light up a quick docker with that version and test to see if it works on that one.



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 13.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/19/26 09:05 AM

    We are running IDG.10.6.0.9



    ------------------------------
    John Parker
    ------------------------------



  • 14.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/19/26 09:34 AM

    Wow.  Now it's really strange.  I've tested on physical 10.6.0.9, VMware virtuals running 10.6.0.7 and 10.6.0.9, and dockers running 10.6.4.0, 10.6.5.0 and 11.0.0.0.  

    Is your form-factor something else?  That is, it *could* be a file system permission issue?



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 15.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/19/26 09:55 AM

    I've only tested on a physical Datapower, since I only run the secure backups on those.  That Datapower is a X1 (844152X )



    ------------------------------
    John Parker
    ------------------------------



  • 16.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/19/26 10:09 AM

    We once ran on X1's.  We're up to X3's now.   I don't recall issues on the X1's, but now, who knows.

    I'm rarely stumped... but stumped I am.   The only suggestion I can make is trying it through cURL via either mgmt interface.  If it doesn't work there, then you're likely dealing with a permissions issue.



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 17.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/20/26 04:19 PM
    Edited by Ivan Heninger 05/20/26 04:19 PM

    The secure-backup CLI has the ability to push the backup file "off box" when done.  I would expect you can call same function from XMI and/or RMI.  

    idg(config)#  secure-backup cd4_gm_cert sftp://user:pass@host/datapower/backup1/ova_cd4_nonprod "" on
    Secure backup to 'sftp://user:pass@host/datapower/backup1/ova_cd4_nonprod' scheduled (may take a few minutes to complete).
    Secure backup is complete.



    ------------------------------
    Ivan Heninger
    ------------------------------



  • 18.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/20/26 05:02 PM

    Ivan's got a good idea.  The validation regular expression for the destination field for the secure backup action is:

      "^(temporary|local):/{2,3}|^ftp://[^/]+(/[^/]+)*$|^sftp://[^/]+(/[^/]+)*$"

    So certainly worth a try to get it somewhere other than temporary, if even to local, which is allowed.



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 19.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/21/26 09:25 AM

    SOMA and REST both have problems working with large files because the files are base64 encodecd and take up a lot of space.

    The best approach to get files from secure backup is to let the secure backup request stream them to a FTP or SFTP server.  You would use something like <destination>sftp://YOUR-ID:YOUR-PASSWORD@YOUR-SFTP-SEVER:YOUR-PORT/YOUR-PATH</destination>

    If you want to diagnose your SOMA problem make sure you have a log target that is subscribed to warn level messages for all types.

    If you think your problem is RBM, you can turn on RBM debugging in the UI.  The debug messages will be in the default log.

    If the above don't show what your problem is, you can open a case with IBM. Be sure to include your SOMA requests and responses.



    ------------------------------
    John Graham
    ------------------------------



  • 20.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/22/26 07:38 AM

    Yeah, probably sending to a sftp server is the best way to go and I will probably try that.  I would have already, but I am running into a non-technical reason that is stopping me.

    The weird thing is that WAMC  was able to do it somehow, and that worked up until a couple of weeks ago.

    I also have an issue with manually creating a secure backup on one of my boxes.  It doesn't generate all of the files, even if I let it run for 10 minutes, while the other boxes take about 30 secounds.



    ------------------------------
    John Parker
    ------------------------------



  • 21.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/22/26 09:58 AM

    >> I have a Python script that will create a secure backup and put the files in the temporary folder ...

    As #Andreas Brand mentioned, have you tried local or on the local:///raid (if you have it and whatever it is named), rather than temporary?

    >> I also have an issue with manually creating a secure backup on one of my boxes.  It doesn't generate all of the files...

    Available space, maybe?



    ------------------------------
    Joseph Morgan
    CEO - Independent
    ------------------------------



  • 22.  RE: Python script to create a secure backup and download using XML management interface
    Best Answer

    Posted 05/25/26 09:31 AM

    Sorry to jump in, when reading this I don't see a reason it wouldn't be possible to fetch the files via SOMA or ROMA.
    We had a long time ago a similar issue with a secure backup on one of our boxes and reapplying the firmware solved it for us. 



    ------------------------------
    Jeroen Willems
    Integration Architect - Managing Partner
    Integration Designers
    ------------------------------



  • 23.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/26/26 11:02 AM

    You are more then welcome to jump in :)   

    I will try reapplying the firmware to see if that fixes the issue.



    ------------------------------
    John Parker
    ------------------------------



  • 24.  RE: Python script to create a secure backup and download using XML management interface

    Posted 05/26/26 11:00 AM

    Yes,  I tried placing them in the local folder, but it still failed.

    As for the manual backup, there's plenty of space.  It just doesn't create all of the files that it should.  Super weird.   This isn't the Datapower that I am testing my script on either.



    ------------------------------
    John Parker
    ------------------------------