Original Message:
Sent: Mon July 15, 2024 09:18 AM
From: Hubert Heijkers
Subject: File half-interface on AWS
Hi Philippe,
Appreciating that change is hard...
Unlike as suggested above, albeit not sure how exactly to interpret "moving away from the file system", TM1 itself has not changed where it comes to reading from files. What has changed is what files it has access to due to the fact that, when running as a service in a containerized environment, it only has access to volumes, PVCs, that is has been given access to. And whilst traditional means to interacting with such volumes might not exist (truth be told we definitely don't want you to either;-) and if the file manager interface provided by PA is to limiting, through TM1's REST API you have full control over the files that are on that volume, including support for multi-part uploads and partial range-based download to speed up both upload and downloads of large files.
And if you'd rather keep your files on a more 'traditional' volume you can do so to and expose them to TM1 through an HTTP[S] endpoint (i.e. by simply putting an NGINX server in front of it for example, hopefully secured using some form or token/secret), as TM1 v12 can read (GET) and write (PUT) directly to such HTTP[S] sources which don't actually have to be backed by files but could be a service that generates/processes such content on the fly as well.
The 100Kb limitation you refer to only applies to responses you are putting into memory (read: a string), if you opt in to storing the response to a file, then there is no limitation to how big that response can be.
Embracing change can be a great thing as TM1 v12 has a whole lot more to offer then its predecessor.
Hope this helps,
PS Whilst IMHO the pub/sub is the way to go, the old transaction log APIs have been re-implemented in TM1 12.4 (it is actually the pub/sub service that caches them for you and makes them available through the same old API for backwards compatibility).
------------------------------
Hubert Heijkers
STSM, Program Director TM1 Functional Database Technology and OData Evangelist
Original Message:
Sent: Mon July 15, 2024 08:31 AM
From: Philippe CHAMPLEBOUX
Subject: File half-interface on AWS
Thanks Edward,
But I think you talk about Cloud Saas Planing Analytics, not about PA As a Service. (With PA Engine)
For sample, get "transaction log" need, (on AWS) a set of complex https requests like
<<
PA as a Service currently allows a customer to subscribe their database to a log endpoint/log service of their choosing (one that they provide). However this is subject to change in coming releases, customer may want to listen to the Planning Analytics AMA recording with Product Management here: https://www.youtube.com/watch?v=BSOXSgECIaA&t=2900s
Existing Steps to enable Transaction Logging:
A POST request to the PA Database allows consumers to subscribe to the transaction logs of that Database. Only the transaction log entries created at or after the point of subscription will be sent to the subscriber. A HTTP or TCP server is required to be able to collect transaction logs of a Database.
To subscribe to the endpoint, the customer issues the following POST request to their PA Database:
etc ....
>>
An other point (about Half-interfaces) .....
You can use http request in TI (Connect to your datasource software with API, get Extraction ......) But your TI Process limit to 100 Kb the result of extraction....
Regards,
Philippe
------------------------------
Philippe CHAMPLEBOUX
Original Message:
Sent: Mon July 15, 2024 04:46 AM
From: Edward Stuart
Subject: File half-interface on AWS
The file manager covers the brief for file extractions and imports - https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=2wnj22-manage-files-file-manager
Managing Database log files is covered here - https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=workspace-manage-planning-analytics-database-log-files
There are a number of changes that will effect existing installations being moved to PAE, on the other hand there are a lot of developments in place and on the Planning Analytics Roadmap which are likely to improve the IBM Planning Analytics Workspace experience. A few stories/ items due for release in Q3 for example:
------------------------------
Edward Stuart
Original Message:
Sent: Sat July 13, 2024 07:50 AM
From: Philippe CHAMPLEBOUX
Subject: File half-interface on AWS
Thanks Vlad ,
But I'm not philosophically ready to defend the idea of paying add-ons because the ibm offering no longer contains certain basic features.
The questions for me are:
- if Cloud Saas is reserved for configurations with more than 65 users and my customer (under 65 users) want to move to the cloud, what am I going to be able to offer him, knowing that :
- Audit and transaction logs can no longer be consulted (how can we ensure traceability and audit trails?).
- All TurboIntegrator developments need to be reviewed or analyzed (including my famous half-interface problem).
- No more automatic interaction with files/directories
- No more Tm1Web
I don't think I can offer him anything other than having Windows VMs in the Cloud and installing "on premise", which seems a bit of a shame, doesn't it?
If I'm wrong about PA's features on AWS, I'd appreciate it if you'd correct me.
Best regards,
Philippe
------------------------------
Philippe CHAMPLEBOUX
Original Message:
Sent: Fri July 12, 2024 12:25 PM
From: Vlad Didenko
Subject: File half-interface on AWS
Hi Philippe, if you consider paid solutions, SPACE has Python and Jupyter Notebook integrated directly into PAW.
This allows you to upload, create, edit, and run Python, Jupyter Notebook, and other files and opens up a wide range of possibilities for integrating IBM Planning Analytics with pretty much anything that has an API. Feel free to ask if you have any questions about specific use cases:
------------------------------
Vlad Didenko
Founder at Succeedium
TeamOne Google Sheets add-on for IBM Planning Analytics / TM1
https://succeedium.com/teamone/
Succeedium Planning Analytics Cloud Extension
https://succeedium.com/space/
Original Message:
Sent: Fri July 12, 2024 11:05 AM
From: Philippe CHAMPLEBOUX
Subject: File half-interface on AWS
Hello !
Hello,
I'm a bit annoyed by PA on AWS which doesn't have an accessible file system.
In Local or Saas Cloud, I've gotten into the habit of developing specific extraction programs (API, ETL, Talend, etc...) and dropping the extraction files (copy or ftps) where PA can read them.
That I call half-interfaces
How do I proceed with PA on AWS? Is there a way to upload these files to Model_Upload?
any ideas and suggestions are most welcome
Kind Regards,
Philippe
------------------------------
Philippe CHAMPLEBOUX
------------------------------