Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Hi Vlad,

Server logs (a.k.a. Message logs), as per the cloud style infrastructure I referenced above, aren't even retained by TM1, they are sent (fire and forgot) to whatever log collector in your setup.

Audit logs, not the once you used to but rather a newly designed - driven by our new underlying infrastructure, we have plans for but can't guarantee, given the high value topics we are driving to get out this year, that it will be this year. But, as a heads up, even there we might end up choosing to use log collector infrastructure to collect those logs rather than having TM1 retain and manage them (tbd).

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Thanks @Hubert Heijkers!

Would you be able to point me to any relevant documentation or briefly explain whether there is any supported way to configure or integrate with a log collector in IBM PAaaS, particularly for forwarding logs to third-party applications like SPACE?

Appreciate your insight as always.

Hi Vlad,

Server logs (a.k.a. Message logs), as per the cloud style infrastructure I referenced above, aren't even retained by TM1, they are sent (fire and forgot) to whatever log collector in your setup.

Audit logs, not the once you used to but rather a newly designed - driven by our new underlying infrastructure, we have plans for but can't guarantee, given the high value topics we are driving to get out this year, that it will be this year. But, as a heads up, even there we might end up choosing to use log collector infrastructure to collect those logs rather than having TM1 retain and manage them (tbd).

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Vlad, AFAIK PAaaS only allows you access to that log through the PAA UI but I'm still hopeful that one day they'll either allow you to forward those messages to your preferred log collector/destination and/or expose an OData compliant (yes, I'm a dreamer) REST API that would give you access to those.

Again, in any other PA deployment, or even bare bones TM1 service deployment, outside of SaaS the customer has the keys to the kingdom and can do whatever they want with these logs already.

Thanks @Hubert Heijkers!

Would you be able to point me to any relevant documentation or briefly explain whether there is any supported way to configure or integrate with a log collector in IBM PAaaS, particularly for forwarding logs to third-party applications like SPACE?

Appreciate your insight as always.

Hi Vlad,

Server logs (a.k.a. Message logs), as per the cloud style infrastructure I referenced above, aren't even retained by TM1, they are sent (fire and forgot) to whatever log collector in your setup.

Audit logs, not the once you used to but rather a newly designed - driven by our new underlying infrastructure, we have plans for but can't guarantee, given the high value topics we are driving to get out this year, that it will be this year. But, as a heads up, even there we might end up choosing to use log collector infrastructure to collect those logs rather than having TM1 retain and manage them (tbd).

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Josh's question is really: transaction logs have a supported subscribe path; message logs on PAaaS do not.

Hubert's point is that those messages are no longer a TM1-owned file. They are emitted to the platform collector. On PAaaS the only supported consumer today is the PAA UI. That is a different contract from on-prem / PACC / PA 3.1, where you own the collector.

So the useful ask is not "put tm1s.Subscribe on the message log." It is a customer-facing integration point for what the platform already collects: either forward to a collector the customer controls, or a read API over the retained stream. Same requirement for the new audit events if those also land only in the collector.

Without one of those, operational monitoring on PAaaS stays a person watching PAA.

Vlad, AFAIK PAaaS only allows you access to that log through the PAA UI but I'm still hopeful that one day they'll either allow you to forward those messages to your preferred log collector/destination and/or expose an OData compliant (yes, I'm a dreamer) REST API that would give you access to those.

Again, in any other PA deployment, or even bare bones TM1 service deployment, outside of SaaS the customer has the keys to the kingdom and can do whatever they want with these logs already.

Thanks @Hubert Heijkers!

Would you be able to point me to any relevant documentation or briefly explain whether there is any supported way to configure or integrate with a log collector in IBM PAaaS, particularly for forwarding logs to third-party applications like SPACE?

Appreciate your insight as always.

Hi Vlad,

Server logs (a.k.a. Message logs), as per the cloud style infrastructure I referenced above, aren't even retained by TM1, they are sent (fire and forgot) to whatever log collector in your setup.

Audit logs, not the once you used to but rather a newly designed - driven by our new underlying infrastructure, we have plans for but can't guarantee, given the high value topics we are driving to get out this year, that it will be this year. But, as a heads up, even there we might end up choosing to use log collector infrastructure to collect those logs rather than having TM1 retain and manage them (tbd).

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.

Actually, and I'm not over the moon about it, the tm1s.Subscribe as it stands is defined as a bound action to a Database, read: not necessarily (just) TransactionLogEntries, yet it's description currently says it is to create a subscription to receive transaction logs of a Database. Perhaps that could be extended ;-)

The thing however is that, unlike transaction logs (for which we now have a separate publisher service which no longer competes for a lock on the transaction log), that each and every [micro-]service in the stack generates log entries, at least those from the TM1 service all have the same form/shared fields, but it is the cluster's collector that collects them. If we wanted to publish them we'd have to do something very unnatural and come up with a way to collect those ourselves in such set up to be able to then publish them as requested and/or forward them to the log collector set up for the cluster. You can image we'd like to refrain from having to do something like that albeit that we have the base plumbing in place for something like that as we do collect logs in our standalone/local (read: non-containerized) deployment. 

Josh's question is really: transaction logs have a supported subscribe path; message logs on PAaaS do not.

Hubert's point is that those messages are no longer a TM1-owned file. They are emitted to the platform collector. On PAaaS the only supported consumer today is the PAA UI. That is a different contract from on-prem / PACC / PA 3.1, where you own the collector.

So the useful ask is not "put tm1s.Subscribe on the message log." It is a customer-facing integration point for what the platform already collects: either forward to a collector the customer controls, or a read API over the retained stream. Same requirement for the new audit events if those also land only in the collector.

Without one of those, operational monitoring on PAaaS stays a person watching PAA.

Vlad, AFAIK PAaaS only allows you access to that log through the PAA UI but I'm still hopeful that one day they'll either allow you to forward those messages to your preferred log collector/destination and/or expose an OData compliant (yes, I'm a dreamer) REST API that would give you access to those.

Again, in any other PA deployment, or even bare bones TM1 service deployment, outside of SaaS the customer has the keys to the kingdom and can do whatever they want with these logs already.

Thanks @Hubert Heijkers!

Would you be able to point me to any relevant documentation or briefly explain whether there is any supported way to configure or integrate with a log collector in IBM PAaaS, particularly for forwarding logs to third-party applications like SPACE?

Appreciate your insight as always.

Hi Vlad,

Server logs (a.k.a. Message logs), as per the cloud style infrastructure I referenced above, aren't even retained by TM1, they are sent (fire and forgot) to whatever log collector in your setup.

Audit logs, not the once you used to but rather a newly designed - driven by our new underlying infrastructure, we have plans for but can't guarantee, given the high value topics we are driving to get out this year, that it will be this year. But, as a heads up, even there we might end up choosing to use log collector infrastructure to collect those logs rather than having TM1 retain and manage them (tbd).

Hi @Hubert Heijkers, do you have an estimated timeline for when Server and Audit logs will be available via the REST API in PAaaS / v12?

Thank you for confirming. 

Hi Josh,

Unfortunately it is not. The server/message logs go wherever the environment the service is running in, which in PAaaS deployments is whatever the underlying MCSP dictates. AFAIK in PAaaS as an end-user you have no access to the collected logs/the log collector, you can only view them through PAA  as there is no API available in PA that serves them up.

In a PACC deployment (read: you deploying PA in your Kubernetes cluster somewhere) on the other hand, you'd have more control technically as you'd be collecting using the log collector that cluster is set up with, and access to those could be granted to those that would require it, if you were granted that access, you'd be able to consume them directly through the log collector's API as well.

In a PA 3.1 setup the TM1 logs are, by default, collected and written to a file but also there can configure TM1 to push to a log collector of your choosing (LogStash/ELK, LogDNA, CloudWatch etc. etc.) and build or use existing UIs to interact/query/take action as things happen and get pushed to such collectors.