IBM Sterling Transformation Extender

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

You said you were keeping the source in Azure DevOps right?  It has the source mms - you can then build the script template out to compile the map or the msl - see ITX documentation for how to compile via command line and build it into the script that is executed by your pipeline.  In TFS, you could configure all of this and it saves the output (compiled object) which is then used to deploy to designated/configured target (your Linux server) with options which should include things like BINARY transfer, and directory overrides - I can't tell you how to do it step by step because I don't have a TFS or Azure DevOps system currently where I could work out all the details so I am commenting from memory as best as I can.   We deploy via SFTP.  It's a manual process here where I am now.  We are starting this path to potentially streamline it a bit but won't be with Azure DevOps or TFS as much as I really like it.  We have very strict security controls that will require a different approach.

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Hi Lisa,

You mentioned " you can then build the script template out to compile the map or the msl".

When I reviewed the documentation on sdeploy i see how to do it(i have used ssftpdeploy from my PC), however I do not see this on my LINUX environment, just on my local PC where I installed Design studio and Flow Designer. 

Am i missing something in Linux, or is my understanding of how I do it via the script template will have to connect to some windows environment where the sdeploy.exe is installed?

Thanks,

Attila

You said you were keeping the source in Azure DevOps right?  It has the source mms - you can then build the script template out to compile the map or the msl - see ITX documentation for how to compile via command line and build it into the script that is executed by your pipeline.  In TFS, you could configure all of this and it saves the output (compiled object) which is then used to deploy to designated/configured target (your Linux server) with options which should include things like BINARY transfer, and directory overrides - I can't tell you how to do it step by step because I don't have a TFS or Azure DevOps system currently where I could work out all the details so I am commenting from memory as best as I can.   We deploy via SFTP.  It's a manual process here where I am now.  We are starting this path to potentially streamline it a bit but won't be with Azure DevOps or TFS as much as I really like it.  We have very strict security controls that will require a different approach.

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Wouldn't you want to deploy it from where you keep your source?  The target is the Linux if I am understanding correctly.  You want to build a pipeline from your Azure DevOps where you keep your source, compile it and then transfer it to the Linux server.  The command line guide for compiling maps is here:

https://www.ibm.com/docs/en/ste/10.1?topic=designer-mcompile-utility-command

I don't know what version you're using but picked 10.1 because I have it bookmarked.  This will give you guidance on how to build out a script to compile your maps which will aid in building this pipeline for Azure DevOps.  I cannot tell you step by step how to do it because I haven't done it - I do not have Azure DevOps and even if I did my environment would be different than yours.  I know that you can do the compiling via script, you can transfer via script with sftpdeploy.  You will need to work with your server people to set up sftp on your Linux box to receive the code and transfer it to the location needed.  Or to assist you with the target of an Azure DevOps pipeline.  I cannot tell you how to do that in your environment.

Hi Lisa,

You mentioned " you can then build the script template out to compile the map or the msl".

When I reviewed the documentation on sdeploy i see how to do it(i have used ssftpdeploy from my PC), however I do not see this on my LINUX environment, just on my local PC where I installed Design studio and Flow Designer. 

Am i missing something in Linux, or is my understanding of how I do it via the script template will have to connect to some windows environment where the sdeploy.exe is installed?

Thanks,

Attila

You said you were keeping the source in Azure DevOps right?  It has the source mms - you can then build the script template out to compile the map or the msl - see ITX documentation for how to compile via command line and build it into the script that is executed by your pipeline.  In TFS, you could configure all of this and it saves the output (compiled object) which is then used to deploy to designated/configured target (your Linux server) with options which should include things like BINARY transfer, and directory overrides - I can't tell you how to do it step by step because I don't have a TFS or Azure DevOps system currently where I could work out all the details so I am commenting from memory as best as I can.   We deploy via SFTP.  It's a manual process here where I am now.  We are starting this path to potentially streamline it a bit but won't be with Azure DevOps or TFS as much as I really like it.  We have very strict security controls that will require a different approach.

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

Hi Lisa,

Again, really appreciate you putting up with all my questions!!

I am struggling with how/where the pipeline would get the compiled maps and launchers. 

Is the idea the developer checks in both the source and the compiled version to ADO, and then ADO takes from there?

or

Is the idea the developer checks in source to ADO, and then the ADO does the compile, and gets it to the Linux Servers?

I am assuming the latter and that is where I am getting hung up.

Thanks,

Attila

Wouldn't you want to deploy it from where you keep your source?  The target is the Linux if I am understanding correctly.  You want to build a pipeline from your Azure DevOps where you keep your source, compile it and then transfer it to the Linux server.  The command line guide for compiling maps is here:

https://www.ibm.com/docs/en/ste/10.1?topic=designer-mcompile-utility-command

I don't know what version you're using but picked 10.1 because I have it bookmarked.  This will give you guidance on how to build out a script to compile your maps which will aid in building this pipeline for Azure DevOps.  I cannot tell you step by step how to do it because I haven't done it - I do not have Azure DevOps and even if I did my environment would be different than yours.  I know that you can do the compiling via script, you can transfer via script with sftpdeploy.  You will need to work with your server people to set up sftp on your Linux box to receive the code and transfer it to the location needed.  Or to assist you with the target of an Azure DevOps pipeline.  I cannot tell you how to do that in your environment.

Hi Lisa,

You mentioned " you can then build the script template out to compile the map or the msl".

When I reviewed the documentation on sdeploy i see how to do it(i have used ssftpdeploy from my PC), however I do not see this on my LINUX environment, just on my local PC where I installed Design studio and Flow Designer. 

Am i missing something in Linux, or is my understanding of how I do it via the script template will have to connect to some windows environment where the sdeploy.exe is installed?

Thanks,

Attila

You said you were keeping the source in Azure DevOps right?  It has the source mms - you can then build the script template out to compile the map or the msl - see ITX documentation for how to compile via command line and build it into the script that is executed by your pipeline.  In TFS, you could configure all of this and it saves the output (compiled object) which is then used to deploy to designated/configured target (your Linux server) with options which should include things like BINARY transfer, and directory overrides - I can't tell you how to do it step by step because I don't have a TFS or Azure DevOps system currently where I could work out all the details so I am commenting from memory as best as I can.   We deploy via SFTP.  It's a manual process here where I am now.  We are starting this path to potentially streamline it a bit but won't be with Azure DevOps or TFS as much as I really like it.  We have very strict security controls that will require a different approach.

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila

The source mms is in Azure DevOps you've stated - per their product page it sounds like it works just like TFS. 
https://azure.microsoft.com/en-us/products/devops/pipelines/

You will configure the yaml pipeline to get the source mms from it's Azure DevOps storage location, tell it how to build it using mcompile guide I shared above (command server utilities will have to be installed on the DevOps environment to use this), and then also configure how/where to deploy on your Linux server.  These are a mix of standard options that you will customize for ITX deployments to run mcompile and sftpdeploy or use their built in transfer service for deploying to your server environment.  There is no need to put the compiled map into DevOps becuase it can compile it as part of the deployment process - this is how it works for C#, or java code as well.

If the Azure DevOps environment is serverless, then you will point to a server where command server for ITX is installed in that yaml configuration and give it the appropriate permissions to run and utilize resources. 

I hope this helps you visualize a solution a bit further.

Good luck!

Hi Lisa,

Again, really appreciate you putting up with all my questions!!

I am struggling with how/where the pipeline would get the compiled maps and launchers. 

Is the idea the developer checks in both the source and the compiled version to ADO, and then ADO takes from there?

or

Is the idea the developer checks in source to ADO, and then the ADO does the compile, and gets it to the Linux Servers?

I am assuming the latter and that is where I am getting hung up.

Thanks,

Attila

Wouldn't you want to deploy it from where you keep your source?  The target is the Linux if I am understanding correctly.  You want to build a pipeline from your Azure DevOps where you keep your source, compile it and then transfer it to the Linux server.  The command line guide for compiling maps is here:

https://www.ibm.com/docs/en/ste/10.1?topic=designer-mcompile-utility-command

I don't know what version you're using but picked 10.1 because I have it bookmarked.  This will give you guidance on how to build out a script to compile your maps which will aid in building this pipeline for Azure DevOps.  I cannot tell you step by step how to do it because I haven't done it - I do not have Azure DevOps and even if I did my environment would be different than yours.  I know that you can do the compiling via script, you can transfer via script with sftpdeploy.  You will need to work with your server people to set up sftp on your Linux box to receive the code and transfer it to the location needed.  Or to assist you with the target of an Azure DevOps pipeline.  I cannot tell you how to do that in your environment.

Hi Lisa,

You mentioned " you can then build the script template out to compile the map or the msl".

When I reviewed the documentation on sdeploy i see how to do it(i have used ssftpdeploy from my PC), however I do not see this on my LINUX environment, just on my local PC where I installed Design studio and Flow Designer. 

Am i missing something in Linux, or is my understanding of how I do it via the script template will have to connect to some windows environment where the sdeploy.exe is installed?

Thanks,

Attila

You said you were keeping the source in Azure DevOps right?  It has the source mms - you can then build the script template out to compile the map or the msl - see ITX documentation for how to compile via command line and build it into the script that is executed by your pipeline.  In TFS, you could configure all of this and it saves the output (compiled object) which is then used to deploy to designated/configured target (your Linux server) with options which should include things like BINARY transfer, and directory overrides - I can't tell you how to do it step by step because I don't have a TFS or Azure DevOps system currently where I could work out all the details so I am commenting from memory as best as I can.   We deploy via SFTP.  It's a manual process here where I am now.  We are starting this path to potentially streamline it a bit but won't be with Azure DevOps or TFS as much as I really like it.  We have very strict security controls that will require a different approach.

Hi Lisa,

Really appreciate your comments..

RE: In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.

Would this be the dev doing the command line script(sdeploy.exe) or is there some other way to get it into TFS

Thanks,

Attila

Yes, I would think they would have methods/pipelines for binary transfer of artifacts to your Linux server - it would be in the configuration.  I haven't played with Azure DevOps but I can imagine that it built upon TFS fundamentals and adapted to the cloud.  You may need a client on the Linux box, like had to be done for TFS deploy, but I don't know for certain.  Permissions have to be granted at the least.  You would want to ensure it is a BINARY transfer for compiled objects through the pipeline.  In order to be really CI/CD best practices compliant, it would want to do the compile ahead of deploy - that can and should be configured via command line script that executes and then uses the results to deploy in your pipeline configuration of tasks.  In previous employment, we had 3 levels of automation corresponding to environment.  DEV was not automated on check-in but triggered by developer.  Once the kinks (if any) were worked out the source/project artifacts were migrated to QA and checked in and deployment was automated by the check-in trigger.   Once approved after testing, a formal change management task was created and source was merged to the production branch where it waited for the change control to be approved and then it was released / triggered by the infrastructure team in an official or emergency change control window.  However, at no time did anyone manually copy code or compile it in the CI/CD system - it was all based on the check-in to source control and / or releasing the change to the server.  Hope this helps.

Hi Lisa,

We use Azure DevOps(ADO), to maintain the source, so would be looking to expand on that. Looking into pipelines?!!?

Are you aware of any best practices using that? How do you get the compiled(.msl  &.mmc) to the linux servers? Would the developer need to put the Source and compiled version in ADO, and the pipelines push that to linux? 

Thanks,

Attila

It all depends on how much you want to automate, how much you want to spend in both time and money to build out a system.  There are complete CI/CD systems such as TFS (Team Foundation Server) where you can have source control and deployment capabilities.  It is higher on the scale of time and money commitment but once it's set up works pretty flawlessly and easily and can be used for any kind of deployable code - not just ITX code.  On the cheap end of the spectrum, you could make something similar using free versions of Git and custom scripts to do the compile for linux and either automate the sftp from a deployment directory or have the Git repo be directly on the server environment with backup replication on developers Git client.  You'd still need to transfer/sync the mms to the repo. There are other less expensive CI/CD tools as well - Octopus Deploy is one that can span on prem, hybrid and cloud environments and is approximately 1$ per day current pricing.  I am sure there are tons of others.  Get/define a budget, understand your security model and work with your infrastructure team, do your research, and get building!  Free tools require more effort on your part, but keep in mind more expensive doesn't always mean there isn't effort and maintenance on your part either.

Hello,

How are teams migrating/promoting their maps and Launchers that are created from their PC to a Linux environment?? I know there is "ssftpdeploy", but for this you have to include server info in the Launchers, and if servers change then this info is all stale, and it also requires ssh connectivity.

How are you then promoting them from one environment to the next and eventually Production.

Looking to standardize & automate this. 

Thanks,

Attila