StreamSets

StreamSets

Connect with experts and peers to elevate technical expertise, solve problems and share insights.

 View Only
  • 1.  Cloud DB migration - query

    Posted Thu October 19, 2023 01:54 PM

    Can we configure multiple tables in same pipeline when we use JDBC Query consumer 1 as origin and JDBC producer 1 as destination. in my case i need to send 20 tables data to send from on-premise DB (db2) to cloud DB (Azure postgresql)?. i am thinking to avoid multiple pipelines creation in streamsets.



  • 2.  RE: Cloud DB migration - query

    Posted Wed November 01, 2023 03:27 PM

    <user-mention data-id="1388">@Sankar Arunachalam</user-mention> You can use MultiTableJDBCConsumer to configure multiple tables in a single stage in one pipeline. jdbc.tables is a record header attribute which might come in handy if you want to pass record to a particular table in destination from source. Read through SDC documents https://docs.streamsets.com/portal/datacollector/latest/help/index.html for the same.