IBM Z and LinuxONE IBM Z

IBM Z

The enterprise platform for mission-critical applications brings next-level data privacy, security, and resiliency to your hybrid multicloud.

 View Only

  • 1.  Spark/IzODA and Scala

    Posted Thu July 23, 2020 09:51 AM
    I am new to Spark/IzODA and Scala.  I can run simple scala programs in Spark-shell but wondering if I have a more complex full application in separate source files, how do I compile them?  Is it possible to compile scala programs into a jar and run in Spark/IzODA?  I've read online about SBT but not sure if it is available on Z or if there is equivalent?

    ------------------------------
    Pacifico Gatela
    ------------------------------


  • 2.  RE: Spark/IzODA and Scala

    Posted Fri July 24, 2020 01:45 AM
    Hi @Pacifico Gatela, you can build using sbt or maven. For sbt, refer section 4.1.3 and 4.2.2 of the redbookhttps://www.redbooks.ibm.com/redbooks/pdfs/sg248325.pdf. This redbook  will also help you ​learn more on the Spark on z/OS. It's bit old but most of the content is still relevant.

    Thank you,
    Nasser

    ------------------------------
    NASSER EBRAHIM
    ------------------------------

    Original Message


  • 3.  RE: Spark/IzODA and Scala

    Posted Mon August 10, 2020 07:02 PM
    Hi Nasser,

    Thanks for the quick response.  I tried to download SBT but it appears the site it blocked by my organizationl.  I will have to work on this with our sercurity admin.

    Thanks,
    PGatela

    ------------------------------
    Pacifico Gatela
    ------------------------------

    Original Message


  • 4.  RE: Spark/IzODA and Scala

    Posted Fri July 24, 2020 10:44 AM
    Additionally, you can use Apache Maven which is part of IzODA, available through Anaconda. This is supported on zOS through IzODA, while sbt can be used but is not an officially delivered product on Z.
    ​

    ------------------------------
    NICHOLAS MARION
    ------------------------------

    Original Message


  • 5.  RE: Spark/IzODA and Scala

    Posted Mon August 10, 2020 07:05 PM
    Hi Nicholas,

    I'm curious to explore this, any idea where to start?  Any steps to get started this path?

    Thanks,
    PGatela

    ------------------------------
    Pacifico Gatela
    ------------------------------

    Original Message


  • 6.  RE: Spark/IzODA and Scala

    Posted Wed August 12, 2020 03:28 PM

    Hi Pacifico,

    If you're developing on zOS; you can install Maven through Anaconda. Find info here: https://izoda.github.io/site/anaconda/optional-packages/#apache-maven

    Additionally, if you're using MDS's Data Service Studio; it has a built in tutorial for creating an application for Spark within the eclipse environment.

    Maven will work the same on Z or off.



    ------------------------------
    NICHOLAS MARION
    ------------------------------

    Original Message


Related Content