watsonx.ai

 View Only

  • 1.  why watsonx support only 3 gpus ?

    Posted 8 days ago

    Hello IBM team 

    I have a question why watsonx.ai support only 3 specific GPU models and also from specific model, also how can i utilize openshift for ai for inference for other card types.

    https://www.ibm.com/docs/en/cloud-paks/cp-data/5.0.x?topic=install-foundation-models

    Regards

    Ali Nada


    #watsonx.ai

    ------------------------------
    Ali Nada
    Senior AI and Analytics Advisor
    Alinma Bank
    Riyadh
    +966557742973
    ------------------------------


  • 2.  RE: why watsonx support only 3 gpus ?

    Posted 8 days ago
    Ali
    Most of the time, the work is done concerning the cards or type of GPUs and their respective drivers that RHOS knows or understands. It is an engineering effort to certify only a few for reasons due to HW and SW compatibility challenges. 

    Thus, it would be difficult to say that it will work without any problems if you move away from the prescribed GPUs

    Hope this helps

    Thanks
    Girish


    This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.



    Original Message


  • 3.  RE: why watsonx support only 3 gpus ?

    Posted 7 days ago

    Hey Manmadkar

    Thanks for replying back , but the thing is i am shocked from this limitation as it doesnt give us flexiability to host llm models with more than 100B paramater, also can you check this below arhcitecture .

    Figure 1. Red Hat OpenShift AI components

    according to this architecture any gpu can be supported by watsonX as long as it is supported by openshift.

    https://www.redhat.com/en/resources/openshift-ai-overview 

    Regards

    Ali Nada



    ------------------------------
    Ali Nada
    Senior AI and Analytics Advisor
    Alinma Bank
    Riyadh
    +966557742973
    ------------------------------

    Original Message