Hello Zachary,
I think this answer depends upon your short term needs and long term goals. Focus the learning energy where you'll provide short term wins for your team, but also be able to provide benefit for your professional growth long term. I embrace both methods (machine learning & prompt tuning) and think examples of each will be best for your journey. I'll choose a public dataset, IBM CloudPak for Data, AutoAI and watsonx to illustrate.
---------- Machine Learning ----------
For machine learning, the public dataset "Titanic" will be used. This can be found at: Link to Titanic Dataset on Kaggle.com
I downloaded "train.csv" and utilized this file only to simplify the example. It was uploaded to the IBM Cloud Object Storage (COS) bucket and run through Auto AI modeling Survivability. The training set was split into 90% of the data for training and 10% for testing. There are many examples online to cross reference and I won't go into it here. I wanted to know the top 3 algorithms that best model survivability, and what is the accuracy using each algorithm? The result is attached here:
The algorithms I selected were XGB Classifier (Accuracy 0.83 out of 1.0), Extra Trees Classifier (Accuracy 0.797 out of 1.0) and Snap Logistic Regression (Accuracy 0.780 out of 1.0). Now the accuracy number is calculated from the 10% testing dataset withheld from the model training. Each pipeline could be exported to a Jupyter Notebook allowing Python cross checks, improving explainability and trust. I could independently try neural networks and regression algorithms, but wanted to reduce time and let Auto AI do all the legwork.
---------- Prompt Tuning ----------
This example is completly text based using Generative AI and a foundation model to accomplish different tasks. We've all heard about Generative AI providing answers, how about changing the prompt given to a Large Language Model (LLM) to modify the task accomplished? These examples will use IBM watsonx with LLM llama-2-13b-chat only. The prompts submitted are black text on a white background, and responses are black text on a light blue background.
First - Modify the prompt to obtain a summarized answer to the question: What does the IBM corporation provide?
------------------------------
Daniel Morvay
------------------------------
Original Message:
Sent: Thu December 14, 2023 12:30 PM
From: Zachary Cairns
Subject: Prompt Engineering - Worth it?
I came to post this message on Prompt Engineering after learning about the several courses offered. What I'm wondering is: Yes, it is clearly useful to be able to 'instruct' (it's not engineering)the AI Model in a way that facilitates what you need...but compared to traditional Machine Learning programming techniques, how worthy is it?
If it's only supplemental job field that's growing, it probably wouldn't hurt to learn it along with traditional AI programming techniques, right? Or should I just focus on the traditional AI programming courses and leave Prompt Engineering for later?
------------------------------
Zachary Cairns
------------------------------
#AIandDSSkills