Data Council Boston virtual meetup: Data Engineering Workflows & ML Ops

Global AI & Data Science

Train, tune and distribute models with generative AI and machine learning capabilities

View Only

This is an online 1h meetup during lunch time. Let's get the meetups going again! We will have 2 exciting short talks this time.

12:00pm
Mary Clair Thompson
Academic Data Engineer from Duke University

Prefect as a Data Engineering Workflow Management Tool

Choosing a workflow management tool that suits a team’s current needs and projected future projects can be a difficult decision. Excellent options abound (NiFi, Airflow, etc), and every team’s use case is unique. In this talk, I will discuss the decisions that led my data analytics and engineering team at Duke’s Office of Information Technology to choose Prefect as our workflow management tool. I will also give an overview of our system’s architecture and integration with CI/CD and version control tools. Finally, I will demonstrate some of the features of the Prefect UI.

12:30pm
William Roberts
Data Science Evangelist from IBM

ML Ops, a Meetup

Consider how the software development life cycle (SLDC) is well-defined at this point: planning, creating, testing, deploying, maintaining – or some variant, depending on your software methodology. The gist remains consistent. Computer software runs “logic” in hardware, the test suites are repeatable, it’s a relatively deterministic process. Consider how, with machine learning, we’re working with probabilistic systems instead. Instead of writing code as instructions, we’re guiding these systems to learn from data. IBM has a mission to help bring machine learning capabilities to all, so we can all participate in the AI economy responsibly. Consequently, there are many different participants and stakeholders in this emerging field of ML Ops. This meetup will review the current state of what it takes to build successful pipelines in this probabilistic setting so that we can build a shared understanding of why we treat our models as living products and ask the right questions: Are they healthy? Are the representative? Are they biased?
#GlobalDataScience

More information