Tyler Su
Flyte Monthly Nov 2022
This blog post is an edited, public copy of Flyte Monthly’s November 2022 newsletter issue. To subscribe, visit getrevue.co/profile/flyte.
Welcome to our newsletter, and thank you for subscribing.
Each month, we’ll keep you up to date with Flyte news, the latest releases and features, tutorials and workshops, events, and maybe a top meme or tweet.
What’s Up?
KubeCon + CloudNativeCon North America 2022
Union.ai represented Flyte at KubeCon + CloudNativeCon North America 2022 in Detroit. In addition to the physical booth on site, we also had a virtual booth that’s still live through Nov 21.
Flyte maintainers Yee Hing Tong, Katrina Rogan and Daniel Rammer hosted the Flyte booth during the event, while Eduardo Apolinario, Samhita Alla and Shivay Lamba demoed Flyte and the all-new Flyte Sandbox during the virtual booth live office hours. Read up on the press release (Orchestration Takes Center Stage at KubeCon), and try out the Flyte Sandbox here.
GITEX Global Dubai 2022
Haytham Abuelfutuh and Niels Bantilan of Union.ai attended GITEX Global Dubai 2022, where they conducted presentations on Flyte, UnionML and Pandera.
Recordings of the talk will be published in the coming months. Until then, you can find the presentation materials here.
Flyte’s Latest...
Release 🚀
Look for a Flyte v1.3.0 release coming out in January 2023. Details on what that release will contain is in the section below, but keep an eye on our GitHub page for the official release.
Have suggestions for Flyte?
We’d love to hear from you during our next Quarterly Planning Meeting in early January. We’ll discuss the future of Flyte’s development, community involvement, and we’ll take feature requests for upcoming releases!
Mark the event in your calendar here.
The first quarterly planning meeting was in October, when Eduardo Apolinario covered the Flyte Roadmap for v1.3.0, and other features for the release in early 2023. The full rundown is in the recording below.
You can also raise suggestions in our Slack, specifically in channels #ask-the-community and #feature-discussions, or directly let Eduardo Apolinario know!
Success Story 🤝
Lacuna.ai is a transportation technology company that creates products to help its customers optimize traffic movement for cities, airports and other complex networks of moving vehicles.
At the most recent Flyte Community Sync, Machine Learning Engineer Wen Shu Tang spoke about the conception of Lacuna.ai and the problems it faced.
When evaluating workflow frameworks for Lacuna.ai, Wen Shu said he needed to balance data overhead and ease of adoption. Flyte’s clean abstractions and active community helped him choose Flyte. Here’s how Wen Shu convinced his team of data scientists that Flyte is critical for their workflows:
It was a difficult case I had to make to the team because these things I’m talking about — operational efficiency, or lower infrastructural overhead, or having good abstractions — they are all very abstract concepts. For somebody who comes from an academic or pure data-science domain, it’s very hard to grasp. So I very much had to play the role of advocacy and try to evangelize it. At the end of the day, it’s not something that’s tangible. So I think being able to start the process of converting some of our work into something tangible, into something you can visualize, that’s another advantage of Flyte that I really like.
Hear Wen Shu’s full journey in guiding Lacuna.ai’s adoption of Flyte below.
This Is How We Do It ⚙️
Flyte Roadmap
In the video below, Eduardo Apolinario details Flyte’s recent v1.2.0 release, which includes changes such as extended workflow execution names, and a large performance improvement by offloading FlyteWorkflow CRD. In addition, Eduardo highlights the timeline for v1.3.0, and the highly requested features in store for that update: human-in-the-loop tasks, and out-of-core back-end plugins!
Flyte Map Tasks
A Flyte blog post by Vijay Saravana Jaishanker of Woven Planet describes how Flyte map tasks offer a simpler user experience than Spark workflows. Vijay goes into detail about Spark’s features and use cases, and where Flyte’s workflows are able to relieve some of the user pain points in areas where Spark’s more advanced features aren’t required. Examples and further comparison can be read in the article here:
Flyte Map Tasks: A Simpler Alternative to Apache SparkApache Spark is a distributed system for large-scale data processing. Learn how Woven Planet migrated some of their Spark workflows to Flyte map tasks...blog.flyte.org
In The News 📣
Union-Hosted Flyte Sandbox
Union.ai has announced that it is hosting a Flyte Sandbox! Essentially a trial and walkthrough, Union.ai’s Flyte Sandbox lets users test Flyte features in a self-contained environment within your browser. Check out this blog post to learn how to get started!
Hacktoberfest Recap
Hacktoberfest 2022 is over, and the Flyte community finished strong! We’re pleased to report that we had 23 contributors, 42 issues and 50 PRs between Flyte and UnionML.
Contributions
Ryan Nazareth is the top contributor and he’ll be soon receiving an O’Reilly individual premium annual subscription worth $500! His contributions include:
- Plugin for supporting Vaex DataFrame type
- TypeTransformer for TensorFlow TFRecord
- Fixing mypy errors for incompatible types
- Tutorial on running an NLP workflow with Flyte
Other noteworthy Hacktoberfest contributions:
- TensorFlow tensor and model types by @VPraharsha03 and @techytushar
- GitPod environment setup in UnionML by @MrKrishnaAgarwal
- Listing differences on conflicting definition when registering by @jerempy
- Using passthrough workflowstore to perform CRD terminations / manipulations by @daniel-shuy
- Dockerizing call to “go generate” in the flyteidl makefile by @daniel-shuy
Hear Us Speak 🎤
Upcoming
Andrew Dye was recently a guest on the MLOps Community Podcast, and we’re thrilled for the release of that recording! Keep an eye out for it here in the upcoming weeks.
If you would like to see Flyte at any of your favorite meetups, let us know! Contact Tyler Su in the Flyte Slack for details.
Coming Up 📅
On Nov. 15 at 9 a.m. PT, we’ll hold our biweekly Community Sync! We’ll feature Calvin Leather from Embark Veterinary, in addition to our usual agenda of community updates.
Mark the event on your calendar here.
Where To Find Us 🔎
Slack
Join us on Slack to ask a question, learn from a thread, follow announcements or just say hi!
Follow us on Twitter at @flyteorg for everything Flyte.
Zoom
- Weekly on Wednesdays” office hours: three 30-minute sessions at 7:00 a.m. PT, 1:30 p.m. PT and 9:00 p.m. PT
- Community Sync: Join us twice a month on Tuesdays at 9 a.m. PT for presentations and open discussions
GitHub
Post a question or comment on GitHub discussions, open an issue, or show us your work!
Thank You Contributors! 🙏
Flyte would like to recognize ✨Contributors of the Month✨ of October, for their valuable work and contributions:
- Ailin Yu, @niliayu implemented container-level configuration on default PodTemplate to improve the configurability of Flyte
- Nicholas LoFaso, @nicklofaso leads the Flyte + MethaneSAT integration. Check out Nick’s talk on how MethaneSAT’s transforming satellite data with Flyte
- Justin Tyberg, @jtyberg helped us discover a security vulnerability on the OAuth authorization server (fixed in flyteadmin 1.1.44)
