May Hu

Who Really Uses Dataform? Spoiler: We Do, and Here’s Why!

Ever wondered, "Who really uses Dataform?" Dive into our hands-on experience of implementing Dataform at Pet Circle. For those wondering, Dataform is a service in Google to develop, test, version control, and schedule complex SQL workflows for data transformation in BigQuery. And no, this isn’t a sales pitch – we’re here to share the good, the bad, and the ugly of our journey. In this beginner-friendly talk, we’ll take you through our journey with Dataform – from initial setup and overcoming integration hurdles to achieving streamlined workflows. Along the way, we’ll also discuss the challenges of moving away from our current systems while implementing this new solution. Learn about the practical aspects, including best practices, common pitfalls, and effective solutions we discovered. Whether you're new to Google BigQuery or seasoned in building data pipelines, this talk will arm you with the pros and cons of using Dataform, empowering you to make informed decisions and elevate your data projects.