Posts

Originally streamed live on August 3, 2023 - LinkedIn Learning’s “The Future in Tech” series Data engineering is the unsung hero fueling the rapid growth and consumption of artificial intelligence. It transforms AI’s potential into reality, driving digital innovation and reshaping the world. In this comprehensive discussion, we explore how data engineering unlocks and enables democratized use of Artificial Intelligence. Video: The Future in Tech - Data Engineering and AI Discussion (1,668 views) ...

In this course, instructor Vinoo Ganesh gives you an overview of the fundamental skills you need to become a data engineer. Learn how to solve complex data problems in a scalable, concrete way. Explore the core principles of the data engineer toolkit—including ELT, OLTP/OLAP, orchestration, DAGs, and more—as well as how to set up a local Apache Airflow deployment and full-scale data engineering ETL pipeline. Along the way, Vinoo helps you boost your technical skill set using real-world, hands-on scenarios. ...

This week on The Data Stack Show, Eric and Kostas chat with Vinoo Ganesh. During the episode, Vinoo discusses how to benchmark cost, optimize your workloads, and Bluesky’s role in addressing your Snowflake bills. Video Link https://datastackshow.com/podcast/optimizing-query-workloads-and-your-snowflake-bill-with-vinoo-ganesh-of-bluesky-data/

The data pipeline has become a fundamental component of the data science, data analyst, and data engineering workflow. Pipelines serve as the glue that links together various components of the data cleansing, data validation, and data transformation process. However, despite its importance to the data ecosystem, constructing the optimal data pipeline is generally an afterthought - if it’s considered at all. This makes any changes to the central pipeline highly error-prone and cumbersome. With the ever-growing demand for new kinds of data, especially from external vendors, constructing pipelines that are scalable and that allow for monitoring is pivotal for the safe and continued use of data. ...

Data pipelines are the foundation for success in data analytics, so understanding how they work is of the utmost importance. Join us for four hours of expert-led sessions that will give you insight into how data is moved, processed, and transformed to support analytics and reporting needs. You’ll also learn how to address common challenges like monitoring and managing broken pipelines, explore considerations for choosing and connecting open source frameworks, commercial products, and homegrown solutions, and more. ...