🦆 Making Magic: MotherDuck with Mage

Here at Mage, we're always looking for the latest and greatest ways to make data engineering simple, accessible, and

fun

.

That's why we've parntered with

to introduce

complete

support for DuckDB and Motherduck! 🪄 🦆

In version

0.9.28

and above, you'll have access to a suite of DuckDB features like:

• Reading from/writing to DuckDB databases:

  Mage can read data from DuckDB databases directly into your piplines

  and

  output to DuckDB. That includes our easy-to-use templates for loading/exporting!

• Streaming data to DuckDB destinations:

  Since Mage supports to streaming to

  any

  of our supported destinations, DuckDB is a de-facto streaming sink!

• Executing dbt models using

  dbt-duckdb

  : With an

  amazing

  contribution by our community, Mage supports executing dbt models in DuckDB SQL!

And new in version

0.9.63

is support for:

• Reading/writing to MotherDuck datasources:

  With DuckDB support comes out-of-the-box support for MotherDuck's serverless analytics platform.

Exporting to a database is as simple as using a data exporter, which can select directly from an upstream block:

You can also

read

from a database in one-line with our DuckDB data loader:

Since DuckDB is implemented in Mage as it's own

IO class

, that also makes DuckDB available as a

streaming

destination. That's right, you can stream from any source

directly

do a DuckDB database.

Last, but certainly not least, you can take this data and execute

dbt models

! Talk about a cherry on top! 🍒

Leveraging the

library, Mage can seamlessly integrate your dbt project with DuckDB sources and destinations, allowing you to leverage our hybrid architecture to execute SQL, Python, and dbt models in the same pipeline. 🤯

You can read more about configuring DuckDB as a storage location

(and the MotherDuck companion

) get started building today! 🦆