Infrastructure
What if a dltHub pipeline fails without errors?
A watchdog to keep all dlt pipelines in line
Infrastructure
Posts related to platforms (usually hyperscalers), pipelines (ELT/rETL, streaming, orchestration) and agents
Infrastructure
70 dlt pipelines, zero comments (and no alert when the schema changed)
Nobody documents dozens of tables with hundreds of columns by hand. So an agent does it, INFORMATION_SCHEMA keeps it honest, and Teams pings when a source API adds a column
Infrastructure
dltHub notifies Slack and email. But my org runs on Teams 😩
dltHub ships Slack and email alerts out of the box. My org runs on Teams, so I built the third one myself and wired it into all ~70 pipelines with one decorator shadow.
Infrastructure
7 shared modules for 27 dlt pipelines
Re-using shared patterns in a dlt workspace
Infrastructure
LinkedIn Ads ➡️ Snowflake, with 3-legged OAuth in the middle
LinkedIn's Marketing API needs a browser to authorize. Scheduled pipelines don't have browsers. I store the refresh token in Snowflake.
Infrastructure
I needed 2,110 municipal contacts. So I built a crawler that refuses to guess.
A public contact list sounds easy... until 2,110 websites, four languages, changing staff, and source evidence get involved. 😅
Infrastructure
Backing up Microsoft Planner with dlt and Azure Blob
Microsoft Planner has no native backup export, so I built a declarative dlt job that writes complete Graph snapshots to Azure Blob. 😎
Infrastructure
Your coding agent keeps a diary
Every opencode session leaves a trail in a local SQLite file. dlt turns it into analytics you actually read.
Infrastructure
dlt on a 4 GB pod: loading multi-GB datasets without the SIGKILL
Exit code 137 is dlt's way of saying "your RAM and I need to talk"
Lakehouse
MotherDuck Flights: pipelines an agent can fly, and why dlt belongs in the cockpit
MotherDuck shipped agent-native Python pipelines. The obvious thing to run inside one is dlt.
Infrastructure
OpenCode: a terminal coding agent without vendor lock‑in
Bring your own model, ditch the subscription.
Infrastructure
Perplexity Search as Code: letting models run the search
Why let a search engine guess what context your model needs? Perplexity's new Search as Code hands the reins to the agent.