Ivan Blinov https://ivanblinov.com/2026-10-31T20:02:15+00:00Ivan Blinov AI agents for a data team, without handing them the keys to productionhttps://ivanblinov.com/writing/ai-agents-for-a-data-team/ 2026-10-31T18:50:11+00:00What I built in 2026 for a 12-engineer billing data team, what the agentic code reviewer actually caught, and which guardrails mattered. Idle CPUs, 60-second queries: a global lock in ClickHouse 23.8https://ivanblinov.com/writing/clickhouse-global-context-lock/ 2026-10-31T18:50:11+00:00How a dashboard widget that needed two numbers took a ClickHouse cluster down for 48 minutes, and why the fix was a cheaper query rather than more hardware. 665 GiB of dictionaries: when ClickHouse dictionaries stop scalinghttps://ivanblinov.com/writing/clickhouse-dictionaries-665-gib/ 2026-10-31T19:59:01+00:00Nine hashed dictionaries quietly grew to two thirds of a terabyte per node, and every reload tried to build a second copy. How we found that only five views used them, and replaced them with joins that read 17 granules out of 282,102. Airflow tasks stuck in "queued" forever after a twelve-second database bliphttps://ivanblinov.com/writing/airflow-tasks-stuck-in-queued/ 2026-10-31T20:01:11+00:00A short network flap between two datacenters left a task in "queued" for two and a half hours, and its DAG with it. The cause is a lost executor event: `SELECT … FOR UPDATE SKIP LOCKED` skips exactly the row it needs. Why restarting the scheduler is the wrong fix. Expired certificates broke production twice, and nothing turned redhttps://ivanblinov.com/writing/expired-certificates-twice/ 2026-10-31T20:01:12+00:00Once as a load-balancer black hole that swallowed 23 of 24 connections for fifteen hours, once as a silent 30-hour replica lag behind a database that kept answering. Both times the metrics existed and the alerts did not. LowCardinality in ClickHouse: I measured the rule of thumb, and it was wrong both wayshttps://ivanblinov.com/writing/clickhouse-lowcardinality-measured/ 2026-10-31T20:02:15+00:00"LowCardinality helps between a thousand and a hundred thousand distinct values, and hurts otherwise." A code review quoted this rule, so I measured it on ClickHouse 23.8. LowCardinality wins at any small number of values, even two, and starts losing badly at about twenty thousand per part. Partitioning by modulo in ClickHouse: the DDL I wrote that made queries 13.7 times slowerhttps://ivanblinov.com/writing/clickhouse-modulo-partitioning/ 2026-10-31T20:01:58+00:00`PARTITION BY supplier_id % 100` sounds like it spreads the load evenly. In practice it destroyed primary-key pruning: a lookup read 64% of the table instead of 3%. Here are the measurements, and why it happens.