Data-Aware Scheduling in Airflow with Datasets
Replace sensor-based waiting with Airflow Datasets. Build producer-consumer DAGs, combine time and data triggers, and design dataset URIs for production.
·12 min read · #airflow#datasets#scheduling
#scheduling
5 posts · page 1 of 1
Kubernetes Taints and Tolerations: A Complete Guide
Control which pods can schedule on which nodes using Kubernetes taints and tolerations. Covers effects, use cases, and common pitfalls.
·6 min read · #kubernetes#taints#tolerations
Kubernetes Node Affinity and Anti-Affinity: A Practical Guide
Understand how nodeAffinity and podAntiAffinity steer the scheduler, with real YAML, hard vs soft rules, and the performance traps to avoid.
·4 min read · #kubernetes#scheduling#affinity
Kubernetes Resource Requests and Limits Explained
What requests and limits really do, how they interact with the scheduler and the OOM killer, and how to set them without overpaying or getting throttled.
·4 min read · #kubernetes#resources#scheduling
Linux Cron and systemd Timers: A Practical Comparison
Run scheduled jobs on Linux with cron or systemd timers. How they differ, when to choose each, and recipes that survive reboots and log rotations.
·4 min read · #linux#cron#systemd