Synthèse
Tiered Storage dramatically reduced our storage cost, but it did not eliminate the need for strong local broker capacity. Kafka writes data locally first and background workers copy closed log segments to S3 async. During heavy inbound traffic, those background workers compete with producer writes, replication, reads, and broker housekeeping. If the instance is too small, remote copy can fall behind. Once remote copy falls behind, local disks begin to accumulate data that should have been offloa
Tiered Storage dramatically reduced our storage cost, but it did not eliminate the need for strong local broker capacity. Kafka writes data locally first and background workers copy closed log segments to S3 async. During heavy inbound traffic, those background workers compete with producer writes, replication, reads, and broker housekeeping. If the instance is too small, remote copy can fall behind. Once remote copy falls behind, local disks begin to accumulate data that should have been offloaded.
At high ingest rates, that disk growth can become linear from an operations perspective: every minute of delay adds more data that the cluster must later catch up on. The key lesson was simple: tiered storage is not a substitute for broker headroom. It is a cost and retention strategy, not a license to run minimal local capacity.
Challenge 3: S3 request rate limits during retention changes
· 28 juil.