106 articles
The real measure of Flock is never the event alone. Another big part is what contributors do once they get home. In the 30 days since Prague, the conversations that started in corridors have continued in Fedora Discussion threads. A fuller picture of survey responses and feedback will be part of a future Fedora Council town hall video meeting later this summer.
Our migration to Kafka was more than an API change; it fundamentally altered how we stored and accessed stream data. Storing billions of events on high performance EBS (Elastic Block Store) volumes would be costly. This made Kafka Tiered Storage our key architectural feature. Tiered Storage splits the storage of Kafka topic data into two tiers: TierDescriptionDetailsLocal Tier (Tier 1)Utilizes local EBS volumes on the brokers to store “hot” data.Configured for the last 5 minutes. Ensures real ti
Tiered Storage dramatically reduced our storage cost, but it did not eliminate the need for strong local broker capacity. Kafka writes data locally first and background workers copy closed log segments to S3 async. During heavy inbound traffic, those background workers compete with producer writes, replication, reads, and broker housekeeping. If the instance is too small, remote copy can fall behind. Once remote copy falls behind, local disks begin to accumulate data that should have been offloa