I wonder how they're going to go about purging all the counters that end up unused once the employee and/or team leaves?
I can see someone setting up a huge number of counters then leaving...and in a hundred years their counters are taking up TB of space and thousands of requests-per-second.
There is a retention policy, so the raw events aren't kept very long. The rollups probably compress really well in their time series database, which I'm guessing also has a retention policy.
If you have high cardinality metrics, it can still be really painful, although I think you will feel the pain initially and it won't take years. Usually these systems have a way to inspect what metrics or counters are using the most resources and then they can be reduced or purged from time to time.
Yes, once the events are aggregated (and optionally moved to a cost-effective storage for audits), we don't need them anymore in the primary storage. You can check the retention section in the article. The rollups themselves can have TTL if the users wish to set that on a namespace. Although doing that, they have to be fine with certain timing issues on when the rollups expire and new events are aggregated. We also have automation to truncate/delete namespaces.
I can see someone setting up a huge number of counters then leaving...and in a hundred years their counters are taking up TB of space and thousands of requests-per-second.