The sprawl of providers contains a lot of stale forks and copies, and on top of that its not easy to say “let’s just use John Smiths provider to setup the root credentials of our new K8S cluster because Hashicorp aren’t fixing issues we care about.
I use a few third party providers and the quality difference is wildly variable, the update cadence is sporadic and unreliable on average, and not wanting to build scripts and tools is why maintaining my own terraform providers forked fro hashicorp isn’t really viable time wise either.
Sprawl of providers due to forks for whatever reason is a completely different issue to the resource sprawl you originally called out.
The point remains though: there’s no incentive system for the people who make the implementations decisions re resource granularity within providers to increase that because of this pricing change. For the most part they don’t work at HashiCorp. And even when they do, those abstractions are more often due to the underlying API they’re communicating with.
I think I may have been too subtle with the point about security. Hashicorp’s providers are more trusted because they come from the tool vendor, they are using them in a commercial product and running them on their own hardware as part of terraform cloud. They are all but “implicitly” trusted since you trust Hashicorp code with secrets in order to have Terraform do its job. Yes you can architect a lot of safety layers around credentials and treat Terraform as untrusted, but it’s a sliding scale.
There is an incentive for project management on the AWS, GCE, Azure, Kubernetes, and the other Hashicorp maintained providers, to not prioritise work that reduces the number of potentially chargeable resources.
The first one I thought of was the time provider. It’s a virtual module like the null provider and all it does is put a logical delay into the dependency chain to handle edge cases… it would be all too easy to start assuming that customers use this module more in order to handle functionality that would require more code in other modules. They probably have metrics on resource and module use via terraform cloud (I don’t have the privacy policy and ToS memorised)
How strong the incentive is and if it’s ever really more than a subconscious influence on Hashicorp’s code the code that customers are more likely to use than 3rd party providers… is basically impossible to tell, but the inventive is absolutely there because Hashicorp’s pricing changes have made “number of resources in use by a terraform cloud customers” into a metric that the management will be looking at… the business development, the parts of the company that are responsible for making the money happen, will be measuring this number because it’s obviously important to them now…
And once you begin to measure something as a metric the incentive to game the metrics begins.
Utility providers aside (e.g., `null_resource` which for many use cases you can replace with `terraform_data` in versions >= 1.4), I think you’ve missed the point I was making. Which is:
- HashiCorp employees directly maintain an astonishingly small number of those providers.
- For the most significant ones (e.g., AWS, Azure) they are working in some form of collaboration with the relevant vendors.
- The primary determination on what is broken out into a separate resource or not is based on the API said vendor exposes.
- For those major strategic providers I mentioned, they’ve been working to have the providers programmatically generated so it has little to no human intervention and increases the likelihood of day 1 support for any new service or features.
Incentives aside, there isn’t opportunity to affect things in the way you’re fearful of. The people with the biggest influence on the design of these things, and whether resources are consolidated or decomposed, work at AWS, Microsoft, Google, etc.
I use a few third party providers and the quality difference is wildly variable, the update cadence is sporadic and unreliable on average, and not wanting to build scripts and tools is why maintaining my own terraform providers forked fro hashicorp isn’t really viable time wise either.