Having managed 10's of thousands of servers and multiple hundreds of configurations at a big web company for many years, I completely see where OP is coming from.
I think a huge problem with puppet/chef is that they try to do it all, ie deployment and OS state management and make it work with vendor specific package management systems.
Unfortunately rpm/debian based package mgmt systems are not well suited for complicated deployment strategies. Most companies I worked at come to a solution similar to:
* Install everything you need in /package_versionstr (except say glibc)
* Point the current version with a symlink
This enables you to do a simple atomic rollback/rollforward and is much easier to reason compared to complicated pkg state.
For simple OS config management (say usermgmt, sysctls), we used a system similar to the ideas expressed by OP.
1. Keep it simple. puppet/chef have horrendous DSL's that make it really complicated to reason about. I shouldn't have
to debug a backtrace 15 levels deep to understand why an useradd didn't work.
2. Server side logic. Don't try to do "intelligent" stuff based on client side state, it will be almost impossible to get it right. All data needed for state needs to derived by group membership. This helps in
* validating all changes upfront
* diffs for state changes across a group of nodes.
3. No orchestration. Except any changes to be applied at any time. This acts as an enforcing function to make your scripts idempotent.
I think a huge problem with puppet/chef is that they try to do it all, ie deployment and OS state management and make it work with vendor specific package management systems.
Unfortunately rpm/debian based package mgmt systems are not well suited for complicated deployment strategies. Most companies I worked at come to a solution similar to:
This enables you to do a simple atomic rollback/rollforward and is much easier to reason compared to complicated pkg state.For simple OS config management (say usermgmt, sysctls), we used a system similar to the ideas expressed by OP.
1. Keep it simple. puppet/chef have horrendous DSL's that make it really complicated to reason about. I shouldn't have to debug a backtrace 15 levels deep to understand why an useradd didn't work.
2. Server side logic. Don't try to do "intelligent" stuff based on client side state, it will be almost impossible to get it right. All data needed for state needs to derived by group membership. This helps in * validating all changes upfront * diffs for state changes across a group of nodes.
3. No orchestration. Except any changes to be applied at any time. This acts as an enforcing function to make your scripts idempotent.