"In [Giles Bowkett's] words, in it's best case scenario, Scrum's a dog-and-pony show. But that best case scenario is rare. In the much more common case, Scrum covers up the inability to recruit (or even recognize) engineering talent, which is currently one of the most valuable things in the world, with a process for managing engineers as if they were cogs in a machine, all of equal value."I'm sure there are places out there using Scrum successfully, but I have never seen them. It seems to be a methodology using the trappings of Agile to hide that it actually comes from the fake world of scientific management. In that world, to achieve improvement, you must have The Metrics. The Metrics govern all processes. It doesn't matter if something is actually understood, it only matters that it is being Measured, and that the Measurement be Tracked.
Hmm. This turned into a bit of a rant post, so I need to look at what I can offer as a solution. In my (again, limited) experience, you get the best results when you trust your people. Of course you can't do it blindly. There must be accountability if something or someone fails to deliver, and management must be aware of and take into account individuals' interactions with other and their own desired goals. It certainly isn't the easiest path, but boy can it go gangbusters when it works. And if your team isn't capable of functioning smoothly together without a convoluted multi-process formal framework, then there are other fundamental issues that need to be solved. Is it too big? Is there a poison person? Are they being mismanaged? Is the end goal of the project unclear or shifting?
More and better minds than me, like Mr. Bowkett and Mr. Eckle above, have pondered the workings of a successful teams and successful businesses. Many trees have been felled and electrons spent in sending forth their conclusions. Yet it remains a hard problem to tackle, and like all people (and software) problems, the solutions will be at least somewhat unique every time.