My company attempted this switch in March or April, and it’s been a disaster – just wondering if anyone’s managed it successfully.
The hardest thing to overcome, I think, is that we promise certain things to customers on a specific date, and they’re non-negotiable, so pretty much everything gets chucked when we get through two sprints are realize we’re not going to hit the deadline. At that point, we’re not handing stuff to QA, not trying to have a workable demo at the end of each sprint, etc, because all we really care about is hitting that final deadline.
But there are small annoyances, too. Like, management loves metrics, so now they want to know how many bugs are caught and fixed in a sprint before they get to QA, which is a crazy metric, and wastes everyone’s time. Or that when we have the sprint reviews where we’re supposed to demo the current state of code, to get input if anyone doesn’t like something, management ditches that part so that they can focus on numbers & dates. And QA doesn’t like to test unfinished code, so they want to get the product at the end instead of testing along the way.
Basically, it feels like waterfall except there are more meetings and less design.
In my company, agile programming is being used as an excuse for having zero requirements, an unclear vision, and an unrealistic delivery date. “Oh, we couldn’t get the prototype out. Our dev team isn’t agile enough.”
I’m sure there are situations where agile development works great - smaller dev and company environments, a tactical non-critical approach to discovery and bug squashing - but too many non-analytical types are latching on to the term as a reason to excuse them from any hard thinking. This results in many false starts, hours lost reinventing the wheel, and an end result of a pile of shitty, poorly-optimized code.
More or less the entire Internet industry uses some kind of Agile and everyone else throughout software wants to or is “introducing Agile elements” or some such bullshit. So for example, my last project at ITA Software and Google was more Agile than not, with small iterations and rolling planning, which I think is the heart of the concept.
But as you know, there is an infinite range of Agile-ness. When consultants come in and tell you about procedures to learn, and when you feel like you are doing things because the procedure tells you to, and when you have hired unproductive quasi-managers who don’t code and don’t manage, then you know you have gone astray.
I think that for many products and code-based services – especially Internet and Cloud things, and anything else that needs to move quickly and react to changing conditions – small iterations and frequent deployments really is a good idea. Naturally you have to adjust traditional approaches to get this to work, and when you’re done, you are mostly Agile, even if you didn’t read any books or hire any consultants.
But when you have these mysterious scrum-masters taking up meeting time, or if you can’t do enough testing for your deployments, or if you are building lots of little quick low-value features because they fit in your iterations and you can’t find time for major new capabilities, you have to look at what you’re doing and take a step back.
It too quickly becomes a tool for middle management to micro-manage and hide inadequecies in the planning department. Analysis, design and testing are usually the first casualties.
I think that there is plenty of scope for iterative development and responding to changing business using any of the more traditional methodologies.
The hardest thing to overcome, I think, is that we promise certain things to customers on a specific date, and they’re non-negotiable, so pretty much everything gets chucked when we get through two sprints are realize we’re not going to hit the deadline. At that point, we’re not handing stuff to QA, not trying to have a workable demo at the end of each sprint, etc, because all we really care about is hitting that final deadline.
If your management team acts like children and refuses to budget on both date, featureset, quality, and resources, yeah, they get what they deserve.
Like, management loves metrics, so now they want to know how many bugs are caught and fixed in a sprint before they get to QA, which is a crazy metric, and wastes everyone’s time.
…
And QA doesn’t like to test unfinished code, so they want to get the product at the end instead of testing along the way.
You need a new job; none of these problems have to do with agile as much as “badly run business.”
I’m not going to say too much as it pertains to my current job and I don’t want to say too much to get myself into trouble.
The company I work with is a brilliantly run business but scrum does force companies into the trap of behaving like scrum is an alternative to effective project management.
The reality is that you should have long, medium and short term plans that are properly specced. “Agility” should be catered for because that is a requirement in our industry. It should definitely not be what you revolve your entire development cycle around.
Iterations, more often than not are self imposed and often unneccesary deadlines that only serve to put a team under pressure. Sadly middle management loves to be able to quantify and reduce everything into a number they can paste into a spreadsheet.
This could so easily turn into a nasty rant since I’ve watched a good company make these mistakes.
Just from personal experience, I think that a lot of the agile methodologies out there are very effective for smaller team projects (small enough that everyone involved knows each other and works together), but for large organizational projects you need far more process and hard hand-offs than agile methodologies provide.
I don’t pretend to be an expert, and maybe there are large organizations out there tackling these issues, but agile in my experience assumes that you have a group of people that are able to put aside enough differences to agree on and focus on distinct short and long term goals and that can be a real challenge in larger organizations.
I think the industry in general loves the idea of having a right way to do everything, but the reality is that large organizations have different needs than small ones. I think agile is about trying to get the most productivity out of a group with the idea that you’ll figure out solutions organically on the way. Having been on both sides, I struggle to see how that would match with a business with strict specific contractual requirements with delivery penalties. Sometimes in some businesses you need people to just do something X way because it will meet a deadline and the contract says to do it X way despite more elegant or efficient ways and I haven’t found agile a good match for the environment.
All that said. While I respect project management’s role in large projects, your company sounds like it has a culture focused on the wrong things. If you’re not in a position to affect change and you’re unhappy with how things are going, I’d suggest at least looking for what else is out there. I made a change in large part due to similar frustrations and I’m very happy I did.
You programmers aren’t the only ones who have to drink the koolaid. QA has to drink the koolaid. Project management has to drink the koolaid. Program management has to drink the koolaid.
I have no idea how we’d ever get quality product out on time without milestones, bug juries, and unit testing.
We’ve adopted elements of it (mainly the sprints, early QA involvement, and less detailed early design), with mixed results. It didn’t work so well when we did a larger project that required collaboration with our German lab. With no control over their resources and allocation, we couldn’t always get people working on related components at the same time, so it devolved back into working on large, separate components independently and then spending a lot of time later on trying to integrate them, which is pretty much anti-agile…
It was fine when it was just our own small internal teams on smaller projects though, where we have a lot more flexibility in scheduling and cutting features.
Many companies decided to go “Agile” and then just fall into “iterative waterfall.” This happens all of the time and I’ve seen it first hand. The problem, as mentioned above is that you CANNOT have a deadline and a locked feature set. In the first place, it’s stupid to even promise something months out because the whole idea of agile mentality is that the needs will likely change by then. Secondly, it’s just setting yourself up for disappointed customers. Why not keep them in the loop the entire time so they see constant progress?
Also of note, QA not wanting “unfinished code” is missing the point. The goal is to delivery small components of FINISHED code, that iterate on one another. For example, let’s say I’m developing a single web page that has a grid of editable business data. Your iterative goals could be:
A web page with a grid with no real functionality, just look & feel.
Page shows data from a database.
Data is editable and can be saved.
Validation logic is applied to data as it is saved.
Search and filter functionality is added
Add the 4 columns the business realizes they need after seeing demos.
Those don’t have to be individual sprints, but those are an example of a series of deliverable, finished, unit-tested code you can hand off to QA and roll into a User Acceptance/Staging environment for your customers to see and try out. Step 6 is the greatest beauty of agile - you can adapt to things customers didn’t even realize they wanted. For all of this to work well (and it can!) it needs buy in from every single person involved, all the way up to managers AND customers. Your managers sound like they are several dollars short of a clue, especially with your crazy talk of metrics.
Side topic: There is an offshoot of the “Agile” topic which is referred to as “Lean.” One of the core ideas is that you are finished with an entire project when the minimum viable product (MVP) is completed in production. It may not have all the bells & whistles but it’s enough to meet the overall goal - at that point, you re-evaluate if another release should be done to add some of those features in. The goal of Lean is to get a completed project in the shortest amount of time.
We’ve been starting this at my office, and to some extent it helps bridge the gap between scrum and waterfall. We do our release planning with the minimum features so that the sales and business units can have some idea what the big picture items are for the next version, but it also allows us to still be flexible enough to respond to changes (or, bring forward shelved items). I thinks its really helped get our product and project managers on board with agile.
My company has had over a hundred successful on-time rollouts of major projects using an Agile methodology. Instead of managers we have productive leads that handle time management and assignment of work from an actual perspective of understanding what resources (time and otherwise) the tasks will actually require to be completed and also bridge the gap between technical realities and business needs.
The development team works on-site alongside the clients - meetings to discuss nearly anything can be put together with very short notice (many things within a few minutes, but generally within a few hours or days at most).
As Jason pointed out above, it sounds less like you guys are doing an actual Agile methodology and more like management read a book that mentioned Agile and thought it sounded nifty. You’ve got non-productive managers running the show at multiple levels rather than just providing a high-level project goal and timeframe and running interference with the clients.
It probably feels a lot like a crappy version of waterfall because what you’ve described is a crappy version of waterfall.
I’ll echo what others have said, and suggest that Agile development methodologies can work great and lead to efficient development of quality software if implemented properly. Unfortunately, implementing them properly is really hard because it requires a mindset change from project managers, development managers, QA managers, etc. If any of those parties doesn’t buy in, and then flexes management muscle to short circuit the process by “mandating” or micromanaging activities then everything goes bad fast.
There are a few good ideas from agile that any software company ought to be implementing, such as iterative development with continuous integration and early test involvement, but doing the whole thing properly is rare.
I am a tad bitter on this topic because I led our companies switchover to agile development methods a few years ago. At first it was all hunky dory and we had well groomed backlogs feeding self managed teams with good buy-in towards our final vision. Then certain managers felt that they had to contribute to improving productivity by mandating a maximum time for sprint planning meetings (meetings are wasted time after all), eliminating retrospectives and demos (more waste!). Oh and they mandated that only developers from certain departments could work on certain code, even if other team members were fully qualified, because department managers were bickering over who “owned” what code.
Needless to say, what we do nowadays bears very little resemblance to agile development.
That’s a tricky one, because if content and delivery deadline are set then the only way to vary progress is by quality. The trick there is to get your sales/business people in on the agile mentality as well and get them to sell things to customers with slightly more flexible terms. Instead of price $X, date $Y, instead think about a target delivery dates with penalties for missing.
I do realize that there are some industries where that simply won’t fly because the clients are massive and able to dictate terms to vendors. So the if client has put out a RFP saying they want X by Y then that’s how a software vendor has to take the job.
My last company had a 30 step iterative waterfall process. They stuck to it until it got half way through development and then the process always devolved to “just get shit done”.
I’d have a hard time going back to a company doing waterfall or anything that wasn’t agile-like. It’s a code smell.
Is “Reverse Waterfall” a real thing? We’ve joked about in the past, in the office, but then I googled it and saw it listed on someone’s LinkedIn profile.
The most important thing about executing agile is not the ceremonial stuff of meetings and charts and product owners and scrummasters, its the practical discipline of tight iterations, continuous integration, good unit test coverage, designs geared toward refactorability, self organizing teams, etc. You can easily start implementing this stuff without announcing to the world that you’re switching to agile and before you know it you’re more agile than not.
