What is this buzz word, Agile?
When we think agile we think of software processes and being able to change the direction of R & D or a project at a rapid pace to allow for the customization of software or the development cycle to provide rapid software features to a client through the act of software development but is that truly how people practice it and does it work?
The article that follows is a bit long but try and read it and understand it and let me know if you disagree with any of my points.
So what does it mean to be agile?
Let start off with a look at the agile manifesto. Agile is not a process and its not even a methodology to develop software.
Some companies try and make money from the buzz word by creating processes and standards around it and they sell courses and certifications relating to the agile process but in the end to be agile is a way of prioritizing some values more than others, its not a process. It’s about being flexible and doing what is needed.
Some of the key values of agile are:
- Individual and Interactions over processes and tools.
- Working software over comprehensive documentation.
- Being able to respond to change vs following a concrete plan.
So with these values in mind you need to find a balance – where you do a little of the one and more of the other.
Can you achieve agility in software delivery through a agile process?
The meaning of the word agile changes depending on who you speak to. When you speak to people involved with the human side of the agile process they will tell you something similar to this. Agile means allowing people to change direction quickly to respond to changing requirements. Leave that task now, the requirements has changed – do this task instead, this task is now more important than that one. Even though the agile teachers, the keepers of the process is in the same organization than the software developers they do not appear to have the same understanding of the word agile.
When you speak to someone in the software development side its seen as a way to deliver software, a method to track and adopt to change to provide the value faster to clients but can you truly deliver software faster if your software is resistant to change? Can swapping this task for another as requirements change make you deliver faster?
Symptoms of an agile environment without agility in the software
- developers unable to do proper estimations. Does anyone know the system anymore?
- developer workload goes up, they spend more and more time doing apparently less and less.
- developer work is not tested when its developed, will test in the next sprint.
- automated testing is dropped in favour of delivery. There are some test that we started in the beginning but its not maintained anymore…
- developers are stuck, they can not easily change something without affecting something else.
- bugs are introduced in unrelated modules.
- management gets involved, usually aggravating the situation by wanting to punish the already overworked developer for not testing their work after they are the ones that are pushing for shortcuts.
- developers jump ship unable to take the heat and the work load.
- suggestion of re-coding the project which is often done and then it goes okay again for a while – repeating the cycle.
Casualties of agility
Anyone that are honest with themselves will agree that in most software houses, agility is often synonymous with cutting corners to gain speed and does not have much to do with changing requirements.
We will come back to fix or test it later, be agile… does this sound familiar?
Some casualties of agility are often the phases ‘planning, design and testing’ this means that we no longer have any analysts or architects in software houses as design and planning is bad, there is no time for automated testing…we can’t afford to have them as they slow things down… and this is where 90% of software companies miss the plot.
Even if the process make provision for design the steps are skipped or applied as quickly as possible to get to the implementation phase to try and speed things up. Once the software is delivered and the client changes his requirements the developer and the process may be agile but how does your software handle the change… it usually doesn’t and then you end up with a patched system that looks like this that no one wants to maintain. As people leave because they no longer want to work on it the knowledge off the spaghetti is lost.
Initially the system above is easy to work on, there are no rules so anything goes but as the system grows it is easier to make mistakes and it becomes impossible to estimate and maintain.
Would this not be easier to work on?
By removing the analyst and architect from the process the integrity and cohesion of the software system is sacrificed in the pursuit of speed and thereby the risk of breaking something within the system increases.
Agility through the act of engineering.
Something that many companies miss is that agility is born out of engineering and not out of a process or sacrificing integrity for the rate of feature delivery. Teams should not deliver features but should be delivering small re-usable components that make up the feature as more and more components are delivered.
Small component can be build quickly as the scope of the component is isolated and the component should not be considered done until its unit tested are part of CI/CD process where it is tested automatically on releases. Components should follow SOLID principles. If requirements change, there is a limited list of components affected which makes estimations and testing easy.
Software systems need to reflect the requirements and values of a company. In the same way that physical objects like mobile phones, motor vehicles were designed according to requirements.
Why do we spent time engineering and testing components of physical objects but don’t want to do it in the virtual world?
If a company needs to design a mobile phone with the non functional requirement of being water proof then it will be one of their primary requirements during the planning and design phase before the design is sent to the factory to be produced. Do you not agree? No manufacturer would try and add water proofing to their phone after it has been constructed. In the same way if you want agility in your business you need to design for it in your software within each component that its made up out off – it is not something that can be easily added on later as an after thought.
The only way to be truly agile is to have a highly customisable software system consisting out of isolatable components that can be re-used or re-ordered and tested automatically to allow for fast development of new use cases or variations with minimum effort.
It should be noted that the decomposition of these components for re-use/ re-order should be done correctly so that they are capable to function by themselves – they should not have any dependencies on any other component. No company can be agile if the software system is not engineered to be agile to start with. This is easily achievable by following SOLID principles.
You can not have agility through a process if the object being manufactured does not allow for agility in its design.
Think of the first motor vehicles that were build at the start of the 19th century, it was a very slow process to deliver a single motor vehicle as one person was assigned to build a car from start to finish. This was called couch building. Usually a single person had to manufacture and fit all the components by hand. (This is similar to companies where agile is applied to the person and not the object being manufactured.) This meant that all motor vehicles built by the same company and even by the same person were unique as each component were unique designed for that car. You could not fit a body panel from motor vehicle A to motor vehicle B. This was a slow process. You could not even speed it up by adding more people to the process as the components influenced the design of each other. The only way possible to speed it up was to remove features… in other words doing less.
Henry Ford came along and revolutionized the industry by building agility in from the engineering point of view. All components and body panels had to be the same and be manufactured upfront according to a certain standard. This required some design and planning.
As a result of this standardization suddenly all motor vehicles had components that could be fitted on any of the same model motor vehicle being built. This was as a result of engineering. Next he sped up the delivery process by implementing a new conveyor belt process (Equal to CI/CD in software development) – this is what he is best known for but most people miss is that this agility could not be achieved without the design and planning he did upfront. This sped up the rate at which motor vehicles were delivered.
But Wayne…. Agile is when requirements change…we don’t care about technical component
Motor vehicles that followed used the same concept and as a result of a little planning upfront by designing standardizes components, different vehicle could be assembled by changing the manufacturing process slightly. This is where the agility was. The agility was in that the engineering allowed for rapid change and deliver. It was not about making people change direction.
The image above shows agility through engineering. By engineering for change ford could produce variations of essentially the same vehicle because the engineering allowed for agility.
How does one build a software system that is agile?
To achieve this one needs one individual or a group of individuals to own / overview the software system and plan and design agility into it – right from the start. Enter the software architect.
To repeat myself, one can not be agile by sacrificing integrity. If agility is a requirement of the software system or the business then it needs to be designed and planned for it.
The software architect sits outside the development process and;
- Receives the requirements for the feature a few weeks before the implementation starts from a BA or client.
- The architect identifies volatilities, re-usable components and creates isolated sub systems.
- From these items tasks are identified to be able to deliver the feature.
- These tasks are then given to the developers to estimate.
- The planned tasks are then given back to the product owner who places this in the backlog.
- The product owner then prioritizes the work and hands the project plan and requirements to the team to implement.
But where is the agility?
This is crazy Wayne! If we have to do the design and planning above delivery will be to slow…We don’t have time for all this planning..
As I have said before the agility is not in the process… In the same way as where Henry Ford did some design and planning upfront which allowed for easy change doing some planning and design upfront will allow for agile software by developing and delivering re-usable components.
By not planning, one will constantly be delivering unique features that takes long to develop instead of working intelligently to delivery re-usable and individually tested components.
As the architect builds up and matures the components and sub systems the delivery rate increases as the system can re-act faster to change. So if a client decides to change his requirements the change can be absorbed easily because it was foreseen and planned as a volatility. So red leather instead of black…no problem, 2 doors vs 4 doors… no problem.
This can obviously not be achieved overnight but by designing a software system based on sub-systems and components the risk is contained and agility is achieved as the components can be re-used and re-ordered to create new use cases or variations of them.
Conclusion
Design & plan your software system around small re-usable components that are testable through automation.
I will end off this article by stating that until a company, its leaders and the business as a whole has bought in and accepted that agility is an act of engineering their only solution to agility will remain to compromise the integrity of the software system and thereby increase their risk.
No process will ever help a software company to have an agile software system.
Amazing video I found that basically says the same thing as my article.
Wayne this is truly a master piece. As the saying goes … the architecture/design of the mouse and the elephant are in actual fact identical. The substitution of design for “process driven agility” leads to the functional decomposition of the system. This leads to an initial perceived success and good velocity, but as the real change creeps in, so does the debt.
The team quickly re-factors and refactors and refactors, only to refactor what has been refactored, as the functional system cannot absorb the change. The source code cyclic complexity grows to such a level, that nobody can anticipate the slightest change. As the product reaches the event horizon, most management will try add more process and more resources, perhaps they will even hire an agile coach.
Alas the agility was lost in sprint 1 when the foundation was cast on a rigid set of requirements . Much like a star implodes under its own gravity, so does the functional project collapse under its own complexity.
Should the system be correctly composed the design will facilitate the change at the foundation level, the more system is worked on the stronger it will become, and the more it will adapt.
But then again, clearly you know all of this.
Well done love your analogies 🙂
Pingback: Agile, the castration of IT - Wayne Clifford Barker