In my many years doing product management or managing the function, the number one blocker to getting the features I want (and the user needs) is……software architecture.

Reading Mike Driscoll’s recent blog on software craftspeople reminded me that this architecture topic has been stewing in my brain for a while now.  Time to write about it.

“Too hard”, “too complex”, “too long” are the persistent reasons behind engineers’ resistance to feature requests or major product pivots.  What I realized is that in every case, it was the software architecture holding us back.  More specifically, the lack of componentization and modularity.

And the pattern spans every experience I’ve had; across lots of different products, across lots of different market sectors, across lots of different architectures (from client-side tools to client/server apps to SaaS/cloud apps), and across lots of different company sizes (pre-revenue to behemoths like SAP and EMC).

Need a new UI presentation tier?  Sorry, that code is co-mingled with the underlying business logic.  Need a new data management tier?  Sorry, the file system is bound to the rest of the code.  Need new business objects to show up in the schema?  Sorry, we can’t split our giant table and it’s already too big to extend.

One can understand how this predicament arose.  When new products are built, what’s required is focus on solving the user problem at hand.  You don’t have time nor money to design for unknown needs and future flexibility.  So why pay for abstraction and modularity without any present-day reason?

The bigger problem is when products mature and the user needs outgrow or diverge from the capabilities of the original architecture.  What to do next?  Re-factor and modularize?  Re-build from scratch?  Limp along by stuffing new features into the code but with huge effort each time?

Nobody knows the magic formula for how to make these decisions.  Re-factoring scares the crap out of engineers lest they “break something”.  After all, by the time this discussion arises, you’ve got spaghetti for code.  And the folks who wrote it might not be around anymore.

Re-write scares the crap out of the business leaders, since it appears to be paying twice for the same product.  And there’s the inherent risk of missing deadlines.  Oh yeah, and you just put your legacy product version on life support so you can afford to staff the engineers on the re-write project.  And you’re losing ground to competitors along the way, since you’ve stopped new feature development to pay for a better architecture.

No wonder products whose architecture devolved to something bad, or started that way, never get fixed.

This vicious cycle is what creates the opportunity for “innovation” in the form of a start-up who has the benefit of a clean sheet of paper: fresh, elegant code using state-of-the art languages, components and tools.  That seems like a wasteful way to solve the problem.

Enter the architect.  If you have a great architect, every problem is reduced in magnitude.

With a great architect, new products have some modularity and flexibility designed in.  A little bit of future-proofing goes a long way. Existing products can be selectively modularized and modernized so the new functional capabilities are delivered without breakage.  And if the time comes for a re-write, you have confidence that all of the lessons learned from the legacy code base are applied to the new design.  Thus, a greater chance of success, especially in meeting a deadline.

So, what makes a great architect?  In many respects, a lot of the same characteristics that make a great product manager: curiosity, an ability to translate what users and salespeople need into technical terms, abstract thinking that enables one to imagine new possibilities, etc.  Of course, the architect also needs the deep technical experience too.

Back to the premise of Mike Driscoll’s article: the best software is being built by people with, dare I say it, experience.  Experience to avoid pitfalls because she messed something up before.  Experience to choose the right tools for the job, much like a fine craftsman that builds furniture, or houses, or bespoke clothing. Experience to know what degree of flexibility to design in, without paying for needless flexibility that feels more like insurance against every conceivable future requirement.

I’ve known some good architects and probably only one or two great ones.  With the great ones, we have had some huge debates thanks to the force of personality that seems to come with great ones.  But in the end, despite the strong personalities, great architects are worth having.  And the great product companies know this, which is why they spend a lot of money on great architects.

I say it’s money well spent.