Reading Mike Driscoll’s recent blog on software craftspeople reminded me that this architecture topic has been stewing in my brain for a while now.  Time to write about it.

“Too hard”, “too complex”, “too long” are the persistent reasons behind engineers’ resistance to feature requests or major product pivots.  What I realized is that in every case, it was the software architecture holding us back.  More specifically, the lack of componentization and modularity.

And the pattern spans every experience I’ve had; across lots of different products, across lots of different market sectors, across lots of different architectures (from client-side tools to client/server apps to SaaS/cloud apps), and across lots of different company sizes (pre-revenue to behemoths like SAP and EMC).

Need a new UI presentation tier?  Sorry, that code is co-mingled with the underlying business logic.  Need a new data management tier?  Sorry, the file system is bound to the rest of the code.  Need new business objects to show up in the schema?  Sorry, we can’t split our giant table and it’s already too big to extend.

One can understand how this predicament arose.  When new products are built, what’s required is focus on solving the user problem at hand.  You don’t have time nor money to design for unknown needs and future flexibility.  So why pay for abstraction and modularity without any present-day reason?

The bigger problem is when products mature and the user needs outgrow or diverge from the capabilities of the original architecture.  What to do next?  Re-factor and modularize?  Re-build from scratch?  Limp along by stuffing new features into the code but with huge effort each time?

Nobody knows the magic formula for how to make these decisions.  Re-factoring scares the crap out of engineers lest they “break something”.  After all, by the time this discussion arises, you’ve got spaghetti for code.  And the folks who wrote it might not be around anymore.

Re-write scares the crap out of the business leaders, since it appears to be paying twice for the same product.  And there’s the inherent risk of missing deadlines.  Oh yeah, and you just put your legacy product version on life support so you can afford to staff the engineers on the re-write project.  And you’re losing ground to competitors along the way, since you’ve stopped new feature development to pay for a better architecture.

No wonder products whose architecture devolved to something bad, or started that way, never get fixed.

This vicious cycle is what creates the opportunity for “innovation” in the form of a start-up who has the benefit of a clean sheet of paper: fresh, elegant code using state-of-the art languages, components and tools.  That seems like a wasteful way to solve the problem.

Enter the architect.  If you have a great architect, every problem is reduced in magnitude.

With a great architect, new products have some modularity and flexibility designed in.  A little bit of future-proofing goes a long way. Existing products can be selectively modularized and modernized so the new functional capabilities are delivered without breakage.  And if the time comes for a re-write, you have confidence that all of the lessons learned from the legacy code base are applied to the new design.  Thus, a greater chance of success, especially in meeting a deadline.

So, what makes a great architect?  In many respects, a lot of the same characteristics that make a great product manager: curiosity, an ability to translate what users and salespeople need into technical terms, abstract thinking that enables one to imagine new possibilities, etc.  Of course, the architect also needs the deep technical experience too.

Back to the premise of Mike Driscoll’s article: the best software is being built by people with, dare I say it, experience.  Experience to avoid pitfalls because she messed something up before.  Experience to choose the right tools for the job, much like a fine craftsman that builds furniture, or houses, or bespoke clothing. Experience to know what degree of flexibility to design in, without paying for needless flexibility that feels more like insurance against every conceivable future requirement.

I’ve known some good architects and probably only one or two great ones.  With the great ones, we have had some huge debates thanks to the force of personality that seems to come with great ones.  But in the end, despite the strong personalities, great architects are worth having.  And the great product companies know this, which is why they spend a lot of money on great architects.

I say it’s money well spent.

I’m proud of those who have played such a big role in building the company to this point.  And I’m proud to have played a small role too.

Time to sit back and watch the ticker with fingers crossed.

Coincidentally, a couple weeks ago a long-time friend sent an email to numerous people in response to President Obama’s State of the Union address.  I don’t recall ever getting an unsolicited email from him about politics in the 15 years I’ve known him.  But it certainly was a rant.

Consider the emotions in some of the words and phrases from his email:

• “A brutally complex issue reduced to a nonsense issue”
• “I want to barf”
• “…it is the duty of everyone to work for his defeat in November”
• “A 2nd Obama term, totally untethered from public opinion will be a total disaster.  I hope all of you can take some role in ensuring his defeat in November”

Which drew the following response from someone else:

• “If we are now outnumbered by liberals, democrats with limited brain functionality, union workers, the uneducated, and young voters, then the chances are very real that the current “President” could win a second term”
• “And if that happens, then the chances are very real for a second American Revolution – this time against our own Congress, which has nearly completely caved in and lost its backbone”
• “If we do win, however, I believe we need to somehow change our system permanently so that Congress DOES WHAT WE SAY TO DO and nothing else, or we will vigorously prosecute them and send them to prison”
• “I fear the continued existence of the USA is in doubt if we don’t! One answer is to decentralize our Federal government and send our representatives and senators back to the states, where they are closer to home and we can watch them more closely. Please think hard about this”

Subjectively, it seems like a pattern.  Something has changed for these persons, and probably not for the better. Is it the stagnation of real income growth in the middle class?  The unrelenting pace of change driven by globalization and the strains of keeping pace?  I won’t speculate other than to say that these are very strong emotions, which begs the question why?

It’s now the “quiet period” according to our friends at the regulators, so I won’t comment beyond that.

The backlash to this announcement was pretty huge.  To the point that Netflix had to “undo” the announcement.  Talk about a black eye.

The timing of the decision is certainly up for debate given how customers reacted.  So let’s say it was premature.  But how premature?

I still contend this is the right decision.  Eventually.  Just like the Pony Express was rendered obsolete by the telegraph, so too will mail-order media delivery be made obsolete by streaming delivery.  Who would believe otherwise?

It’s darned hard to time such changes, however.  Most companies never make the leap at all, hence books like The Innovator’s Dilemma.  For those who have the bravery to do so like Netflix, the timing is a perilous choice.  Too soon?  Investors punish you for cannibalizing current revenue and alienating happy, paying customers.  Too late?  You’ll probably never catch up.

If I knew the answer, I’d be a rich man.  My sense is that it’s impossible to make a formula.  Instead, it’s about getting the whole set of stakeholders on the same page.  So when it’s time to jump off the cliff, everyone is holding hands.  CEO, executive team, Board of Directors, large investors.  Not a small task.  At lest you’re trying, Reed.

I was reading the interesting but flawed article on Techcrunch about “product guys”.  As in, Product Managers.  Let’s at least start with the right title.  Cripes.

One of the author’s contentions was that good product managers must have coded in their past, in order to properly collaborate and empathise with their engineer co-workers.

This assertion made me think about where good product managers came from.

The first answer is that product managers don’t come from product management.  As in, there is no university degree for product management that is a mainstream path into product management as a job.  (Yes, there are a handful of product design degrees, but sadly at present a tiny portion of product managers carry this pedigree.)

Therefore, product management is more of a trade, with the apprenticeship starting elsewhere.  Good product managers come from many places.

In enterprise software, pre-sales engineers are a common path to product management.  Why?  They spend all day demonstrating their products and even deploying them in customer environments.  Maybe pre-sales engineers get so frustrated with the product that they want to take it upon themselves to fix the issues.  It’s also the case that pre-sales travel can burn you out, for which a headquarters job can be appealing.

Product managers can also come from customer care.  People who spend all day troubleshooting users’ issues can develop an acute sense of what product features work, don’t work, and why.

Product managers can also come from engineering, especially those who tend to interact with product managers such as architects.  Again, the motivation can be rooted in frustration and the sense that “I can do this job better”.  Or, the realization that coding products isn’t a long-term career goal.

I have also seen great product managers come from quality assurance.  The attention to detail that QA instills helps condition product managers to “sweat the details” of a great user experience.

Every path into product management – pre-sales, engineering, customer care, quality assurance – brings a valuable facet to the product manager job.  And experiences that product managers per se wouldn’t otherwise have to the same depth.

Which leads to my final point.  No matter what your heritage, you are only bringing one facet of the role based on past experience.  Thus, product management is a journey of curiosity, learning, collaboration, trial & error, persistence, passion and so forth.  Product managers who demonstrate those traits tend to succeed.  And the rest probably less so.

This post is a bit long, but it ends with why your personal life will be spied upon by your company’s competitors.  Curious?  Read on.

Information – data turned into meaning – is going to (further) disrupt just about every industry there is.  Many of the strong will become weak.  Tiny upstarts, like Davids, will topple Goliaths.  And this could all happen at the expense of your personal privacy.

I know, I know, the “information revolution” has been predicted for decades.  Except that like many predictions, exactly when & how the predictions come true will differ from what’s first assumed.  “Video calling” has been predicted as inevitable since the 1950’s.  But nobody saw Skype as the inevitable means for it to come true.

To see the future, let’s look at what has happened on Wall Street and how the same is about to happen in other industries.

What’s happened on Wall Street

Fortunes on Wall Street are made through arbitrage.  Years ago, the winners were those who had better research teams: who had the better information about a company’s stock?  Currency arbitrage followed: who can spot temporary asymmetries in currency prices and execute a trade the fastest?

Across every tradable instrument, the arbitrage game on Wall Street has moved from high-latency (you know something I don’t, for days or weeks at a time) to near-zero-latency.  New fiber optic networks have been built to shave milliseconds from the average trade.  And entire data centers have been relocated to close proximity to these new networks.  Whatever advantage you have must now be exploited in real time.  Or it’s not an advantage.

As the arms race of zero-latency arbitrage has unfolded, two other trends have been its critical enablers.  The first is data acquisition.  Every trade on every exchange can be captured and analyzed, alongside reams of other data about the companies, markets and countries those pertain to those financial instruments.  The scope and scale is increasing by orders of magnitude.

• An example of scope: the latest algorithms even ingest Twitter feeds in real time to discern investor sentiment.  Including your latest Tweet about some stock you like
• An example of scale: banks and securities firms store more information per company than any other industry (see this great report from McKinsey)

The second enabler is the ability to make sense of the collected data.  No longer do freshly-minted MBA’s toil into the night to build Excel models with quaint calculations like “EDITDA” or “Return on Invested Capital”.  Today, PhD statisticians build algorithms so complex that they themselves have trouble making sense of them in use.  Michael Lewis’ “The Big Short” is a great read on this topic.

From Wall Street to Main Street?

Wall Street started its journey by going on a data collection binge, from which it developed its algorithms, from which it automated its trading such that milliseconds mattered.

It’s happening now in online consumer businesses.  Wonder why Google, Facebook et al are under such scrutiny for collecting your data to the point of privacy invasion?  Because they understand that data is the raw material that feeds their statisticians, that feeds their algorithms, that feeds their money-making.

Except “money-making” in this case is the price they can charge an advertiser for an ad on one of their web pages.  The more relevant the ad to a consumer’s interests, the higher a premium the ad will command.  How to discern consumer interest?  Profile the heck out of them.

It’s happening elsewhere too.

The “Measurement Wars”

New companies, or ones that successfully reinvent themselves, will start their innovation and disruption journey by gathering reams of data, then finding the relationships between that data.

The forthcoming data acquisition binge is going to amplify online privacy issues.  Collecting competitive intelligence will border on spying on other companies’ employees.

It will also tempt companies to sell their internal data to others.  Could we see new “data merchants” emerge?  Would you like to know the kilowatt-per-hour energy consumption for every household in America?  Someone would, and would like an electric utility to sell them that data for some commercial advantage.

Orwellian, or progress?

Every boundary that separates individuals, companies, cultures and countries will be subject to elimination or reduction.  Our ability to learn, empathize and understand differences across these boundaries will be exponentially enhanced.  Which is good.

The ability to use that same understanding to exploit others will also increase exponentially.  Which is bad.

My take?  We have dealt with many past innovations that could be used to exploit someone or something.  Each time, a new equilibrium was established and humankind moved on.

But it was the period of rapid transition, and resulting destabilization, that was the most dangerous.  We are in such a period now, I reckon.

Governments must understand the coming hunger for massive data collection, and act to mitigate the risks, if we are to emerge on the other side unscathed.  Or even emerge better off.  But what are the risks, and what are the remedies?

Spy vs. spy

We’re seeing the privacy issue play out now in the consumer sector.  Facebook has been scolded by the Federal Trade Commission for its privacy abuses.  Politicians in the European Union search for ways to legislate much more stringent consumer protections.  I won’t cover this ground because the media has done so many times over.

But we have yet to come to grips with the implications of how companies will compete with each other in the Measurement Wars.  The old rules were about the theft of intellectual property.  But what about when a company profiles its competitors’ employees?  Such as their Tweets, Facebook activities or LinkedIn profiles?  Individual persons will be deeply profiled as part of compiling a dossier of competitive intelligence gathering.

This is where it gets creepy.  Heck, if you were imaginative, you might think that things I write about in my blog pertain to issues I experience at work.

And you wouldn’t be all wrong.

Credits:

I have borrowed from the great work done by others.  In particular, Michael Lewis and McKinsey as cited previously.  Also, O’Reilly Media for their extensive coverage of the underling technologies of “Big Data” and their Big Data’s use in consumer online businesses.

I recently came face-to-face with the murky issues of social media and hiring in the workplace.

Days before the person’s interview, a candidate to join my organization started sending Twitter messages addressed to my username.  So I got a notification from Twitter that I was “mentioned” in someone else’s Tweet (Twitter is designed in such a way that another user cannot contact me privately unless I choose to “follow” that user explicitly.  Which I wasn’t).

What was strange was that the person was addressing questions to me about the upcoming job interviews.  Asking something about my company’s job benefit package if I recall.  In other words, questions that could, and should, have been addressed in the setting of an interview.  Even more curious is that the person sent those same Tweets to another employee with the same questions.  And that other employee didn’t even work in my department.

A flood of questions came into my mind…..

• Could I read this person’s Tweets?
• Should I read them?
• Are somebody’s Tweets in the public domain?  With over 100 million users, and the ability of any user to see the Tweets of any other, one could argue yes
• Are things in the public domain fair game for evaluation in a hiring process?  Beyond the obvious off-limits discriminators such as age, gender, sexual preference, etc.
• Even if such information wasn’t in the public domain, was it still something I could, or should, use in my evaluation?  In many parts of the world, employers are entitled to collect information about an employee beyond what is offered by the employee him/herself

I found that the answers to these questions were far from obvious.  I suspect it will be years before the law and business behavioral norms will catch up with these issues.

I won’t tell you what I did in answer to these questions, out of respect for the privacy of the individual and the fear of opening up a legal can of worms.

What I will say is that I approach online life as if everything I say can be read by others, and thus used by them to form some judgment of me: my blog, Facebook account, Twitter, LinkedIn, etc.  Is this a form of self-censorship?  Yes, to a degree.  I guess there’s still a role for offline communications and “antique” forms of online communication like email or SMS.

There are also some very highly regarded places I have yet to visit, notably La Degustation Boheme Bourgeoise.

That said, here are the places I’ve been to and like, in no particular order:

V Zatisi.  High-end, nice “design-your-own” tasting menu.  They match wines well with the courses.  Professional and attentive service.  A good place for a special night out.

The Pind.  Quickly developing a reputation for the best Indian food in Prague.  I’ve liked every dish thus far, and have been exposed to a few selections on the menu I hadn’t seen before. Which is good, as the variety of food from India is infinite and worthy of constant exploration.

Sokolovna.  Nicely done traditional Czech cuisine.  A really good chicken soup.  Warm decor.  Inexpensive.

Las Adelitas.  The place to scratch the itch for Mexican cuisine and margaritas.  Anything ”el pastor” there is very good.  The horchata is delicious if you don’t feel like an alcoholic beverage.

Artisan.  A wonderful higher-end restaurant tucked away on a side street, far from any tourists. Very good service and value for money.  The outdoor patio in the summer is intimate and really pleasant.

Sansho.  Creative southeastern Asia cuisine with some other influences thrown in.  A fixed price, set menu comprised of many bold flavors.  The staff is wonderfully accommodating and will tailor the menu to dietary needs or even diner preference. Not a hint of pretense in the place; probably by design.  Perhaps amongst the most expensive in Prague.  But satisfying nonetheless.

Grosseto Marina and Vinohrady locations.  This small chain reminds me of Paparazzi in Boston: very consistent, decent Italian food with good value for money.  Tasty thin crust pizzas.  The Marina location has a great view of the castle.  The Vinohrady location has a nice summer garden.

Noi.  Solid Thai food in a setting with nice decor.

Sasazu.  Not unlike the cuisine at Sansho: pan-Southeast Asian influences.  But in every other respect quite different: dramatic decor inside a transformed warehouse, featuring huge light fixtures, bold colors and trendy music.  A place where the vibe is as much a reason to go as the food.  And the food is quite good.

Mirellie.  Mediterranean cuisine with a bias toward Italian.  Good value for money.

Sahara Cafe.  Wonderful decor and some strong North African/Middle Eastern food choices.  The hummus with all of its “enhancements” is my favorite.  A great outdoor terrace for summertime.

El Barrio de Angel.  One of the better places for steaks.  The chimichurri sauce is really good paired with a medium-rare or medium-cooked steak.  Service and atmosphere are nothing special.  Nor is the wine list.  But definitely a place to satisfy the carnivore craving.

Vytopna.  This one is my son’s vote for best restaurant.  Of course, he’s 4 years old and loves this restaurant because of the model trains that deliver drinks to your table.  If you have kids, well worth a visit.  The Czech cuisine is decent.

That’s all for now.  What do you think?

A future post: Prague watering holes.

He embraced it, by seeking a merger between his baby Pure Software (he was the founder) and Atria.  Soon thereafter, the combined company was part of Rational and is now a product suite at IBM.

Reed is at it again.  Netflix is separating its DVD-by-mail service from its live streaming service.  I loved the quote from Engadget today:

What really happened here is quite simple: Reed Hastings just put a gun to the side of his DVD-by-mail business and pulled the trigger. Given that he aimed for the ankle, though, it’ll probably take a while for it to completely bleed out. But hey — proactively putting a fading business out of its misery sure beats bleeding for it on the balance sheet.

Joseph Schumpeter and more recently Clayton Christensen have written about creative destruction and disruptive technologies.  Reed is one a few high tech leaders that has the courage to implement what the rest of us know: do unto oneself before it’s done to you.

Here’s the bit that stops others from doing the same: Netflix’s share price, and perhaps even near-term revenue, could suffer.  For most of the industry, one can’t tolerate the thought of taking a step back to take two forward.  And hence the balk at such bold moves, fearing the reaction of others.  Like shareholders or pundits.

Perhaps the definition of “courage” is not fearing the reaction of others?  Game on, Reed.