Category Archives: Technology trends

My hero, the Software Architect

In my many years doing product management or managing the function, the number one blocker to getting the features I want (and the user needs) is……software architecture.

Reading Mike Driscoll’s recent blog on software craftspeople reminded me that this architecture topic has been stewing in my brain for a while now.  Time to write about it.

“Too hard”, “too complex”, “too long” are the persistent reasons behind engineers’ resistance to feature requests or major product pivots.  What I realized is that in every case, it was the software architecture holding us back.  More specifically, the lack of componentization and modularity.

And the pattern spans every experience I’ve had; across lots of different products, across lots of different market sectors, across lots of different architectures (from client-side tools to client/server apps to SaaS/cloud apps), and across lots of different company sizes (pre-revenue to behemoths like SAP and EMC).

Need a new UI presentation tier?  Sorry, that code is co-mingled with the underlying business logic.  Need a new data management tier?  Sorry, the file system is bound to the rest of the code.  Need new business objects to show up in the schema?  Sorry, we can’t split our giant table and it’s already too big to extend.

One can understand how this predicament arose.  When new products are built, what’s required is focus on solving the user problem at hand.  You don’t have time nor money to design for unknown needs and future flexibility.  So why pay for abstraction and modularity without any present-day reason?

The bigger problem is when products mature and the user needs outgrow or diverge from the capabilities of the original architecture.  What to do next?  Re-factor and modularize?  Re-build from scratch?  Limp along by stuffing new features into the code but with huge effort each time?

Nobody knows the magic formula for how to make these decisions.  Re-factoring scares the crap out of engineers lest they “break something”.  After all, by the time this discussion arises, you’ve got spaghetti for code.  And the folks who wrote it might not be around anymore.

Re-write scares the crap out of the business leaders, since it appears to be paying twice for the same product.  And there’s the inherent risk of missing deadlines.  Oh yeah, and you just put your legacy product version on life support so you can afford to staff the engineers on the re-write project.  And you’re losing ground to competitors along the way, since you’ve stopped new feature development to pay for a better architecture.

No wonder products whose architecture devolved to something bad, or started that way, never get fixed.

This vicious cycle is what creates the opportunity for “innovation” in the form of a start-up who has the benefit of a clean sheet of paper: fresh, elegant code using state-of-the art languages, components and tools.  That seems like a wasteful way to solve the problem.

Enter the architect.  If you have a great architect, every problem is reduced in magnitude.

With a great architect, new products have some modularity and flexibility designed in.  A little bit of future-proofing goes a long way. Existing products can be selectively modularized and modernized so the new functional capabilities are delivered without breakage.  And if the time comes for a re-write, you have confidence that all of the lessons learned from the legacy code base are applied to the new design.  Thus, a greater chance of success, especially in meeting a deadline.

So, what makes a great architect?  In many respects, a lot of the same characteristics that make a great product manager: curiosity, an ability to translate what users and salespeople need into technical terms, abstract thinking that enables one to imagine new possibilities, etc.  Of course, the architect also needs the deep technical experience too.

Back to the premise of Mike Driscoll’s article: the best software is being built by people with, dare I say it, experience.  Experience to avoid pitfalls because she messed something up before.  Experience to choose the right tools for the job, much like a fine craftsman that builds furniture, or houses, or bespoke clothing. Experience to know what degree of flexibility to design in, without paying for needless flexibility that feels more like insurance against every conceivable future requirement.

I’ve known some good architects and probably only one or two great ones.  With the great ones, we have had some huge debates thanks to the force of personality that seems to come with great ones.  But in the end, despite the strong personalities, great architects are worth having.  And the great product companies know this, which is why they spend a lot of money on great architects.

I say it’s money well spent.

“Creative Destruction and Netflix”: Part Two

I wrote a while ago in admiring terms about how Reed Hastings was trying to disrupt his own business before others did it to him.  As in, splitting the mail-order DVD business from the online streaming business at Netflix.

The backlash to this announcement was pretty huge.  To the point that Netflix had to “undo” the announcement.  Talk about a black eye.

The timing of the decision is certainly up for debate given how customers reacted.  So let’s say it was premature.  But how premature?

I still contend this is the right decision.  Eventually.  Just like the Pony Express was rendered obsolete by the telegraph, so too will mail-order media delivery be made obsolete by streaming delivery.  Who would believe otherwise?

It’s darned hard to time such changes, however.  Most companies never make the leap at all, hence books like The Innovator’s Dilemma.  For those who have the bravery to do so like Netflix, the timing is a perilous choice.  Too soon?  Investors punish you for cannibalizing current revenue and alienating happy, paying customers.  Too late?  You’ll probably never catch up.

If I knew the answer, I’d be a rich man.  My sense is that it’s impossible to make a formula.  Instead, it’s about getting the whole set of stakeholders on the same page.  So when it’s time to jump off the cliff, everyone is holding hands.  CEO, executive team, Board of Directors, large investors.  Not a small task.  At lest you’re trying, Reed.

The “Measurement Wars”

This post is a bit long, but it ends with why your personal life will be spied upon by your company’s competitors.  Curious?  Read on.

Information – data turned into meaning – is going to (further) disrupt just about every industry there is.  Many of the strong will become weak.  Tiny upstarts, like Davids, will topple Goliaths.  And this could all happen at the expense of your personal privacy.

I know, I know, the “information revolution” has been predicted for decades.  Except that like many predictions, exactly when & how the predictions come true will differ from what’s first assumed.  “Video calling” has been predicted as inevitable since the 1950’s.  But nobody saw Skype as the inevitable means for it to come true.

To see the future, let’s look at what has happened on Wall Street and how the same is about to happen in other industries.

What’s happened on Wall Street

Fortunes on Wall Street are made through arbitrage.  Years ago, the winners were those who had better research teams: who had the better information about a company’s stock?  Currency arbitrage followed: who can spot temporary asymmetries in currency prices and execute a trade the fastest?

Across every tradable instrument, the arbitrage game on Wall Street has moved from high-latency (you know something I don’t, for days or weeks at a time) to near-zero-latency.  New fiber optic networks have been built to shave milliseconds from the average trade.  And entire data centers have been relocated to close proximity to these new networks.  Whatever advantage you have must now be exploited in real time.  Or it’s not an advantage.

As the arms race of zero-latency arbitrage has unfolded, two other trends have been its critical enablers.  The first is data acquisition.  Every trade on every exchange can be captured and analyzed, alongside reams of other data about the companies, markets and countries those pertain to those financial instruments.  The scope and scale is increasing by orders of magnitude.

  • An example of scope: the latest algorithms even ingest Twitter feeds in real time to discern investor sentiment.  Including your latest Tweet about some stock you like
  • An example of scale: banks and securities firms store more information per company than any other industry (see this great report from McKinsey)

The second enabler is the ability to make sense of the collected data.  No longer do freshly-minted MBA’s toil into the night to build Excel models with quaint calculations like “EDITDA” or “Return on Invested Capital”.  Today, PhD statisticians build algorithms so complex that they themselves have trouble making sense of them in use.  Michael Lewis’ “The Big Short” is a great read on this topic.

From Wall Street to Main Street?

Wall Street started its journey by going on a data collection binge, from which it developed its algorithms, from which it automated its trading such that milliseconds mattered.

It’s happening now in online consumer businesses.  Wonder why Google, Facebook et al are under such scrutiny for collecting your data to the point of privacy invasion?  Because they understand that data is the raw material that feeds their statisticians, that feeds their algorithms, that feeds their money-making.

Except “money-making” in this case is the price they can charge an advertiser for an ad on one of their web pages.  The more relevant the ad to a consumer’s interests, the higher a premium the ad will command.  How to discern consumer interest?  Profile the heck out of them.

It’s happening elsewhere too.

The “Measurement Wars”

New companies, or ones that successfully reinvent themselves, will start their innovation and disruption journey by gathering reams of data, then finding the relationships between that data.

The forthcoming data acquisition binge is going to amplify online privacy issues.  Collecting competitive intelligence will border on spying on other companies’ employees.

It will also tempt companies to sell their internal data to others.  Could we see new “data merchants” emerge?  Would you like to know the kilowatt-per-hour energy consumption for every household in America?  Someone would, and would like an electric utility to sell them that data for some commercial advantage.

Orwellian, or progress?

Every boundary that separates individuals, companies, cultures and countries will be subject to elimination or reduction.  Our ability to learn, empathize and understand differences across these boundaries will be exponentially enhanced.  Which is good.

The ability to use that same understanding to exploit others will also increase exponentially.  Which is bad.

My take?  We have dealt with many past innovations that could be used to exploit someone or something.  Each time, a new equilibrium was established and humankind moved on.

But it was the period of rapid transition, and resulting destabilization, that was the most dangerous.  We are in such a period now, I reckon.

Governments must understand the coming hunger for massive data collection, and act to mitigate the risks, if we are to emerge on the other side unscathed.  Or even emerge better off.  But what are the risks, and what are the remedies?

Spy vs. spy

We’re seeing the privacy issue play out now in the consumer sector.  Facebook has been scolded by the Federal Trade Commission for its privacy abuses.  Politicians in the European Union search for ways to legislate much more stringent consumer protections.  I won’t cover this ground because the media has done so many times over.

But we have yet to come to grips with the implications of how companies will compete with each other in the Measurement Wars.  The old rules were about the theft of intellectual property.  But what about when a company profiles its competitors’ employees?  Such as their Tweets, Facebook activities or LinkedIn profiles?  Individual persons will be deeply profiled as part of compiling a dossier of competitive intelligence gathering.

This is where it gets creepy.  Heck, if you were imaginative, you might think that things I write about in my blog pertain to issues I experience at work.

And you wouldn’t be all wrong.

Credits:

I have borrowed from the great work done by others.  In particular, Michael Lewis and McKinsey as cited previously.  Also, O’Reilly Media for their extensive coverage of the underling technologies of “Big Data” and their Big Data’s use in consumer online businesses.

The great privacy debate comes home to roost

I recently came face-to-face with the murky issues of social media and hiring in the workplace.

Days before the person’s interview, a candidate to join my organization started sending Twitter messages addressed to my username.  So I got a notification from Twitter that I was “mentioned” in someone else’s Tweet (Twitter is designed in such a way that another user cannot contact me privately unless I choose to “follow” that user explicitly.  Which I wasn’t).

What was strange was that the person was addressing questions to me about the upcoming job interviews.  Asking something about my company’s job benefit package if I recall.  In other words, questions that could, and should, have been addressed in the setting of an interview.  Even more curious is that the person sent those same Tweets to another employee with the same questions.  And that other employee didn’t even work in my department.

A flood of questions came into my mind…..

  • Could I read this person’s Tweets?
  • Should I read them?
  • Are somebody’s Tweets in the public domain?  With over 100 million users, and the ability of any user to see the Tweets of any other, one could argue yes
  • Are things in the public domain fair game for evaluation in a hiring process?  Beyond the obvious off-limits discriminators such as age, gender, sexual preference, etc.
  • Even if such information wasn’t in the public domain, was it still something I could, or should, use in my evaluation?  In many parts of the world, employers are entitled to collect information about an employee beyond what is offered by the employee him/herself

I found that the answers to these questions were far from obvious.  I suspect it will be years before the law and business behavioral norms will catch up with these issues.

I won’t tell you what I did in answer to these questions, out of respect for the privacy of the individual and the fear of opening up a legal can of worms.

What I will say is that I approach online life as if everything I say can be read by others, and thus used by them to form some judgment of me: my blog, Facebook account, Twitter, LinkedIn, etc.  Is this a form of self-censorship?  Yes, to a degree.  I guess there’s still a role for offline communications and “antique” forms of online communication like email or SMS.

“Creative destruction”: Netflix gets it

About 15 years ago (!) I worked for Reed Hastings.  He was not one for sitting pat.  At the time, he realized that the software category of Software Quality Assurance (SQA) was going to consolidate.  And that you could either embrace that eventuality,  or hold on to the past.

He embraced it, by seeking a merger between his baby Pure Software (he was the founder) and Atria.  Soon thereafter, the combined company was part of Rational and is now a product suite at IBM.

Reed is at it again.  Netflix is separating its DVD-by-mail service from its live streaming service.  I loved the quote from Engadget today:

What really happened here is quite simple: Reed Hastings just put a gun to the side of his DVD-by-mail business and pulled the trigger. Given that he aimed for the ankle, though, it’ll probably take a while for it to completely bleed out. But hey — proactively putting a fading business out of its misery sure beats bleeding for it on the balance sheet.

Joseph Schumpeter and more recently Clayton Christensen have written about creative destruction and disruptive technologies.  Reed is one a few high tech leaders that has the courage to implement what the rest of us know: do unto oneself before it’s done to you.

Here’s the bit that stops others from doing the same: Netflix’s share price, and perhaps even near-term revenue, could suffer.  For most of the industry, one can’t tolerate the thought of taking a step back to take two forward.  And hence the balk at such bold moves, fearing the reaction of others.  Like shareholders or pundits.

Perhaps the definition of “courage” is not fearing the reaction of others?  Game on, Reed.

Staring chauvinism in the mirror

We recently hired a bright, talented young woman into my organization as a product manager.  She had interviewed with members of both my staff and that of colleagues by the time she met with me.

The people in my staff and peer departments are almost entirely comprised of males.  I asked her whether she sensed any chauvinism during the round of interviews.  To which her response was, “no”.

Later, I felt regret for asking the question.  Whereas I was asking in part to establish empathy (as in, “I’m not a chauvinist and won’t abide it in my team”), I realized the dilemma I could have exposed her to.

In effect, she could only say “no” because if she said “yes, I sensed chauvinism” then she could think I had a reason not to hire her on the basis of fit or avoidance of future conflict.

On further reflection, I realized I asked the question because my own ability to assess the situation is limited.  Limited by a cultural gap between me and the primarily Czech workforce she would be working with.  In other words, I don’t know what chauvinism is, or is not, in the Czech culture.

In the year I have been here, there have been times where I felt that I was able to sense cultural differences.  This situation reminded me that there are many things I don’t (yet) understand.  And that my cultural norms can’t simply be projected onto another culture.  Disconcerting….

Let the learning continue.

“Why can’t we build cool products like Apple?”

If I hear this question in a high tech company again, I’m going to puke.  For starters, because your/my company doesn’t have Steve Jobs.  He’s a freak of nature.  A genius of sorts.  And one of a tiny number of people in the history of the technology industry that have his skills.

But lots of companies make great products and lots of money without needing a Steve Jobs.   Shouldn’t we focus on that instead?

First, let’s stop doing the things that are sure to deliver a bad product.  That should raise the chances of success by 50%.  Such as: catering to a small percentage of a user base with esoteric features that will appear complicated to the rest.  Or, rushing to deliver something without proving it’s useful and useable.

Then, let’s do the things that look like best practice. Spend the time upfront designing it right.   Explore contrarian approaches and avoid mimicking competitors without knowing exactly why.

Sweating the details.  Avoid making the user’s life too complicated with extraneous features.  Wait a minute, isn’t this what Apple does so well?  ;-)

I’m having a case of “TED envy”

The TED Conference is going on this week.  I wish I was there, even as I’m consoled by the fact that the weather in Tel Aviv is gorgeous during my business trip.

While I’ve never been to the TED Conference, I have adopted the habit of watching TED Talks online.  The premise of the talks is that a (presumable) expert gives the “talk of their life” in 20 minutes on their area of their expertise.

An aside: I noted that humorist John Hodgman is speaking there this week.  He wrote a wickedly funny and strange book called “Areas of My Expertise”. Maybe that’s why he was invited.  The book is highly recommended.

While TED and TED Talks have been pretty interesting stuff, I thought about the undercurrent of TED.  Which seems to be the unspoken: “let’s all get together, call each other smart, and be confident that the high cost of conference admission weeds out the others”. This type of self-referential, self-reinforcing elitism usually brings out the contrarian and cynic in me.  As in, “the really smart people probably avoid this type of conference like the plague”.

But when you look at the caliber of the speakers, you have to ask yourself: is there a still-higher caliber of people left out?  If so, what percentage of the “smart people” population are they?  Probably very small indeed.

In the end, I decided TED people are way smarter than me.  Ergo, I’ll keep watching TED Talks and wishing I was there.

My parents are on Facebook; I’m outta here!

I’m not the first to write about how parents’ arrival on Facebook has driven their children elsewhere.  I wouldn’t be surprised to see some other social media site overtake Facebook in popularity amongst teens and young adults .  Just as Facebook did to MySpace years ago.

But for people over 25, Facebook is here to stay.

Being a teenager is about experimentation.  Teens go through phases of trying on “personas” through the cliques they belong to, the way they dress, their tastes in music, TV, movies, books etc.  A lot of those experiments are best forgotten, even if they form some facet of the future adult.

For example, I had the nickname “Bambi” in college when I wore my hair shaved to half an inch. I also wore Stranglers t-shirts with swear words on them.  The hairstyle and the t-shirts are gone now, but the music remains in my collection.

If you’re leaving evidence of one of those experiments online, you might prefer to forget about it later.  And you might also prefer others (read: parents) not to see the details along the way.  Hence, the reluctance to share Facebook with your parents.

So if being a teen is about forgetting , is being an adult about remembering?

I think the attraction of Facebook to adults is the ease of remembering by staying connected.  As an adult, friendships get left behind not out of embarrassment but out of practical necessity.  Getting married?  Your single friends might be superseded by couples.  Having kids?  You’ll probably hang out with other parents.  Moving cities?  It’s hard to keep in touch with your friends in the prior city.  Changing jobs?  Your old work friends will drift away.

Facebook helps keep you connected.   100 years ago, people were less mobile and had circles of friends that didn’t change much over time.  Today we change so fast.  But that doesn’t mean we want to divorce ourselves from the past.  Facebook plays a valuable role in helping us stay connected.

Global workforces: when will we get it right?

From time to time, my company looks at its strategic initiatives and asks the question: “where in the world should we locate the people for this work?”  Every company does it (or should),  so what follows should in no way be construed as being about my current employer.

But these internal conversations led me to reflect on how we still don’t seem to have the formula right in high tech.  And perhaps by extension in other industries that can have globally distributed workforces.

It seems like every country in the universe of high tech choices is stereotyped and typecast.  The USA is a place to export work from.  India or Czech Republic are places to export work to.  What about the inverse options?  Or the distribution of work between two (stereotypically) low cost countries?

I hate to say it, but management consultants’ use of the term “value chain” is apropos.  We need to get better at framing our choices and thinking about the integrated whole.

Let me suggest a simple scorecard.  Maybe each element can be rated on a 5-point scale.

First is labor costs.  For a given type of work, what’s the fully burdened cost in each locale?  And what’s the rate of wage inflation that forecasts costs in 2-3 years?  For example, I think a lot of people were disappointed in their near-term return on investment in India given the wage inflation, especially in high tech.  Those who have taken a long-run view of costs in India have been rewarded with an ever-deepening talent pool.

Second is critical mass of the labor pool.  Does the locale in question have an ecosystem to tap into?  Such as large companies you can raid.  Or lots of venture-based start-ups.  Or strong universities.  Bottom line: will there be enough people to choose from?

Third is critical mass of the team. Will the work being performed reach critical mass to form a team?  When teams feel ownership over their assignment, they can accomplish great things together.  Individuals working on teams located elsewhere perhaps not so much.

Fourth is distance from dependent resources.  This could be the distance from key executives.  Or peer departments.  Or subordinate functions.  It’s a hassle for people to stay late at work, or make calls from home at night, or get up really early, all to simply interact with the rest of a team.  Hence the critical mass comment above.

But it’s also a huge tax on managers’ and executives’ time and energy to travel long distances to oversee a distant operation.  After a few trips to a new site, the number of visits from “headquarters” starts to dwindle and the sites become divergent.  I had the pleasure of working for a company that was truly committed to its global locations, and the executives traveled accordingly.  I’m not sure this is the norm.

Last is infrastructure.  What kind of communication infrastructure do you need to link these sites?  This is not trivial if you want to maximize the flow of information, as you must.  For example, Cisco’s 3-screen Telepresence is a wonderful product.  But at $100,000 per site means most people will opt for something inferior.

I’ll sum it up.  People need to feel that they belong.  That they “own” the work they do.  That they don’t have a target on their back because the company is unsure if they are good “value for money”.  That they have easy access to others on whom they depend.

If we get this right, good things will surely happen.