I wrote a while ago in admiring terms about how Reed Hastings was trying to disrupt his own business before others did it to him.  As in, splitting the mail-order DVD business from the online streaming business at Netflix.

The backlash to this announcement was pretty huge.  To the point that Netflix had to “undo” the announcement.  Talk about a black eye.

The timing of the decision is certainly up for debate given how customers reacted.  So let’s say it was premature.  But how premature?

I still contend this is the right decision.  Eventually.  Just like the Pony Express was rendered obsolete by the telegraph, so too will mail-order media delivery be made obsolete by streaming delivery.  Who would believe otherwise?

It’s darned hard to time such changes, however.  Most companies never make the leap at all, hence books like The Innovator’s Dilemma.  For those who have the bravery to do so like Netflix, the timing is a perilous choice.  Too soon?  Investors punish you for cannibalizing current revenue and alienating happy, paying customers.  Too late?  You’ll probably never catch up.

If I knew the answer, I’d be a rich man.  My sense is that it’s impossible to make a formula.  Instead, it’s about getting the whole set of stakeholders on the same page.  So when it’s time to jump off the cliff, everyone is holding hands.  CEO, executive team, Board of Directors, large investors.  Not a small task.  At least you’re trying, Reed.

This post is a bit long, but it ends with why your personal life will be spied upon by your company’s competitors.  Curious?  Read on.

Information – data turned into meaning – is going to (further) disrupt just about every industry there is.  Many of the strong will become weak.  Tiny upstarts, like Davids, will topple Goliaths.  And this could all happen at the expense of your personal privacy.

I know, I know, the “information revolution” has been predicted for decades.  Except that like many predictions, exactly when & how the predictions come true will differ from what’s first assumed.  “Video calling” has been predicted as inevitable since the 1950’s.  But nobody saw Skype as the inevitable means for it to come true.

To see the future, let’s look at what has happened on Wall Street and how the same is about to happen in other industries.

What’s happened on Wall Street

Fortunes on Wall Street are made through arbitrage.  Years ago, the winners were those who had better research teams: who had the better information about a company’s stock?  Currency arbitrage followed: who can spot temporary asymmetries in currency prices and execute a trade the fastest?

Across every tradable instrument, the arbitrage game on Wall Street has moved from high-latency (you know something I don’t, for days or weeks at a time) to near-zero-latency.  New fiber optic networks have been built to shave milliseconds from the average trade.  And entire data centers have been relocated to close proximity to these new networks.  Whatever advantage you have must now be exploited in real time.  Or it’s not an advantage.

As the arms race of zero-latency arbitrage has unfolded, two other trends have been its critical enablers.  The first is data acquisition.  Every trade on every exchange can be captured and analyzed, alongside reams of other data about the companies, markets and countries those pertain to those financial instruments.  The scope and scale is increasing by orders of magnitude.

• An example of scope: the latest algorithms even ingest Twitter feeds in real time to discern investor sentiment.  Including your latest Tweet about some stock you like
• An example of scale: banks and securities firms store more information per company than any other industry (see this great report from McKinsey)

The second enabler is the ability to make sense of the collected data.  No longer do freshly-minted MBA’s toil into the night to build Excel models with quaint calculations like “EDITDA” or “Return on Invested Capital”.  Today, PhD statisticians build algorithms so complex that they themselves have trouble making sense of them in use.  Michael Lewis’ “The Big Short” is a great read on this topic.

From Wall Street to Main Street?

Wall Street started its journey by going on a data collection binge, from which it developed its algorithms, from which it automated its trading such that milliseconds mattered.

It’s happening now in online consumer businesses.  Wonder why Google, Facebook et al are under such scrutiny for collecting your data to the point of privacy invasion?  Because they understand that data is the raw material that feeds their statisticians, that feeds their algorithms, that feeds their money-making.

Except “money-making” in this case is the price they can charge an advertiser for an ad on one of their web pages.  The more relevant the ad to a consumer’s interests, the higher a premium the ad will command.  How to discern consumer interest?  Profile the heck out of them.

It’s happening elsewhere too.

The “Measurement Wars”

New companies, or ones that successfully reinvent themselves, will start their innovation and disruption journey by gathering reams of data, then finding the relationships between that data.

The forthcoming data acquisition binge is going to amplify online privacy issues.  Collecting competitive intelligence will border on spying on other companies’ employees.

It will also tempt companies to sell their internal data to others.  Could we see new “data merchants” emerge?  Would you like to know the kilowatt-per-hour energy consumption for every household in America?  Someone would, and would like an electric utility to sell them that data for some commercial advantage.

Orwellian, or progress?

Every boundary that separates individuals, companies, cultures and countries will be subject to elimination or reduction.  Our ability to learn, empathize and understand differences across these boundaries will be exponentially enhanced.  Which is good.

The ability to use that same understanding to exploit others will also increase exponentially.  Which is bad.

My take?  We have dealt with many past innovations that could be used to exploit someone or something.  Each time, a new equilibrium was established and humankind moved on.

But it was the period of rapid transition, and resulting destabilization, that was the most dangerous.  We are in such a period now, I reckon.

Governments must understand the coming hunger for massive data collection, and act to mitigate the risks, if we are to emerge on the other side unscathed.  Or even emerge better off.  But what are the risks, and what are the remedies?

Spy vs. spy

We’re seeing the privacy issue play out now in the consumer sector.  Facebook has been scolded by the Federal Trade Commission for its privacy abuses.  Politicians in the European Union search for ways to legislate much more stringent consumer protections.  I won’t cover this ground because the media has done so many times over.

But we have yet to come to grips with the implications of how companies will compete with each other in the Measurement Wars.  The old rules were about the theft of intellectual property.  But what about when a company profiles its competitors’ employees?  Such as their Tweets, Facebook activities or LinkedIn profiles?  Individual persons will be deeply profiled as part of compiling a dossier of competitive intelligence gathering.

This is where it gets creepy.  Heck, if you were imaginative, you might think that things I write about in my blog pertain to issues I experience at work.

And you wouldn’t be all wrong.

Credits:

I have borrowed from the great work done by others.  In particular, Michael Lewis and McKinsey as cited previously.  Also, O’Reilly Media for their extensive coverage of the underling technologies of “Big Data” and their Big Data’s use in consumer online businesses.

I recently came face-to-face with the murky issues of social media and hiring in the workplace.

Days before the person’s interview, a candidate to join my organization started sending Twitter messages addressed to my username.  So I got a notification from Twitter that I was “mentioned” in someone else’s Tweet (Twitter is designed in such a way that another user cannot contact me privately unless I choose to “follow” that user explicitly.  Which I wasn’t).

What was strange was that the person was addressing questions to me about the upcoming job interviews.  Asking something about my company’s job benefit package if I recall.  In other words, questions that could, and should, have been addressed in the setting of an interview.  Even more curious is that the person sent those same Tweets to another employee with the same questions.  And that other employee didn’t even work in my department.

A flood of questions came into my mind…..

• Could I read this person’s Tweets?
• Should I read them?
• Are somebody’s Tweets in the public domain?  With over 100 million users, and the ability of any user to see the Tweets of any other, one could argue yes
• Are things in the public domain fair game for evaluation in a hiring process?  Beyond the obvious off-limits discriminators such as age, gender, sexual preference, etc.
• Even if such information wasn’t in the public domain, was it still something I could, or should, use in my evaluation?  In many parts of the world, employers are entitled to collect information about an employee beyond what is offered by the employee him/herself

I found that the answers to these questions were far from obvious.  I suspect it will be years before the law and business behavioral norms will catch up with these issues.

I won’t tell you what I did in answer to these questions, out of respect for the privacy of the individual and the fear of opening up a legal can of worms.

What I will say is that I approach online life as if everything I say can be read by others, and thus used by them to form some judgment of me: my blog, Facebook account, Twitter, LinkedIn, etc.  Is this a form of self-censorship?  Yes, to a degree.  I guess there’s still a role for offline communications and “antique” forms of online communication like email or SMS.