Often is the case when consuming fresh evidence that the truism is reconfirmed: what is not visible is far greater than what is; in terms of opportunity, risk, and cost. Few concepts are so profoundly transformative when this string of characters begins to take shape in the mind’s eye.
Unlike biological evolution where logical adaptation occurs over many generations, the dominant complex systems on earth today are driven by technology, evolving very rapidly, and of course subject to human manipulation.
From competitive advantage to utility tax
Similar to healthcare, education, and finance where complexity creep has been manipulated for long periods until surpassing sustainability, enterprise software has become primarily a game of ‘heads I win; tails you lose’ for those paying the bills. While the first generation of enterprise software provided a strong competitive advantage to early adopters, the bulk of systems have long since resembled a tax on admission.
Today most leading enterprise systems offer a competitive advantage to very few organizations where innovation is focused on optimizing commoditization, scale and capitalization; not products, service, or improvement. The result is a breakdown in product design, tragically to include a negative influence on innovation for customers worldwide.
The tipping point for market dysfunction in the enterprise occurred more than a decade ago when most of the innovation began to be outsourced to consumer ventures in Silicon Valley that serve a very different purpose for a different customer. Even if the majority of SV VC firms and young social networking entrepreneurs understood the needs of organizations, globally adopted consumer technology cannot provide a competitive advantage to organizations. It should be obvious that if everyone is using the same rigid programs with little if any adaptation or customization, then the advantage falls only to those with scale.
Another sign of market dysfunction in IT more generally is visible by comparing cash generation of industry leaders with their competitors. A recent article in the WSJ revealed that nine tech companies generated $68.5 billion in new cash during the Great Recession while the other 65 competitors in the S&P 500 generated $13.5 billion combined.
The war chests at IT leaders tells a different story for each company, including recent product success at Apple, the power of the network effect with Google on the Web, effectively managed monopolies, and an economy dominated for many years by investment banks, weak regulation and judicial enforcement, and last but not least; apathy in enterprise customers. I argue that this level of consolidation of market power isn’t good for anyone, including the stockholders of those with the largest war chests as the dividends are either very small or non-existent, with the majority performing poorly in stock appreciation.
Need for ‘market farming’
It’s essential to understand the difference between proprietary applications and proprietary standards, particularly relating to network effect, market power and systemic risk. IT architecture in the network era is of course global and largely unregulated outside of minimal anti-trust enforcement, with voluntary standards bodies, so it should not be surprising then that proprietary standards still dominate enterprise computing, or that proprietary systems are manipulated for self gain. That is after all the fiduciary duty of public companies with bonus incentives, stock options, and careers dependent upon same.
Of course it’s also true that given government’s terrible track record in regulating technology, combined with nationalist conflicts that too-often suffocate innovation with protectionism, very few of the vastly divergent stakeholders in the ecosystem favor increased regulation, which leaves us with the heavy lifting of systems design and market education to attract intelligent decisions by early adopters in an environment that is predatory to new entrants. Markets do eventually depend upon well-informed customers acting in their best interest, even in expert professional roles; particularly in markets dominated by lock in.
From the perspective of a consultant and venture capitalist who coached hundreds of businesses of all sizes on defending against predation, it is an understatement that enterprise customers need to take a far more proactive role as experts in market farming. The financial incentive to do so is certainly sufficient, as is the risk in not doing so. Until such time that more effective global governance of standards is enforced, however, national governments, non-profits, consumers, and especially CIOs have a special responsibility to be proactive as ‘market farmers’ within the IT ecosystem.
Ownership of data
In the modern world, data represents knowledge, security, identity, health, and wealth. Those who control proprietary standards within the network effect in an unregulated environment not only can extort a high price of entry to the utility, but can also greatly influence if not control ownership of data. We see that often today on the public web (see LinkedIn’s license agreement for an example).
Our position with Kyield is that while we have every right and indeed a responsibility to protect our proprietary intellectual capital within our platform, we do not have the right to extend the power enabled by the platform over others. I would even go so far as to say that we have a responsibility to defend customers from others doing so to the best of our ability. Our power should only extend to the borders of our internal systems, beyond which we should embrace universal standards, so we do, even though they are far from perfect and extremely slow to evolve relative to need.
Networks are not new, they are just much larger, contain data representing far more than ever before, and are of course global. Nations created standards for electric grids presumably to prevent the electric companies from owning every appliance maker, if not their economy. The ‘extend and conquer’ strategy is predatory, obviously very dangerous to the global economy, and should therefore be regulated and aggressively enforced. Nationalist conflicts and protectionism have prevented proper regulation.
It’s my personal position that no one should have power over the languages used for public transactions, nor should any company or group of companies (or any entity including governments) have influence, power, or ownership of data other than the original owner of that data or those who have been granted the rights of use. Ownership and control of data is I believe a fundamental right for individuals and legal entities. It would surprise me if the U.S. Supreme Court failed to better clarify data standards and ownership at some point in the near future.
Of course standards bodies have a higher calling not only to manage the standards process properly within meaningful time constraints, but also consider the influence of conflicting interests on their financial model, as well as sustainable economics relating to adoption. I recently suggested to the W3C for example that their Wiki was insufficient for communicating the benefits of the languages the body created and approved. The response was that they didn’t have the bandwidth and that communications was up to vendors. The response fell well short of what leaders in science have been teaching for many years, suggesting to me that significant change was needed in the model and culture.
With very few exceptions, during the 15 years I have been observing the evolution of computing standards, it has been confirmed often that the IT industry is no more able to self-regulate than the finance or healthcare industries. I wish they could, but wishing doesn’t make it so.
Beauty of simplicity
Those who doubt the need for reducing complexity in our society probably haven’t spent much time working pro bono for a worthy cause that cannot be saved due to entrenched interests owing their existence to manipulated complexity. I have seen the dilemma many times during my career manifest in legislation where the individual interests of the few manipulate the outcome for the many, thereby threatening the whole, to include themselves eventually of course, with little or no apparent understanding of the impact of their self centered activism. The current political situation in the U.S. is obviously not what the Founding Fathers (and Mothers) intended, and indeed feared most.
A very similar dynamic evolved in computing over the past few decades, which I began to recognize in the mid 1990s when we converted our consulting firm to one of the first commercial Internet labs and incubators. It was only when I trained to become a network engineer and object oriented programmer that the degree of manipulation and systemic risk became apparent.
“Complicated systems and their combinations should be considered only if there exist physical-empirical reasons to do so.” –Albert Einstein
There have been reasons for the increasing complexity in enterprise software, but have been primarily limited to protecting cash cows and extending market share in the global network ecosystem. While economies of scale and Moore’s law pushed hardware costs lower, software costs escalated while arguably delivering less value, despite a massive reaction by open source. The parallel evolution in healthcare, education, and government in the U.S. was very similar; we were all paying more for less.
The problem today of course is that our collective resources can no longer afford even the status quo, much less the trajectory, from the costs due to manipulated complexity. In enterprise software, manipulated complexity also creates a very high maintenance cost now exceeding 20% of product cost per year in some cases, and of course a much higher level of security risk. This is a dangerous game and largely unnecessary.
One of the elements I liked best when reviewing the papers of the Google founders and later their early beta was the elegance of simplicity. Google was among the first substantial efforts in the Internet era to reduce a highly complex environment to a very simple interface designed for humans in their own natural language, and it worked very well indeed.
Ironically I found Google while searching for raw IP in an effort to achieve something similar for the enterprise, but enterprise data had not been converted to simple HTML like the public web, rather was subject to incompatible languages and data silos, and of course HTML lacked the ability to provide the embedded intelligence we were seeking, so we had a long wait for standards to catch up to the vision often associated with the semantic web. I like to think we used the time wisely.
Simplicity for the enterprise has not yet arrived at the station, but the train is full and fast approaching.
Great insight into the role complexity plays in 21st century organizations and how the monopolistic tendencies of large software vendors (the few) stifle innovations and hence benefits for the many.
Thanks Rob — the draft was much longer with more detail on complexity and technology, but this was all I felt comfortable sharing in public.
Absolutely agree!!
Some thoughts….
Complexity is created by successively adding layers of functionality on top of existing systems without fully understanding the current functionality. On top of it, enough time is not spent on refactoring the code and applying modern technology. The problem is the business users don’t understand technology and the IT team is not savvy or sophisticated enough to explain in a fashion that business understands. The result is that sufficient budget / time is not allocated and IT fumbles and slowly loses credibility and it is a downward spiral from there.
With the technologies available today, there is absolutely no reason to not make the enterprise knowledge base searchable and useful. Like you mentioned there are entrenched power bases that owe their existence to complexity. Compounding the problem is apathy and lack of knowledge. If only the enterprise knowledge is searchable like Google, organizations would be much more nimble and more productive.
Magesh,
No doubt that the incremental layering approach is very common in both client organizations and vendors, but I was referring to something a bit more sinister in this piece mainly on the vendor side — manipulated complexity.
A common complaint one hears for example in the relationship between Intel and Microsoft is that they both benefit from ‘bloat’ with some claiming that it’s intentional in much the same way we see healthcare vendors silently abusing the referral system. Neither system benefits the customer, only the vendor, until such time that eventually the entire system crashes due to unsustainable costs or systemic risk associated with network dynamics.
This is but one example of many, however — all of the leaders in enterprise software are targeted for such claims — with such high maintenance fees in ERP for example, what incentive is there to reduce the cause of maintenance in their architectural design? None. Thanks for your comments.
Mark- I’d love to hear more about your thoughts on this topic.
“Ownership and control of data is I believe a fundamental right for individuals and legal entities. It would surprise me if the U.S. Supreme Court failed to better clarify data standards and ownership at some point in the near future.”
Hi Deborah,
Well I am not a legal scholar, but am an informed citizen who has invested a fair amount of energy reading the works of the Founding Fathers with a great deal of time and energy with the technical issues on the Web and Internet since the commercialization. Many scenarios have come up over time that might make good test cases for the Supreme Court — the most obvious would be an abuse of health records, but perhaps a less obvious case might be at the junction between the power of network effect and intellectual property.
For example are rights freely transferred when it’s a requirement for entry for one’s career, enter the economy, etc.? Does such a license represent extortion or free choice? Similarly, when a vendor controls say 75% of a global market and negotiates the purchase of related patents or other IP, particularly in networked world, does that represent a free and open market?
How about when a small group of very close collaborators own essentially all of a market? Can they then not decide which standards are adopted and which are not over what is now obviously an essential public utility?
Just a few thoughts on a Monday morning. Appreciate your interest and comments — Mark
Mark,
While I would love to see the US Supreme Court rule for a privacy right for personal data that follows the lines of, say, Germany’s, I think it unlikely.
Rulings from the last forty years have chipped away much of the territory that the court used to consider the domain of “reasonable expectation of privacy.” Much of this took place in the arena of law enforcement, but you also see quite a bit of erosion in corporate law, as well.
It would be interesting to see a copyright-based claim prevail. A ruling that click-wrap licenses are unenforceable because there is no possibility of negotiating terms, or a ruling that such agreements are unconscionable and unenforceable on their face, might also do the job.
It’s unlikely that the court, as constituted, would rule to kill of the marketing, credit reporting, and advertising industries in one fell swoop.
Given that the courts have repeatedly ruled that the government may buy and use data – data that it is legally barred from collecting directly from and about individuals – from third-party companies, I think that a privacy argument based on a fundamental right of individuals to ownership of data about themselves would fail.
Barring a constitutional amendment to establish individual ownership of personal data, or stripping corporations of “personhood” and individual rights, or the appointment of justices in the mold of Brandeis, I’m not that optimistic. -Wayne
Wayne,
I agree that unenforceable contracts offer the most likely scenario, but I could see a ruling that confirms that a contract is simply invalid when no choice exists. I’d need to research it before understanding if this is the same bird of a different feather in the U.S.
I see IP coming up in many ways — it needs to be redefined — tech giants that dominate standards have a direct conflict in architecture that would protect IP (meaning they haven’t/won’t for everyone but their own, of course), and the consequences for nations that don’t protect IP will eventually be very costly. The U.S. talks a good game, but has long since abandoned any meaningful protection for IP outside of institutions.
If defense is unaffordable, and/or markets are dominated by a few, then obviously justice doesn’t exist. .02– Thanks, MM
Resultant software complexity is also dependent on the delivery model of IT services within large complex organizations such as County Government. County Government is comprised of many Departments, some of which are managed by Elected Officials. The Elected Officials of these larger Departments are keenly attuned to constituent’s needs and/or demands. Effective delivery of services of course depends on software deployed and managed within the Department. How IT is structured greatly affects the ability of the Departments to effectively deliver their line of business services. Centralized IT is typified of command and control structures that define the aura of what is acceptable. Anything outside the “standard” is not acceptable. This limitation greatly increases the complexity of how software is utilized and designed across the Enterprise and over autonomous County Departments. The effect in many cases is a set of increasing complex deployment of solutions that don’t quite measure up to the needs of any one Department but provides a “watered down” enterprise solution – that most folks are unhappy with and increase complexity for Departments and that…..comes with a substantial price tag. This situation can then be attributed to higher costs in terms of additional personnel, inefficient delivery of services and dissatisfied constituents.
A more effective solution in this scenario would be a federated model of IT service delivery. Within this model the central IT organization would collaborate with Departments (many of whom have internal IT staff specialized in meeting the unique needs of the larger Department) in developing more effective software solutions. Department IT offloads the demands on Central IT that has little understanding of line of business needs let alone the time or resources to do so. In this manner, Elected Officials are more able to deliver the services they were elected to do so without being constrained or held hostage by a central IT group that has little accountability. Without accountability there can be no responsibility. Indeed, it is quite possible for irresponsibility to develop.
Bob,
I find very little here to disagree with– no question that organizational management can increase unnecessary complexity–indeed I agree with you that organizational management is the driving force not just within the organization but also their collective impact on the market. CIOs are now a threatened species in many organizations–at least in terms of management, due to not taking a proactive stance in the marketplace–it took some years but has finally become a priority (market farming in IT), so now we are seeing CIOs becoming part of the board in some cases while in others the CFO is assuming the CIO duties– due to the enormous costs and importance. Even a few CEOs are becoming more proactive (humor).
The only warning I would share given my experience advising government agencies, although this issue is not limited to organizational type, is that building internal IT systems is not necessarily the business organizations should be in. Labor is often the highest cost item in the P&L in organizations that are now struggling greatly due to decades of expansion of HR and associated costs–in fact from the CFO perspective that’s one of the only reasons the most expensive software systems can be justified. I’ve observed many situations where organizations have adopted open source for example, but due to the very high costs associated with internal IT folks– certainly to include pensions and healthcare, that the total cost of ownership is actually much, much higher than the most expensive proprietary systems. Indeed I have observed public employee unions and universities embracing open source as a means to grow membership, protect jobs, etc. not unlike proprietary software relationships with IT–a broader culture and strategy of empire building that is now threatening entire nations and the global economy.
That said, organizations must be enabled and empowered to manage their own destiny, management should ultimately be held accountable, and IT systems must be designed to be adaptable so that organizations can manage instead of being managed by cartels via commoditization of IT systems. As high as the cost of IT systems are, it is nothing compared to the stagnation of a global economy using commoditized IT. Thanks for your comment.