Is your innovation covered in DIRT?

Every company is becoming a software company. Now that we are fully ingrained in the digital age, that statement sounds like an old adage. However, when you dig into the trope, it’s really saying that innovation in the digital space — specifically application development — is becoming a key business differentiator. It is driving new business creation and competitive advantage. The speed at which a new application can be deployed, along with the number of innovative features it includes, have a direct correlation to business success.

If this is in fact true, then applications are the currency of this new economy we are living in, and the development teams who build the apps are the market makers. Despite their importance and the continued focus on innovation and speed, these teams are constantly misunderstood, mismanaged and marginalized within companies of all sizes.

This is not a conscious decision, but it is a costly one. It quickly becomes a tax on the amount and quality of innovation that an organization can produce. One place this tax manifests is when there is a failure to understand the work that developers do and how they do it, or a failure to provide a productive environment for them to work in. In the digital economy, failure to address this can be devastating.

The reality of an Innovation Tax

Innovation tax, like income tax, is real. It has an obvious impact on morale, resulting in attrition and churn, but it impacts the bottom line in other ways as well. The pace of innovation at taxed organizations can slow to a crawl as people and resources are locked into maintenance instead of innovation.

Another way to think about this is as DIRT, or the Data and Innovation Recurring Tax. Why? Let me explain. The tax is fundamentally rooted in data (D) because it is so often predicated on the difficulty caused by trying to use legacy database technologies to support modern applications that need access to real-time data to create rich user experiences. It impacts innovation (I) because teams spend most of their time supporting complex, tricky architectures instead of innovating. It is recurring (R), because this is not a one and done tax (T). In fact, DIRT applies to every new project, making each a bit more difficult because it adds new components, frameworks and protocols that have to be managed and maintained.

This is not news to technology leaders. They recognize this tax — they just haven’t traditionally been able to put a name to it. They can also immediately identify the degree to which it is caused — or cured — by their data architecture. Data is sticky, strategic, heavy, and intricate, yet it is still the core of the modern digital company. Applications today are much more sophisticated, and so are their data requirements. Add to this the growing amount of data, and the expectation that companies will react more quickly and more cleverly to all the signals in that data, and you are at a crossroads. Legacy technologies, including single-model, rigid, inefficient and hard-to-program relational databases, just won’t do.

DIRT is everywhere

If this was a rare problem, it wouldn’t be such a big deal. But when you think about the fact that large enterprises can have hundreds or thousands of applications, each with its own source of data and pipelines, over time, data architecture starts to look like spaghetti. This only multiplies as the variety of technologies, each with its own framework, protocol, and sometimes language, adds to the mix. It becomes extremely difficult to scale because every architecture is bespoke and brittle. This means development teams that are supposed to be working on building new applications and features — that the business needs and customers will love — instead spend time trying to integrate this mess.

Given the continued acceleration of innovation, it doesn’t come as a surprise that organizations are ready for a new approach. They know they need to simplify their data architecture, but due to its daunting nature, it’s easy to postpone this work, sometimes indefinitely.

Often, data architecture becomes so complicated and difficult to maintain that even considering new technology is impossible — never mind using it for transformation. This is why a lot of senior people at enterprise companies are sitting with their fingers in the transformation dike, waiting for retirement — because modernization seems impossible.

Start with understanding

The first step toward cleaning up your DIRT lies in getting a better understanding of just how it’s impacting your team. Do your developers struggle to collaborate because the development environment is fragmented? Do schema changes take longer to roll out than the application changes they support? Do you struggle to build a 360-degree view of your customers? And if so, why? These are good places to start digging in the DIRT. It is also a good idea to take a hard look at your applications and data sources. What would it take to move your data onto an application data platform, and support future innovation?

This will not be easy. This is no doubt not the first time you’ve tried to simplify a data architecture, but that also means you know progress when you see it. Cleaning up your DIRT will put you on the right footing to move forward.

At TechRadar we've featured the best productivity tools.