Why 70% of Software Projects Fail

The Bad & The Ugly

McKinsey estimates that 70% of digital transformation projects fail, that is, they are unable to achieve the transformation promised in their charter. That’s a broad swath across various kinds of digital projects, but product innovation failure rates are even higher - up to 95%. That’s enough to make you wonder if it’s worth it at all. 

The lazy option is to chalk this up to luck.

The hard truth is that most failures are avoidable, meaning success in established markets can be nearly guaranteed given the right investment, but it requires a cohesive, well-designed plan with the future state at its center. In short, we have to start paying attention to the fact that the transformation we’re trying to drive often hinges on our ability to mold human behavior. And that means admitting that we as an industry routinely fail at both change management and understanding what motivates our audience’s choices.

But for all that rampant failure, there’s the paradox that to remain the same is also to die slowly in most industries, because although many fail, some succeed. There are sparks of inspiration: projects that created a seismic shift in an organization's culture or market position, products that took off and changed the landscape of an industry. This is the hope that most organizations are rightfully chasing when they undertake transformational projects.

When we’re talking about transformation of any kind, we normally mean two things: transforming the technology itself, in other words, altering the flow of bits on a wire, and transforming behavior or culture. 

Why Pure Technology Failures Aren’t The Biggest Problem

Technology failures are the easiest to solve. They usually result from a team that lacks the necessary skills or maturity of process to succeed. Hire or acquire a skilled team, give them time to plan and architect before they hit the 1s and 0s, runway to test before release, and failures of the technology itself are quite avoidable, or to be a bit more precise, failures of technology itself are routine, but knowledgeable project managers plan for this and allow time to refine the parts that don’t perform quite as expected on the first go. A team that fails here has either attempted to punch above their weight, failed to plan, or failed to take their commitment seriously. All of these are quickly remedied by the right team and manager.

But what, one might ask, about interactions with third-party systems which are nearly required these days? Tech teams don’t control third party vendors and are therefore at their mercy. While there is a lot that can’t be controlled about third-party systems, most of it can be overcome with an adequate time runway. Triple the calendar time you think it should take in the project plan to give yourself time to work around any issues. Most of these challenges are solvable given the time and resources to do so.

This is comforting, in a sense. It feels good to be in control of the outcome. Yet a technically solid project will still fail immediately if it’s not capable of driving the desired behavior. This is a sad state of affairs as technology is perfectly positioned to shape behavior. However, too often, this is left as an afterthought. Projects focus on the hard requirements, the checkboxes and dropdowns and how big the logo is, and avoid the much squishier problem of how to get those chaotic humans to do what we want them to do. I’ll dive deeper into each of these three tenets in its own post, but for now, here are some principles.

Principle 1: Protect Day 0

We can hyper-fixate on our grand vision for how things will be weeks or months after the release, but we won’t get there unless we also provide the right experience on the first day. Most people who encounter a lackluster experience once won’t check back to see if you’ve improved it later. Your first best chance to capture momentum is at launch. 

This encompasses the pace of change (is it all at once, or a bit at a time?), the communication strategy around the change we want to introduce (email most won’t open? in-app tutorial that people will skip out of? release notes that people won’t read? some other strategy?) helps determine success. 

Despite the possibly ominous list above of less-than-successful change communication tactics, it’s possible to craft an overall strategy that introduces change seamlessly and in a way that uplifts and invites the user in. 

Beyond communicating, we also need to ensure the on-the-ground experience is top notch, even as a product is building momentum. This may involve seeding content via web scraping, APIs or manual means, coordinating the first user batches in a two-sided market to control the proper ratios, providing a guided path to success, and separately considering value for existing users as well as new users.

I cover more on the specifics of this strategy in this post: Protect Day 0.

Principle 2: Nail the Feedback Loop

Maybe we want people to be more accurate, record their time, work faster, buy more, follow through on a process, communicate more, log in more, share more, or a myriad of other things. In every case, what we’re asking them to do is change a behavior. And too often, we assume if we put the nuts and bolts based on the data we desire in front of our users, they will assemble the desired behavior for us. But that neglects a key fact: the current behavior, whatever it is (not tracking the data, not recording time, not following through, not communicating enough, etc) is being driven by some sort of motivation. In order to change the behavior, we have to provide a stronger motivation than the existing behavior. 

When I say that, many people’s minds go to bribing behavior. This isn’t about $10 Starbucks gift cards (those mostly don’t work anyway). We need to create experiences that make the desired behavior uniquely attractive compared to all other options:: the tiny dopamine hits that keep reinforcing choices and make it enjoyable for the user.

When we get it right, it just flows. You know it when you click it. Organizations that get it right don’t need so many policies and checklists and procedures. This involves careful application of a number of principles about human psychology around choices and optimization. That’s covered in more depth in our post Nailing the Feedback Loop.

Principle 3: Prioritize the Spark of Connection

Greater than the behavior reinforcement of specific feedback events, can we build positively memorable moments in the experience that rise above the mundane details?  We tend to think people require some rationally advantageous reward to enjoy things – we don’t. 80-90% of our decision-making is not grounded in rational thought. What we do need is something that feels right in the moment. The magic comes into the mix when we bolster our audience’s secret hopes (or allay unspoken fears). At this moment, we allow ourselves to connect to technology in a way that we might not allow ourselves to connect with another person. Need proof? Think how much Google knows about your hopes and fears based on the questions you’ve asked – things you probably wouldn’t confide in most people you know.

An important aspect of this is to time the moment of connection to the emotional arc of the user. With the search, the user is in control of receiving that feedback - it appears at the moment they choose. Sometimes this works, but sometimes we also need to guide the experience a bit more. If you want to read more about the details of this process, that’s covered in The Spark of Connection aka Experience Doping.

Summary and a Note on Speed

Most of the failure points and success points above come down to being willing to invest in the process - this has the near-miraculous effect of simultaneously increasing innovation and decreasing risk. Given enough investment, the theoretical risk approaches zero. Yes, you might say, but where’s the fun in that? I like to ship fast and see where it lands. Speed matters, and precise is fast. Notice I say precise, not perfect. In reality, organizations will always choose something less than perfectly risk-free to gain a better balance of other factors. That said, we as an industry have a lot of room to grow in ensuring successful outcomes through our focus on change management, driving behavior, and creating moments of connection. When we do these things we don’t have to guess or hope, and we don’t have to bribe or badger. What we have to do is learn to confront the discomfort of dealing with messy concepts like motivation, emotion and how users perceive us, and take responsibility for the true outcomes.

If you liked this content, you can have articles like this one sent to your inbox regularly, free of charge, by subscribing here.