Benchmark Your Project Delivery Outcome

We’ve got a fascinating review for you today on some very interesting research we’ve been doing over the last little while, benchmarking the industry on the capabilities to deliver project outcomes.

Now there’s an opportunity for you to benchmark your company and see how you compare to our research results. And I’ll tell you how you can get involved in that at the end. But what I wanted to do was jump on here and give you a preview of the research that we’ve been conducting as we’re working on finalizing our reports that will be released on our benchmarking research.

Because I think we’ve known for quite some time that project completion is no longer enough, right? There’s a gap in getting to the end of construction versus actually delivering operational outputs on projects. The original intent of why projects were created in the first place.

This gap has existed for quite some time, but we’ve never really understood why this gap exists and what to do about it. While we’ve uncovered some fascinating research here that I’m going to show you, we’ve researched across 1400 global companies, and these are the biggest companies that are involved in major capital projects and what we’ve done.

I’ll explain our methodology here. In researching these 1400 projects or companies that are contributing to projects, what we’ve discovered is that the average ability of all of these companies across all industries to deliver project outcomes is only 31%, which is quite low.

But I don’t know, I would say that’s all that surprising because we know that projects are late and over budget and not meeting their intended outcomes. But the data actually proves that only 31% of companies actually have the structural capability to be able to deliver project outcomes.

And that’s the problem that ICxA solves. This is the actual measurable evidence to show an organization’s capability to deliver project outcomes. Lots of companies are measuring what we’re familiar with for cost and schedule, and safety, but there hasn’t been until now an organization that actually measures how capable organizations actually are to deliver outcomes.

And you’ve heard the saying many times before, you can’t improve what you don’t measure. Right now, a tool exists to be able to measure an organization capability to deliver project outcomes.

ICxA’s Research on Outcome Readiness and System Integration. So rather than focusing on the typical project management aspects you’ve heard for cost, schedule and safety, ICxA measures outcome readiness, system integration, and operational performance capability. What really matters is delivering a project’s intended outcome.

And when we look across the research, there are actually very few mature organizations that have built these capabilities to be able to deliver project outcomes. In fact, there are almost no companies that are above our scoring metrics above 60%. There’s a very strong banded cluster in the 20 to 40% range, showing a low maturity of ability to deliver project outcomes.

When we look at the data across all the industries, across all geographies, it’s clear that the industry, the construction industry, hasn’t actually plateaued. It’s just that it’s never actually reached maturity levels to deliver project outcomes.

What’s typically measured on projects isn’t what actually delivers project outcomes, and what you find is that most projects don’t fail at execution. Projects are actually really good at delivering construction activities and delivering installed components, right?

But what fails on projects is that they fail at the outcomes at the end of the projects. When a project is delivered, that does not mean that it’s operational. When equipment is installed, that does not mean that it’s integrated correctly. And when a project is complete, that does not mean that it’s performing operationally on day one.

And most organizations don’t measure this gap; they just leave it to chance, right? Outcomes are not well measured, which is what’s prompted us to undertake this research across 1400 projects so quickly. Let me just know in the chat what your organization’s capabilities are.

When you look at your organization’s ability to deliver project outcomes, what would you say? Yes, you’re able to formally and consistently deliver project outcomes? Would you say somewhat, maybe hit and miss, there are some activities to deliver outcomes, but it’s maybe not quite as structured as it should be? Or would you say no, that outcomes on your projects are typically just left to chance? Let me know in the chat, yes, somewhat or no.

This is the research that we undertook across over 1400 projects or companies. It was 1435. We created this list of all companies that are involved in delivering outcomes on projects. There were four classes. Class A is your asset owner or operator.

There are 546 companies in our research list, Class B delivery integrators. So this would be your EPC contractors. We listed 275 companies that were directly involved in major capital projects, Class C. These would be the engineering authorities, the consulting firms, and the groups that are involved in FEED or involved in the detailed engineering design.

We listed 385 companies involved in that class, and Class D, we listed the OEM and equipment suppliers; there were 229 of them. So these four classes make up the core groups involved in delivering projects, and it was a fairly extensive list of 14 / 1400 companies that largely make up the mega project industry.

Deep Dive into Project Outcome Capability Gaps. Now this was across every geography in the world too. We had companies listed in North America, in Europe, in Asia Pacific, the Middle East and Latin America. So this was a global distribution and representation of all companies and all projects all over the world.

What we found is that these companies are actively letting the external world know what their capabilities are, right? And that makes sense. They’re advertising their skills, their services and what they would like to offer to other project participants in the world, right?

So it’s pretty easy to read those signals on what those companies are saying they’re good at. We can read those signals and get a fairly good interpretation of their engineering capabilities, their construction capabilities and more importantly, the five areas that we were looking at, interested in this research, particularly their external signals on capabilities for outcome governance, for system integration, for performance verification, for operational readiness and for outcome delivery.

So these are still just external signals, but a fairly good proxy for the industry’s capability to actually deliver project outcomes. So what we did was we scored these across what would be the minimum score required to be compliant with the ICxA outcome assurance standards.

So industry, industrial and manufacturing were the highest, still only at 42%. Oil and gas was, second in nature. Powering and utilities were not too far behind mining and metals. And then the transportation industry ranked the lowest. Generally, these are all well below the standard levels in what’s required to deliver project outcomes. When we were looking at the minimum requirements of of 80% or higher, this still falls quite short across all industries in the ability to deliver project outcomes.

So it would show that the problem is not industry-specific, it’s systemic across all industries. Who is actually feeling the pain of these missing outcomes? Well, these are the assets or the organizational classes that we looked at. The OEMs and the technology, the suppliers on projects, they’re showing one of the highest abilities to produce integrated outcomes at 41%, still quite low, but they were the highest ranking. Asset owners were not too far behind at 37%. The EPC’s, the specialist groups or sorry, this one is the commissioning service providers at 37%, engineering and consulting at 36%, and then the EPC contractors were at 34% at the lowest in the mark there.

So still falling quite short of the ability to deliver outcomes regionally. What were we seeing? Are there pockets of the world that are better at delivering project outcomes than others? While still it’s across all regions, it’s not a regional issue. It’s really a global capability gap to deliver project outcomes. You can see in all regions of the world, all global areas are falling short of the ability to deliver project outcomes. And I think that makes sense with regard to the results that we see in projects and the results that we’ve gathered here.

There’s no strong leader in the world who is able to deliver project outcomes. It’s a universal problem across all areas of the world. How Outcome Assurance Solves the Systemic Project Problem. This chart was key. From our data that we gathered here in that orange line, what’s required to actually maintain the ability to control a project’s outcome versus the industry’s capability that is outwardly being displayed by these companies across the world? And the gap is clear that the industry is not where it needs to be. The ability to deliver outcomes is just a missing skill set that does not exist in the industry, that the industry has not matured to that level yet, as other industries have.

The industry has developed maybe some components of outcome assurance, but not the system that connects them. It’s very much a delivery-focused industry rather than an outcome-focused project plan. The scores across all aspects are quite low, and there was only a handful of organizations that outwardly displayed the ability to deliver project outcomes. Now, to be clear, there were some organizations and companies that scored quite well off the charts, well above the minimum baseline to be able to deliver project outcomes. And it was encouraging to see that some companies are leveraging this gap and developing those skill sets to actually deliver results that mattered.

But I would say that there were very few companies that actually met that bar. Across all pillars. It was clear that fragmentation exists, and it’s not a cohesive system to deliver project outcomes. And of course, when you’re not able to deliver a project outcome, this explains why we always end up with, or often end up with, late projects, cost overruns and systems that don’t work when the focus is purely on execution and delivery, and there’s no focus on the outcome.

While then it’s no surprise when that outcome isn’t achieved at the end, right? We end up with poor startup performance and projects that don’t work. So when a system is structured this way, failure is not random. This is how the system is structured. This way is to focus on delivery rather than on outcomes. It’s not much of a surprise when you look at this data to see why we’re ending up in the state that we are for delivering projects. What are the leading organizations doing differently that are allowing them to achieve project outcomes?

What’s missing is what the leading organizations are actually doing is they’re following an outcome assurance process, right? They have a strong governance layer across the project. They’re implementing robust stage gates to confirm outcomes along the way rather than deferring all the project delivery and project outcome risk to the end of the project. They’re assigning a strong outcome authority at the beginning of projects that can lead the team and the groups involved to cohesively keep focused on the outcome and make system integration a priority rather than just leaving it to chance at the end of projects. Outcome assurance is the missing discipline that pulls this all together to allow organizations to have this organizational capability to deliver project outcomes.

And for the organizations that seize this opportunity, be aware and recognize that this gap exists. The entire industry is really still in its early stage of maturing to the levels that it needs to be, right? There’s no dominant leader who has stepped up and is delivering and focusing on outcome assurance to lead project outcomes. That the gap is massive. And there’s a huge opportunity that exists for any company that wants to get ahead and fill this gap to be the outcome leaders and deliver project outcomes. And it’s very likely that we’ve already scored your company on our list of 1400 companies. This includes a wide range of companies across all industries that are involved in major capital projects. And it’s if we haven’t, it’s easy for us to do is to do the same research and incorporate new companies into this research and score as well. There’s an opportunity if there are organizations that are out there that want to see where you stand within this range of data and see how you compare to the standards, how you compare to other industries; we can certainly do that. Reach out to us, and we can have a chat and provide you with your score because we likely already have it.

What we’ve done is this third line, this blue line is that we can take a company and compare these 5 core outcome capabilities to compare with what the industry average is, and how does that compare with the minimum standard requirements for outcome control. And we can overlay this blue line and companies can easily see how you compare with your peers in your industry and see how you compare to the minimum standard requirements. And this is a huge opportunity for companies that want to fill this gap and become outcome leaders to drive project success. Key Insights and How to Benchmark Your Organization. So if I can get you to only remember 5 things from this presentation, these are the five key takeaways that I’d like to leave you with: the average outcome index. The outcome assurance index score is only 31%. The industry is still very fragmented and in its early stages of evolving to the maturity levels that are required to deliver project outcomes. Very few companies are mature. We did see some standout stars, but there’s no strong leader who exists to be able to deliver project outcomes.

While execution is strong, there are lots of companies that are very strong in execution and delivery and installing and doing the engineering requirements that are needed to deliver projects. But when it comes to system integration and outcome delivery, the industry across all aspects is still very weak, and it’s still not at the maturity levels that are required to deliver project outcomes.

Now we can continue to focus on execution, and we absolutely should, but the solution is not. Better execution and better delivery. There’s more to the system that we need to deliver here and develop in order to achieve project outcomes.

So while the industry for the last several decades has been focused on better execution, more digitalization of execution, focusing on strategic elements to make execution better, that’s not actually the core fundamental issue with the industry.

The missing element is outcome assurance to take execution and take all the other components of the projects in order to deliver project outcomes. The fragmentation between pillars is essentially the root cause, where we’ve got pockets of strength for execution, engineering and procurement.

The missing piece is the fragmentation between those pillars, and that is the root cause of why we have a low success rate for project outcomes. There is clearly a system problem that requires outcome assurance. The industry is just not structurally set up to deliver project outcomes, and the industry is performing exactly as it’s designed. It’s achieving the limits of what the structure can achieve, and it will only improve once we have a new structure to focus on project outcomes.

Now, if this is resonating with you and you’re interested in getting involved in ICxA, there are lots of ways to get involved to be able to get your outcome Assurance index score, to see how you compare in the industry and to perhaps elevate some core capabilities to fill this gap that exists in the industry. The first one is a corporate membership. That’s an easy 1 to get involved. You get access to all the resources that ICxA has to be able to elevate capabilities to fill this gap. That includes your Outcome Assurance index score, where you can see how you compare to industry averages and see how you compare to the benchmark standards.

You can get your full benchmark and capability reports. There are lots of ways to develop teams and elevate with ongoing engagement to help your organization move into this gap and fill this industry opportunity to lead project outcomes. So I definitely encourage you to get involved to benchmark your company and get your outcome assurance index score and see how you compare to the core capabilities that exist in the industry. Because the organizations that win in the next decade are not going to be the ones that strive for an incremental 1% improvement in execution.

They’re going to be the companies that measure and deliver project outcomes much better. And this is the opportunity and the gap that exists for companies that want to move into that frame. So definitely get involved. You can reach out to our email address info@icxa.net to get started, and you can find out where you stand within the research that we’ve conducted here today.

Q & A

Q: All right. First question I see is to an extent, for some companies that I have worked for, there seems to be a commonality of B deficiencies which are acceptable for completion in summer, then become A deficiencies if the project gets delayed into October, and the B turns into a delaying project further.

A: Yes, I’ve seen that too, where there’s any subjectivity in the classification of deficiencies. That’s always a debate, right? Everybody’s trying to move from one to the other just to get through. But when you set up an outcome assurance framework, then that becomes crystal clear, right? There is no debate. And these are the methods that you need to achieve to demonstrate with evidence each outcome of the project. There is no debate, and it’s black and white. You either achieve this outcome milestone, or you don’t, and it becomes crystal clear.

Of course, there needs to be mechanisms for change as projects are evolving and whatever. There needs to be some discretion by very specific individuals with outcome authority to control those stage gates. But when this is all structured that way, then it becomes crystal clear and you can avoid that debate of classifications you’re trying to get through to the end, and reclassify things that maybe shouldn’t be that way. Yep, good question rate.

Q: Next outcome assurance process. What should be improved after the system integration?

A: The what Outcome Assurance does is it defines the entire governance process that’s required right from the start of projects, not only to manage system integration but also to manage all the aspects of projects that are required to achieve project outcomes. Of course, system integration is 1 important aspect, but it’s only one. There’s an entire suite of activities that need to be completed upfront to follow a true outcome assurance process. This is all outlined in our ICxA standards in our 3 levels of standards that we have.

The Outcome Assurance standard is the highest level that defines what this governance process looks like. Then the second level of standards defines our commissioning process, the commissioning capabilities that are required for system integration, as well as our operational readiness processes that are required to achieve project readiness. These three core standards are what make up the entire outcome assurance framework for delivering project outcomes. So the baseline is set with these three standards, and that’s where that orange line was coming from to define what an organizational capability needs to look like to align with these standards.

And in the research that we’ve conducted to gauge where companies actually exhibit these external signals for capabilities to outcome assurance, that’s the gap that exists between the industry capabilities and the benchmarks that are set by our standards.

So, definitely a good question related to system integration. But to see the entire process, I would definitely encourage you to reach out to ICxA and get involved, get access to the standards, and you can see the entire process that is required to actually deliver successful project outcomes.

So definitely some good questions.

If there are any other questions, feel free to reach out to us, and we’ll be able to help you out. So by all means, contact us at info@icxa.net. We’ll be able to get you plugged in with the frameworks and the scoring and help your organization elevate to the capabilities that are actually needed to deliver project outcomes.

Visit icxa.net for more information.