It's no coincidence that some of the most-spectacular failed projects are in I.T. : the payroll system, ArriveCAN, the original "ObamaCare" web portal. (The TIME magazine cover on the latter is illuminating, describing jaw-droppingly stupid failures that the I.T. newbie can spot at once. The first was that the database of recipients wasn't "indexed" - that's like a library card catalog being in random order, so that you have to search the whole library list for each book found.)
Custom I.T. , you see, was turned over to consultants nearly 30 years ago. Larger companies had "data processing" departments, programmers who spent a career managing the mainframe payroll system, after writing it. But, computing got more complex, and the Data Processing Department struggled to keep everybody on their beloved mainframe. Their "customer departments" that did the actual corporate function, broke out to use PCs and small servers, started calling consultants if the DP department refused to help.
So, about 1995 (in my case), our IT Department flipped to embracing the PC as more than a word-processing toy, started getting programs that ran on servers - which they didn't really understand, so more and more programming moved to consultants.
By 2010, I was begging my bosses to believe that a system I'd had done for $400K, was meeting all needs, had no complaining customers (by "customers", I mean our draftsmen and our internal map-users - it was a drafting system for city maps of water and sewer), didn't really need replacement, just some upgrades and tweaks.
But the IT consultants, very much at the behest of the IT Department that hated that Water had done its own IT, insisted that the system didn't use the coming paradigm of mapping, ESRI GIS, was based on a CAD program that was going to go obsolete, had to be replaced entirely. Budget: $10 million.
Three years later, with the project about to need an impossible amount of everybody's time for testing and transition, they cancelled it entirely, with $8M spent.
I retired two years after that, after struggling and begging to get $200K spent on those tweaks that were all that were ever needed, content that the system would be fine for a decade to come. Nearly 8 years later, my 2005 system is still the one running. Nobody speaks of the deliberately-forgotten project that was never needed.
And, no, no accountability for anybody, and certainly no "I told you so" was allowed for me.
The IT Department had gone straight up 3 levels above me, to the Director of Water, and told him, Director-to-Director, that the project was needed. After that, every objection could be batted down with "your Director has signed off". At a "Lessons Learned" meeting afterwards, where I did get to say "I told you so" to all the lower staff, and one Manager, I asked how there could be no statement that the Director had made a bad call. (He was retired by then.)
That one Manager came up to me right after everybody left, and told me how unacceptable it was that I tried to bring up the Director, how he was still around there, consulting, had many high-level friends, and naming him was a very career-limiting move. He did not appear in the final report.
A few days later, another Water employee, who dealt with a lot of Managers, told me that she'd asked the Fire Director, at his retirement, why he didn't step in to the mess that became of our 911 system, basically another IT project that collapsed. He told her that at $5M, it was "too small" for him to spend much time upon. And yet, his approval was needed for anything. That system meant that such projects were turned entirely over to consultants as soon as they'd convinced the Director it was on.
Which explained a lot about why all my protests - from the guy who'd managed the previous system - fell upon deaf ears. The Director was just waiting for me to wind down in the climactic meeting I asked two levels for, with him; he'd made his decision a year earlier, planned to spend no more time upon it. If he'd accepted my story, he'd have had to do a hundred hours of work.
Your phrase about demoralizing the staff really hit me. It was the biggest disappointment of a career that I couldn't stop the train-wreck, made me realize how little I was *really* respected, for all the praise heaped upon me.
This continues to be shocking! What is wrong with the public service that they need outside consulting? Many, and I mean many, years ago the public service were able to do the work now farmed out, internally. The latest figures I have are that in 2015 Canada had 257,034 Public Servants and outsourced $8.4 billion. In 2020 there were 319,601 Public Servants and outsourced $11.8 billion. Would you not think with more staff that Government would be outsourcing less? I remember when AAFC could put on an Outlook Conference in Ottawa and document what the outlook was for all farm commodities for the next year and do it well. (attended by approx 800 farm leaders at the old Conference center). Could they organize an afternoon picnic today? I also know as a former MP that the system is slower to solve cases than it was 15 years ago. I would suggest Donald Savoie has it right and it is time for a Royal Commission to fix the problems with the Public Service
Terrific that you are writing about this. Thank you. What's missing from the reporting though is all the govt staff working at McKinsey (former DM of Industry, ISED policy advisors, PMO staff etc.) #revolvingdoor
It’s a particularly daunting challenge because there are three related problems here: the governance failure (routine outsourcing of policy development to consultancies at inflated prices), the integrity failure (consultancies’ self-dealing and governments’ corresponding opacity on conflicts), and the quality failure (the low value of much of the work consultancies produce).
I want to comment briefly on the third point, having seen a fair bit of consulting in the professional services industry. Part of the quality problem stems from the accumulated complacency and even cynicism within organizations like McKinsey, BCG, and the Big 4, where everyone knows they’re being hired not to provide solutions so much as to provide cover. Landing the contract is more important than doing the work, which is why the work product has become so lackadaisical and formulaic. It’s only a slight exaggeration to say that ChatGPT 3.5 could now write a pretty decent McKinsey report.
But the other part of the quality problem lies with the client. When you see a mediocre consulting report, you can often infer that the commissioning instructions were vague, muddled, or even contradictory. Corporate and institutional clients often don’t know what they really want, or can’t articulate their objectives in any meaningful way (or worse, the instructions are basically “reach this conclusion, we don’t care how”). I expect that quite a few consulting contracts are handed out with ill-formed objectives, incorrect assumptions, and fearful self-protectionism. Garbage out, garbage in, if you will.
The whole model of outside professional consultancy feels like it’s gone off the rails, so it’s particularly bad news that so many governments have piled onto the train in recent years. Robust governance reform feels like the best first step towards disentangling governments from the wreckage.
A very important issue. I worked briefly for one of the biggest consulting firms. I also worked in government as a political staffer. In my view, consulting firms add very little value ( recycled powerpoints, produced by recent young MBAs with little real life experience, ) at huge costs. Agree entirely the Senate has some very experienced, smart people who should examine the use of outside consultants. In fact, it’s long past time for a review of our increasingly inefficient, demoralized and unproductive civil service.
All in the incentives are in the wrong direction unfortunately. The only way to fight this, so far as I can see, is to work on exposing some particularly awful specifics and thus cause someone somewhere some political embarrassment, because overall the whole scheme is irresistibly perfect for all concerned.
I read your post about Contract Government. I can provide a view from the ground floor of one federal department.
I spent 37 years working for the government of Canada. During the last ten years of my tenure, I witnessed a significant change in how government operated. I saw a significant uptick in the use of consultants during my last 10 years working for the GOC. In my view, this can be attributed to a couple of primary factors:
A) the overall talent pool within the public service has been severely watered down with fewer and fewer subject matter specialists. Identity politics is the primary cause of this erosion but there was also a total absence of any succession planning strategy involving a transfer of knowledge. Management simply didn’t see a need for such expertise.
B) there was a significant increase in the number of senior managers with little to no knowledge or experience (this was the result of what was aforementioned, the abandonment of the Delegation of Authority instrument and the centralization of same under the Harper regime - but that is a story for another day). These individuals never benefited from a graduated learning curve of financial and hr authorities management. Out of fear, they rallied around each other as a collective to avoid any individual responsibility. Decisions were driven less by results and more out of mistake avoidance. They didn’t want to be blamed if tasks/projects performed by their own team failed. In other words, the use of consultants was a fail safe - from a career perspective.
I would not resort to a Senate-based solution to better learn the dynamics of “contract government”. Short of a political tonsillectomy, let this archaic institution continue to engage in studies that have little to no value and are summarily filed away after completion. Besides, it has been my experience that academics have little value outside a classroom. If one wants to understand what is really happening, talk to those who perform the day to day activities. The challenge then becomes (however) finding someone willing to shed light on subject.
I think the other issue is the separation between strategy development and program implementation which is even an issue when the work is done in house. But this division undermines accountability as the strategy and policy developers don't have to implement or live with the consequences.
Interesting article, Paul. I wonder if part of the reliance on outside consultants comes from the incentives within government for management development. For many years now "Managers" have been seen as generic; "Management" was not linked to specific content areas. Indeed, for career advancement, Managers were encouraged to work in diverse departments - a Manager in Health Canada today could be working in Fisheries tomorrow and Transport after that. A stay of more than two years in any position or department was seen as a career limiting move. I believe a result of this constant Management in Motion has been a hollowing out of our capacity to think deeply about issues.
To be truly effective, Managers require not only management skill, but a deeper appreciation of the issues for which they are responsible, and the ability to think and act strategically on those issues. They also require working relationships with important stakeholders inside and outside government. All of these things are lost when Managers, in pursuit of career advancement, seek constant motion.
I do not know whether this specific issue has been studied in government - these are only observations from the outside. But I can say that I have observed this much greater reliance on outside consultants than in the past, and that they often turn to consultants for content expertise that governments should have internally.
These are a few observations from my career as a consultant. I have seen Canada move from being a world leader in some areas to a country that might do passable work, but certainly not leading edge - many of the Managers, while being bright, motivated people, are not grounded well enough in the substance of their work to be really effective.
I worked for the federal government for 14 years, and as a consultant in Ottawa for 20 years. (I also worked for Nortel and Bell in between.) As a consultant I quickly caame to the conclusion that consultants, in Ottawa at any rate, fell into two groups: Those who worked exclusively for the government and those who worked exclusively for the private sector. After two projects for the government, I realized that wasn't for me. The required quality of work was very low. In return, the pay was very low, and the bureaucratic hassles were enormous.
I preferred the private sector, where the quality expected was much higher, the pay was much higher, and the bureaucratic nonsense was minimal. Heck, I used to take on projects after an oral discussion and a handshake. The quality assurance was maintenance of reputations: it was a small world, and reports of bad work or unfair treatment spread.
The exceptions were Commissions of Inquiry and other extra-public service projects. Here, depending on the Chair, standards could be very high, and he or she usually obtained exemptions from Treasury Board rules and bureaucracy. Here I could do proper work and feel properly compensated, without trying to game the system.
I think that governments have fallen victim to the notion of saving money. They define a minimum acceptable standard and then hire the lowest bidder who may meet that standard. That automatically cuts out the quality work, or indeed the incentive to do more than the bare minimum. That may be one way to earn a living; but it is quite demoralizing for both sides.
The bottom line for me is Canadians should have the right to see these contracts and know where our money is going? If you have nothing to hide, you hide nothing. How do we fix this?
Thanks for writing about this Paul. This 'contract government' is happening across governments at all levels, but is rarely spoken about publicly. I think one of the main points in your article is the impact that this subbing out of work has on the public service. It is, indeed, brutally demoralizing to see work that staff can and should be doing contracted out at exorbitantly higher rates to external agencies for political expediency which are, essentially, largely staffed by former senior bureaucrats anyway. How can the public service expect to succeed in doing this work without the opportunity to even try?
In all likelihood if I were to sit at my kitchen table and share some of this information with my Liberal friends, they would roll their eyes and say, "Another conspiracy theory."
There was a time, 25 - 30 years ago, when government hired experts and experience. You could debate their views and convince them with facts and new perspectives. Ministers listened to them. Wayne makes the point at ag, same at industry and resources.
Now the job is to manage process not decision-making.
There is a genuine loss in capacity in the public sector.
The revolting aspect of all this outsourcing and hired gun consultancy work is the lack of transparency that accompanies it. If the contracts are specifying non-disclosure of the cost and terms of engagement how do Canadians make an objective analysis of the outcome? How do we determine whether this consultant work is to shape public policy or to craft political strategy? (There is a difference.)
A final observation: As noted in the article, the Public Service has grown by leaps and bounds since 2015 and yet basic service delivery in many areas of government is struggling. The Liberal Government has become reliant on consultants, there are plenty of new hires but the work has piled up. This is indefensible and leads me to wonder: what is everyone doing?
I haven't read this, but suspect it'll make fascinating reading on this general topic.
https://www.penguinrandomhouse.com/books/634029/when-mckinsey-comes-to-town-by-walt-bogdanich-and-michael-forsythe/
It's no coincidence that some of the most-spectacular failed projects are in I.T. : the payroll system, ArriveCAN, the original "ObamaCare" web portal. (The TIME magazine cover on the latter is illuminating, describing jaw-droppingly stupid failures that the I.T. newbie can spot at once. The first was that the database of recipients wasn't "indexed" - that's like a library card catalog being in random order, so that you have to search the whole library list for each book found.)
Custom I.T. , you see, was turned over to consultants nearly 30 years ago. Larger companies had "data processing" departments, programmers who spent a career managing the mainframe payroll system, after writing it. But, computing got more complex, and the Data Processing Department struggled to keep everybody on their beloved mainframe. Their "customer departments" that did the actual corporate function, broke out to use PCs and small servers, started calling consultants if the DP department refused to help.
So, about 1995 (in my case), our IT Department flipped to embracing the PC as more than a word-processing toy, started getting programs that ran on servers - which they didn't really understand, so more and more programming moved to consultants.
By 2010, I was begging my bosses to believe that a system I'd had done for $400K, was meeting all needs, had no complaining customers (by "customers", I mean our draftsmen and our internal map-users - it was a drafting system for city maps of water and sewer), didn't really need replacement, just some upgrades and tweaks.
But the IT consultants, very much at the behest of the IT Department that hated that Water had done its own IT, insisted that the system didn't use the coming paradigm of mapping, ESRI GIS, was based on a CAD program that was going to go obsolete, had to be replaced entirely. Budget: $10 million.
Three years later, with the project about to need an impossible amount of everybody's time for testing and transition, they cancelled it entirely, with $8M spent.
I retired two years after that, after struggling and begging to get $200K spent on those tweaks that were all that were ever needed, content that the system would be fine for a decade to come. Nearly 8 years later, my 2005 system is still the one running. Nobody speaks of the deliberately-forgotten project that was never needed.
And, no, no accountability for anybody, and certainly no "I told you so" was allowed for me.
The IT Department had gone straight up 3 levels above me, to the Director of Water, and told him, Director-to-Director, that the project was needed. After that, every objection could be batted down with "your Director has signed off". At a "Lessons Learned" meeting afterwards, where I did get to say "I told you so" to all the lower staff, and one Manager, I asked how there could be no statement that the Director had made a bad call. (He was retired by then.)
That one Manager came up to me right after everybody left, and told me how unacceptable it was that I tried to bring up the Director, how he was still around there, consulting, had many high-level friends, and naming him was a very career-limiting move. He did not appear in the final report.
A few days later, another Water employee, who dealt with a lot of Managers, told me that she'd asked the Fire Director, at his retirement, why he didn't step in to the mess that became of our 911 system, basically another IT project that collapsed. He told her that at $5M, it was "too small" for him to spend much time upon. And yet, his approval was needed for anything. That system meant that such projects were turned entirely over to consultants as soon as they'd convinced the Director it was on.
Which explained a lot about why all my protests - from the guy who'd managed the previous system - fell upon deaf ears. The Director was just waiting for me to wind down in the climactic meeting I asked two levels for, with him; he'd made his decision a year earlier, planned to spend no more time upon it. If he'd accepted my story, he'd have had to do a hundred hours of work.
Your phrase about demoralizing the staff really hit me. It was the biggest disappointment of a career that I couldn't stop the train-wreck, made me realize how little I was *really* respected, for all the praise heaped upon me.
This continues to be shocking! What is wrong with the public service that they need outside consulting? Many, and I mean many, years ago the public service were able to do the work now farmed out, internally. The latest figures I have are that in 2015 Canada had 257,034 Public Servants and outsourced $8.4 billion. In 2020 there were 319,601 Public Servants and outsourced $11.8 billion. Would you not think with more staff that Government would be outsourcing less? I remember when AAFC could put on an Outlook Conference in Ottawa and document what the outlook was for all farm commodities for the next year and do it well. (attended by approx 800 farm leaders at the old Conference center). Could they organize an afternoon picnic today? I also know as a former MP that the system is slower to solve cases than it was 15 years ago. I would suggest Donald Savoie has it right and it is time for a Royal Commission to fix the problems with the Public Service
Terrific that you are writing about this. Thank you. What's missing from the reporting though is all the govt staff working at McKinsey (former DM of Industry, ISED policy advisors, PMO staff etc.) #revolvingdoor
It’s a particularly daunting challenge because there are three related problems here: the governance failure (routine outsourcing of policy development to consultancies at inflated prices), the integrity failure (consultancies’ self-dealing and governments’ corresponding opacity on conflicts), and the quality failure (the low value of much of the work consultancies produce).
I want to comment briefly on the third point, having seen a fair bit of consulting in the professional services industry. Part of the quality problem stems from the accumulated complacency and even cynicism within organizations like McKinsey, BCG, and the Big 4, where everyone knows they’re being hired not to provide solutions so much as to provide cover. Landing the contract is more important than doing the work, which is why the work product has become so lackadaisical and formulaic. It’s only a slight exaggeration to say that ChatGPT 3.5 could now write a pretty decent McKinsey report.
But the other part of the quality problem lies with the client. When you see a mediocre consulting report, you can often infer that the commissioning instructions were vague, muddled, or even contradictory. Corporate and institutional clients often don’t know what they really want, or can’t articulate their objectives in any meaningful way (or worse, the instructions are basically “reach this conclusion, we don’t care how”). I expect that quite a few consulting contracts are handed out with ill-formed objectives, incorrect assumptions, and fearful self-protectionism. Garbage out, garbage in, if you will.
The whole model of outside professional consultancy feels like it’s gone off the rails, so it’s particularly bad news that so many governments have piled onto the train in recent years. Robust governance reform feels like the best first step towards disentangling governments from the wreckage.
A very important issue. I worked briefly for one of the biggest consulting firms. I also worked in government as a political staffer. In my view, consulting firms add very little value ( recycled powerpoints, produced by recent young MBAs with little real life experience, ) at huge costs. Agree entirely the Senate has some very experienced, smart people who should examine the use of outside consultants. In fact, it’s long past time for a review of our increasingly inefficient, demoralized and unproductive civil service.
All in the incentives are in the wrong direction unfortunately. The only way to fight this, so far as I can see, is to work on exposing some particularly awful specifics and thus cause someone somewhere some political embarrassment, because overall the whole scheme is irresistibly perfect for all concerned.
Paul,
I read your post about Contract Government. I can provide a view from the ground floor of one federal department.
I spent 37 years working for the government of Canada. During the last ten years of my tenure, I witnessed a significant change in how government operated. I saw a significant uptick in the use of consultants during my last 10 years working for the GOC. In my view, this can be attributed to a couple of primary factors:
A) the overall talent pool within the public service has been severely watered down with fewer and fewer subject matter specialists. Identity politics is the primary cause of this erosion but there was also a total absence of any succession planning strategy involving a transfer of knowledge. Management simply didn’t see a need for such expertise.
B) there was a significant increase in the number of senior managers with little to no knowledge or experience (this was the result of what was aforementioned, the abandonment of the Delegation of Authority instrument and the centralization of same under the Harper regime - but that is a story for another day). These individuals never benefited from a graduated learning curve of financial and hr authorities management. Out of fear, they rallied around each other as a collective to avoid any individual responsibility. Decisions were driven less by results and more out of mistake avoidance. They didn’t want to be blamed if tasks/projects performed by their own team failed. In other words, the use of consultants was a fail safe - from a career perspective.
I would not resort to a Senate-based solution to better learn the dynamics of “contract government”. Short of a political tonsillectomy, let this archaic institution continue to engage in studies that have little to no value and are summarily filed away after completion. Besides, it has been my experience that academics have little value outside a classroom. If one wants to understand what is really happening, talk to those who perform the day to day activities. The challenge then becomes (however) finding someone willing to shed light on subject.
I think the other issue is the separation between strategy development and program implementation which is even an issue when the work is done in house. But this division undermines accountability as the strategy and policy developers don't have to implement or live with the consequences.
Interesting article, Paul. I wonder if part of the reliance on outside consultants comes from the incentives within government for management development. For many years now "Managers" have been seen as generic; "Management" was not linked to specific content areas. Indeed, for career advancement, Managers were encouraged to work in diverse departments - a Manager in Health Canada today could be working in Fisheries tomorrow and Transport after that. A stay of more than two years in any position or department was seen as a career limiting move. I believe a result of this constant Management in Motion has been a hollowing out of our capacity to think deeply about issues.
To be truly effective, Managers require not only management skill, but a deeper appreciation of the issues for which they are responsible, and the ability to think and act strategically on those issues. They also require working relationships with important stakeholders inside and outside government. All of these things are lost when Managers, in pursuit of career advancement, seek constant motion.
I do not know whether this specific issue has been studied in government - these are only observations from the outside. But I can say that I have observed this much greater reliance on outside consultants than in the past, and that they often turn to consultants for content expertise that governments should have internally.
These are a few observations from my career as a consultant. I have seen Canada move from being a world leader in some areas to a country that might do passable work, but certainly not leading edge - many of the Managers, while being bright, motivated people, are not grounded well enough in the substance of their work to be really effective.
I worked for the federal government for 14 years, and as a consultant in Ottawa for 20 years. (I also worked for Nortel and Bell in between.) As a consultant I quickly caame to the conclusion that consultants, in Ottawa at any rate, fell into two groups: Those who worked exclusively for the government and those who worked exclusively for the private sector. After two projects for the government, I realized that wasn't for me. The required quality of work was very low. In return, the pay was very low, and the bureaucratic hassles were enormous.
I preferred the private sector, where the quality expected was much higher, the pay was much higher, and the bureaucratic nonsense was minimal. Heck, I used to take on projects after an oral discussion and a handshake. The quality assurance was maintenance of reputations: it was a small world, and reports of bad work or unfair treatment spread.
The exceptions were Commissions of Inquiry and other extra-public service projects. Here, depending on the Chair, standards could be very high, and he or she usually obtained exemptions from Treasury Board rules and bureaucracy. Here I could do proper work and feel properly compensated, without trying to game the system.
I think that governments have fallen victim to the notion of saving money. They define a minimum acceptable standard and then hire the lowest bidder who may meet that standard. That automatically cuts out the quality work, or indeed the incentive to do more than the bare minimum. That may be one way to earn a living; but it is quite demoralizing for both sides.
The bottom line for me is Canadians should have the right to see these contracts and know where our money is going? If you have nothing to hide, you hide nothing. How do we fix this?
Thanks for writing about this Paul. This 'contract government' is happening across governments at all levels, but is rarely spoken about publicly. I think one of the main points in your article is the impact that this subbing out of work has on the public service. It is, indeed, brutally demoralizing to see work that staff can and should be doing contracted out at exorbitantly higher rates to external agencies for political expediency which are, essentially, largely staffed by former senior bureaucrats anyway. How can the public service expect to succeed in doing this work without the opportunity to even try?
In all likelihood if I were to sit at my kitchen table and share some of this information with my Liberal friends, they would roll their eyes and say, "Another conspiracy theory."
There was a time, 25 - 30 years ago, when government hired experts and experience. You could debate their views and convince them with facts and new perspectives. Ministers listened to them. Wayne makes the point at ag, same at industry and resources.
Now the job is to manage process not decision-making.
There is a genuine loss in capacity in the public sector.
The revolting aspect of all this outsourcing and hired gun consultancy work is the lack of transparency that accompanies it. If the contracts are specifying non-disclosure of the cost and terms of engagement how do Canadians make an objective analysis of the outcome? How do we determine whether this consultant work is to shape public policy or to craft political strategy? (There is a difference.)
A final observation: As noted in the article, the Public Service has grown by leaps and bounds since 2015 and yet basic service delivery in many areas of government is struggling. The Liberal Government has become reliant on consultants, there are plenty of new hires but the work has piled up. This is indefensible and leads me to wonder: what is everyone doing?