Government departments could build up internal teams of experts. (They already have this in technical fields where then Big 4 don’t have expertise.) But government pay is crap compared to these giant corporations, so the best will be scooped up by the private sector while the less impressive employees stay employed for life in government.
To make it work would require a different system to what the government has now. A system with merit-based remuneration and much simpler hiring-and-firing.
(I say this as someone who originally worked for the government, then moved to the private sector.)