Have you noted as I have how agencies, once established, endure well past their function? Consider the work of the foley artist.
This sound-effect job was birthed with radio, giving depth to radio drama, drawing the listener in to the action. With the advent of movies and television, though, the foley artist continued. Once a job is made, the enterprising continue to work hard to find their niche.
The same goes for the bureaucrat. Once graced with a budget, an enterprising bureaucrat will work hardest to justify his or her continued necessity, even perhaps at the cost of emerging needs. How does one go about disbanding an agency that has outlived it's usefulness? There will be protests and pain, for sure, as its supporters mourn it's loss.
An agency that may have outlived their origins for example, is the Victorian Order of Nurses (Florence Nightingale's practical nursing has proved itself more useful). In their history linked here, the VON use emotive language like "vital" and "treasure". Is there any service unique or valuable that the VON provide that could not be absorbed in to regular nursing? This is an example of an agency, budget, and bureaucracy that frankly, isn't needed.
Squirreling out obsoleted bureaucracies and agencies is one way to keep taxes low while improving service levels. This is not painless or simple though. There will be loud, emotive protests. This is the sort of activity that a new government may want to consider in the early days of their administration, before re-election looms.