Scientific Wild Ass Guesses, Rectally Extracted Estimates and Risk Assessment
SWAG (Scientific Wild Ass Guesses) and REE (Rectally Extracted Estimates) are sufficiently commonplace in engineering that these terms will be appearing in my Dictionary of Chemical Engineering, out next year. In case you wondered what a REE is, it's a number you “pulled out of your ass”.
There are also “plug numbers” (numbers you plug into a formula, which you may or may not have pulled out of your ass), “guesstimates”, the “guess and check” methodology, etc. Engineering is an approximate business.
Those mathematicians I regularly see on LinkedIn berating engineers for doing qualitative risk analysis exercises, rather than “proper” “rigorous” mathematical quantitative risk assessment, clearly do not understand this.
There are really only qualitative risk assessments. Doing precise calculations based on, at best, SWAGs (as all meaningful estimates of the risk of something happening are) is a sure-fire way to generate exactly the kind of spurious precision they accuse engineers of exhibiting, especially in respect of the ubiquitous risk matrix. It’s far better to be knowingly approximately right than unknowingly precisely wrong.
I can see why outsiders are confused, but proper engineers actually do understand the strengths and weaknesses of the risk matrix. It’s quick and easy to produce, and produces a nice graphical representation of the amount of effort which should be expended in addressing some risk.
Anyone who starts thinking that the numbers associated with bands of probability sometimes added to a risk matrix mean something, has lost track of where they came from. Anyone who wants to start doing sums with these numbers is an idiot.
So why do some engineers find themselves being pressurised by managers to do just this? Well, what should happen, if it is decided to consider proceeding with something which the risk matrix suggests might be a bit dodgy, is to generate some better estimates. There are only estimates in engineering, but a SWAG is better than a REE.
Better estimates however cost more money. It’s far cheaper to pressurise some hapless engineer into agreeing to stand by his SWAG than it is to pay for a better estimate. I have personally found that a lot of engineering managers aren’t really engineers, whatever their degree certificate says.
Their career path takes them out of engineering itself and into management ASAP, so they miss out on developing real engineering judgement. What they develop instead, all too often, is the political ability to take credit for the achievements of others, and to dodge the blame when things go wrong.
Getting someone else to take responsibility for a risk estimate may well be the most economical way to cover their own arses, but it doesn’t actually promote process safety.
Most mathematicians, and quite a few managers, however think that engineering judgment means nothing. They don’t want opinions – they want facts! They want scientific rigour! They want precise estimates of the cost of each outcome in a risk matrix, and precise estimates of the probability of each outcome. Then science and maths can be used to produce a scientifically valid answer.
Some of those managers even hire mathematicians and/ or use modelling software to produce the precise answers they crave. At least the mathematicians will state their assumptions, whereas the thousands of default assumptions built into modelling software will be invisible.
In both cases, the apparent rigour of the methodology lends the answers a false appearance of correctness. Precision is not however the same as accuracy.
It is also the case that a lot of engineers, not to mention many well-established international standards, don’t even agree with each other about what risk is. This is something I came to realise during the consultation exercise for my dictionary.
The idea implicit in the risk matrix – that “risk” is the probability of something bad happening, multiplied by the severity of it happening – is itself highly contested. Even if you did manage to expend the time and effort required to produce exceptionally accurate estimates of probability and severity, many would question whether what you had calculated should be called “Risk”.
It matters far more to know what matters, how much faith can reasonably be put in it, and how much certainty is appropriate to the case in point, than it is to insist on spurious precision in either language or calculation.
