I keep seeing projections about the future being made - but they seem to use seemingly current data.
In fact many organisations over the past few years have made many analyses & reports.
Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
Is there a branch of statistics or economics which uses known errors in OLD data to correct or adjust NEW estimates?
If so, are the governments, oil gurus, oil corps ... and us here ... using these techniques?
PeakOil is You
What can we estimate from ERRORS in old reports?
First unread post • 4 posts
• Page 1 of 1
What can we estimate from ERRORS in old reports?
by SoothSayer » Tue 09 May 2006, 12:36:33
-
SoothSayer - Heavy Crude
- Posts: 1167
- Joined: Thu 02 Mar 2006, 04:00:00
- Location: England
Re: What can we estimate from ERRORS in old reports?
by MrBill » Wed 10 May 2006, 04:40:11
$this->bbcode_second_pass_quote('SoothSayer', 'I') keep seeing projections about the future being made - but they seem to use seemingly current data.
In fact many organisations over the past few years have made many analyses & reports.
Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
Is there a branch of statistics or economics which uses known errors in OLD data to correct or adjust NEW estimates?
If so, are the governments, oil gurus, oil corps ... and us here ... using these techniques?
In fact many organisations over the past few years have made many analyses & reports.
Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
Is there a branch of statistics or economics which uses known errors in OLD data to correct or adjust NEW estimates?
If so, are the governments, oil gurus, oil corps ... and us here ... using these techniques?
Most projections fail for one of two reasons. One they are linear and expect historical trends to continue far into the future without taking into account regression to the mean. And secondly, they are regression analysis which assumes that the future will look like the past without taking into account new inputs that change historical patterns. Take your pick.
Yes, we can say for example, that because Core CPI without the more volatile food & energy component consistantly under estimates inflation, so all we have to do is look at Headline CPI and we have solved the problem. Or you can say, no, if you look at Headline CPI, you are double counting and over-estimating Core Inflation, as eventually the higher Headline number including food & energy will be reflected in higher Core inflation if and when it feeds through into the Core number.
Is it not better to take a range of possibilities and assign probabilities to their outcome based on a set of assumptions, and then red flag the assumptions, so you know when one assumption changes (from forecast to a hard number) that you know to change your underlying prediction accordingly.
If oil prices remain in the $70 range than this may add 0.5% to underlying inflation in 2006 over and above core inflation. Then if oil prices overshoot $70 and move to $100 you can add 0.5%(+) to your underlying inflation forecast. And if oil falls below $70 then you might reduce your estimate of inflation by 0.5%(-) over and above core inflation.
Therefore, I would suggest that the underlying assumptions are more important than the forecast itself. Then you manage by exception. That is measure how much variance there is from the original prediction. It is not that the prediction was wrong or worthless. But sometimes the underlying assumptions change. These are your feedback loops. The red flags.
The organized state is a wonderful invention whereby everyone can live at someone else's expense.
-
MrBill - Expert
- Posts: 5630
- Joined: Thu 15 Sep 2005, 03:00:00
- Location: Eurasia
Re: What can we estimate from ERRORS in old reports?
by Doly » Wed 10 May 2006, 08:45:27
$this->bbcode_second_pass_quote('SoothSayer', '
')Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
')Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
You are happily assuming here that errors are biased always in the same direction. Generally they aren't. Oil prices might be an exception, in that certain forecasters may have had a bias towards low or high oil prices.
-
Doly - Expert
- Posts: 4370
- Joined: Fri 03 Dec 2004, 04:00:00
Re: What can we estimate from ERRORS in old reports?
by MrBill » Wed 10 May 2006, 09:16:53
$this->bbcode_second_pass_quote('Doly', '')$this->bbcode_second_pass_quote('SoothSayer', '
')Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
')Can we learn anything from seeing how far predictions in these old reports were in error?
For example if reports are consistently 10%-20% too low for a predicted data item, can we assume that CURRENT estimates might also be in error by that amount?
You are happily assuming here that errors are biased always in the same direction. Generally they aren't. Oil prices might be an exception, in that certain forecasters may have had a bias towards low or high oil prices.
I think that looking at time series data and using the same methodology to collect the data and make predictions from it that most often there is a directional bias versus a random error in it? Maybe there are exceptions, but I would assume a government agency would consistantly gather information in the same manner each month. Then when the methodology changes as it often does they restate the historical data to make comparisons more representative. At least in my experience in any case.
The organized state is a wonderful invention whereby everyone can live at someone else's expense.
-
MrBill - Expert
- Posts: 5630
- Joined: Thu 15 Sep 2005, 03:00:00
- Location: Eurasia