Verification of
2005 Old Farmer's Almanac:
National and California

by Jan Null, CCM
Golden Gate Weather Services

 

ABSTRACT
The latest Old Farmer's Almanac is out with its usual boast of being 80% correct.  But are they really that good?  From what I have seen in past years their accuracy is much, much worse than advertised!  How about this year?

This study found that the national regional forecast for Winter had both the temperature and precipitation correct only about 13% of the time and Summer forecast was correct just 6% of the time.

The California forecasts for temperature were right only 13% of the time and the precipitation was correct 27% of the time.


DATA
Last November's 2005 edition of the Old Farmer's Almanac (Yankee Publishing, Dublin NH,  2004) was examined.  There are seasonal weather graphics for winter and summer (Fig. 1) from the 2005 Old Farmer's Almanac (OFA).  The weather forecast section of the is also divided into 16 regions.  Region 16 encompasses the southern three-fourths of California and is broken down by month with "forecasts" of how the temperature and precipitation compare to normal. 

Figures 2 is the actual forecast for Region 16 from the Almanac. The OFA seasonal forecasts for the nation were evaluated for the Winter (November through March) and Summer (June through August) using the temperature and rainfall departures from normal by month for the United States Climate Divisions (NOAA-CIRES Climate Diagnostics Center US Climate Divisions Plotting Page).

Fig. 1 Fig. 2


To evaluate the Region 16 forecasts each monthly forecast was compared to the actual monthly temperature and precipitation anomalies by climate division
.  Data for climate divisions was chosen because of the large number of sites that are used in determining the division average, thus eliminating the bias of a single station.  Temperatures were evaluated for all 12 months while precipitation was only looked at for the the California "wet" season of November through April.


Seasonal U.S. Climate Division Temperatures and Precipitation (Nov. 2004 - Mar. 2005) and (Jun. - Aug. 2005)

Winter Temperature Winter Precipitation Summer Temperature Summer Precipitation


The "graded" seasonal forecasts below are a subjective verification of the Old Farmer's Almanac seasonal forecasts.  The basic methodology was to deduct a grade for each part portion of a regional forecast that was incorrect and two grades if the sign was also incorrect.  If both the temperature and precipitation forecast were correct and had the appropriate magnitude then the region would get an "A" grade".  Overall the seasonal forecasts averaged a grade of about "C", meaning at least the precipitation or temperature was wrong.  Only 2 of the 16 (13%) Winter forecasts got both the temperature and precipitation right and just 1 of the 18 (6%) Summer forecasts verified.

"Graded" Seasonal Forecasts

Winter Summer

 

Monthly U.S. Climate Division Precipitation (Nov. 2004 - Apr. 2005)

Nov 2004

Dec 2004

Jan 2005

Feb 2005
Mar 2005 Apr 2005    

 

 

Monthly U.S. Climate Division Temperatures (Nov. 2004 - Oct. 2005)

Nov 2004

Dec 2004

Jan 2005

Feb 2005

Mar 2005

Apr 2005

May 2005 Jun 2005
Jul 2005 Aug 2005 Sep 2005 Oct 2005
       

ANALYSIS
The subjective analyses in the table below compares the monthly forecasts for Region 16 from the 2005 OFA with the actual observed monthly climate division anomalies of temperature and precipitation.  Because of the scarcity of rainfall in the California in the summer months, precipitation for the months May through October was not evaluated.  Furthermore, if either the forecasts or the observed data were geographically split then separate analyses were done for each geographic area.

The observed data has been color-coded to reflect each of three categories.  Forecasts which were judged correct are blue.  Those which had the right sign (i.e., above normal observed when above was forecast or below normal observed when below was forecast) but where the quantity was incorrect are coded in yellow.  Forecasts that had a large range that was partially corrected are colored orange. And forecasts which had the wrong sign are coded red.

SUMMARY
For California, Region 16, a total of 11 precipitation forecasts and 23 temperature forecasts were evaluated.  Of the precipitation cases 27% were correct, 36% were quantitatively wrong and the remaining 36% had the incorrect sign.  The temperatures for region 16 were only correct 13% of the time, had the wrong sign in 22% of the cases, were quantitatively wrong 39% of the time and were partially correct 26% of the time.  Overall the California forecasts were correct only 23% of the time.


Table 1.  Analysis Region 16 - California

Precipitation

Temperature

  OFA Observed OFA Observed
Nov. 04 1" below normal 2"-3" below normal North
Normal South
4 above normal 0-2 below normal North
2-4 below normal South
Dec. 04 1.5" below normal 1"-3" above normal 1 below normal East
1 above normal West
0-1 above normal North
Normal South
Jan. 05 2" above normal 0"-1" below normal North
2"-4" above normal South
2.5 above normal Normal
Feb. 05 1" below normal 2"-3" below normal North
0"-1" above normal Central
2"-4" above normal South
Normal 1-3 above normal North
0-2 below normal South
Mar. 05 1" below normal 0"-1" below normal North
Normal South
3.5 above normal 1-3 above normal North
0-2 above normal South
Apr. 05 Normal Normal 3 below normal East
Normal West
2-3 below normal East
0-2 below normal West
May 05     2 above normal Normal North
0-1 above normal South
Jun. 05     1 above normal 2-4 below normal
Jul. 05     1 below normal North
2 above normal South
2-4 above normal North
0-3 above normal South
Aug. 05     1 above normal 0-2 above normal North
Normal South
Sep. 05     3 above normal 0-3 above normal North
0-2 above normal South
Oct. 05     Normal Normal North
0"-2" below coast South
1"-2" above inland South