Berkeley Earth, The Arbitrary Use of Parameters to Create a Spurious Correlation


Temperature, CO2, and volcano data | More recent data | High-resolution image The annual and decadal land surface temperature from the BerkeleyEarth average, compared to a linear combination of volcanic sulfate emissions and THE NATURAL LOGARITHM of CO2. It is observed that the large negative excursions in the early temperature records are likely to be explained by exceptional volcanic activity at this time. Similarly, the upward trend is likely to be an indication of anthropogenic changes. The grey area is the 95% confidence interval. (Capitals mine)

The graphic above has been widely used to establish a relationship between human CO2 and temperature. I was unable to replicate it. Stephen Mosher very kindly steered me towards a link on the Berkeley Earth site with an excel sheet.

Using their spreadsheet I first analyzed the components.


The volcanic correlation is interesting, but a very large negative temperature excursion in 1758 seems unsupported by major volcanism, and many temperature drops seem to have begun before the corresponding volcano.

There is certainly no important correlation (.46) between CO2 and the Land Only data. So I wanted to see how the correlation in the Berkeley Earth graphic was achieved. We are given an equation for the “fit”: Fit = alpha + beta * log( CO2 / 277.3 ) + gamma * Volcanic. The values of the parameters are given as:

alpha:  8.342105
beta:  4.466369
gamma:  -0.01515

It is well known that there is an approximately logarithmic diminution in transmission as CO2 in a jar is increased, so a logarithm is a reasonable place to look for getting CO2 in line with temperature.


You can see that the log of CO2 is no fun at all. It is way too flat. We are given the common log, not the natural log, so we take this as a error in the caption of the Berkeley graphic.


I know, let’s add a parameter. We hear a lot about parameterization, but rarely get to see it in action. Here we step through the equation one term at a time.


Hot damn! That’s a much better fit at the end, but there are some jagged toes on that foot.


And with the volcanic parameter the shoemakers have finished their fit.

I am not at all impressed with this process. The author states the rationale:

Because the RCP estimates that most forcing time series are highly correlated to CO2,

it follows that 98.5% of the variance in their total forcing curve can be mapped by just

CO2 and volcanism.

He started with a preconception based on Potsdam forcing data that the correlation must be close, and felt no qualms about using tunable parameters to create a spurious correlation.

True scientists don’t run in those shoes.

Posted in Climate | Leave a comment

Twitter is a Sack

If you can’t figure out what kind of sack…

The real problem is that it is a confirmation bias, groupthink, censored, scored by perceived approval, no dissent allowed, we’re all grooving’, in crowd, don’t rock the boat, typical human doom scenario.

It’s the way we roll. Pick a shaman so you can quit thinking. Watch TV.  Tweet something, knowing it cannot be challenged. Even if you are full of the same thing the sack is.

Posted in Climate | Leave a comment


One often hears it said that the “heat absorbing” properties of CO2 in the atmosphere are “well known” or “well understood”. This is definitely not the case. The properties of CO2 in the atmosphere are presupposed  to be simple, and held up as doctrine, but the investigations outlined below reveal many surprising complexities and ambiguities.

MODTRAN is a very mature model derived from thousands of measurements by very serious folks in the US Air Force with no agenda but to get it right. The public release in a form that allows comparison of both up and down perspectives from any altitude, in one meter increments, and to isolate individual gasses; allows an insight into the role of CO2 in the atmosphere that to the best of my knowledge has not been previously explored.

MODTRAN is a model. It is a good one, but it is not perfect. It is possible that some of the results found here are model artifacts. MODTRAN is about light, not heat. Even though the modern custodians of the program have labeled a principal value “Downward IR Heat Flux”, the currency is photons.

Although interpretations of the surprising results of this investigation are presented, no claim is made that these are the only possible interpretations. In fact, my own thinking has changed as this series progressed.

The first surprising discovery in this series was that in the first 300 meters of the atmosphere MODTRAN looking down sees no deviation from a blackbody spectrum in the CO2 bands; but looking up it sees strong deviation. The upwelling radiation (seen looking down) seems to be so completely thermalized that the atmosphere radiates as a unit, a brick if you will. CO2 is not individually distinguishable in reducing or increasing the radiation.The down welling radiation seen looking up from the same elevation is strongly increased in intensity, and the CO2 bands are clearly distinguishable.

1Modtran up and down 1m

In all of these images the “looking down” is the red background run or “Model Alt” and the blue is looking up.

It is fundamental in physics that radiation takes place equally in all directions (Isotropy). If from any altitude  looking down there is no distinct signature of absorption coming up, we expect that when we turn around from the same place and look up we will also see no distinct signature of absorption  coming down…but we do, and we see the same thing consistently from one meter to 300 meters in elevation.

CO2 molecules should not be able absorb or radiate in one direction only. The same molecules should not appear to be a brick looking down and a powerful absorber looking up. It is interesting to consider the blue curve from which the huge “spike” of downwelling intensity emerges. One should really try to imagine the graphic upside down because the warmest temperatures and greatest intensity are usually at the surface.

The Planck curves are the magic of this general approach to radiation because they unify temperature and intensity. When the blue downwelling curve from which the blue spike emerges clearly is not a Planck function we have no way to know if it harks from temperature (which can be equated to altitude by the lapse rate) or intensity. The most intense and warmest part of the blue spike clearly is a Planck function. The part that conforms corresponds to the fundamental wavenumber 667.4 “bend” and its rotational sidekicks.


In the illustration above the relationship between the wavenumber ranges of the fundamental vibration (bend) of the CO2 molecule and the sloped Planck (blackbody) parts of the up and down CO2 signatures is shown for 5 kilometers elevation. At this elevation the downwelling blue spike emerges from a zero flatline showing only blips of weak third order transitions. The second order transitions (618, 667.8, and 720.8) are also shown. The 667.8 second order bend is so close to, and so much weaker than the 667.4 zero to 1 transition, it is essentially subsumed by it. The relationship of the ranges is what is important at this point. Details of CO2 transitions will be explored a little later.

3Modtran up and down 100m

At 100 meters in the image above we see much the same thing, except that the fundamental CO2 bands looking up are no longer warmer than the brick.

4Modtran up and down 1km

At a kilometer we see deviation in the red from the blackbody curve as evidence of specific absorption (and radiation) in the CO2 bands looking down. CO2 is individually beginning to  reduce the intensity of upwelling radiation.

4Modtran up and down 10km

At ten kilometers the CO2 spectral signature in upwelling radiation is very well developed and we can see a second surprising feature, the kissing conundrum. Up and down welling CO2 signatures always meet (kiss) at the wave number 667.4 spectral spike above 300 meters in the troposphere. We will get back to this this.

In the bottom left hand corner of the MODTRAN screens there are some numbers whose meaning is not immediately clear. “Downward flux” seems easy, but is it for just the selected gas or the entire atmosphere? “Background” seems general. To help understand them they are plotted for the first kilometer of the atmosphere below.


The two series to the left correspond to the “Downward IR Heat Flux” numbers on the MODTRAN screens and the two right series the “IR Heat Loss (Background)” for CO2 only and water only respectively. The background upwelling for CO2 only and water only is very similar.

I have not found a satisfactory explanation of what these numbers represent in either the raw model output or the MODTRAN guidance, but this could be a personal problem. In any event the graphic above constrains the possibilities considerably and the common terms upwelling for “background” and downwelling for “downward IR flux” seem reasonable.

Incoming solar radiation to the planet is thought to be 340 W/m2, so the sheer magnitude of the “background”, always over 400, tells us that it must be the composite radiance of the entire atmosphere. The MODTRAN intention seems to be to subtract the downwelling from the background to get a planetary radiance for each altitude, given your selection of gasses. Below is what this looks like.


Both CO2 only and water only “net” radiation increase in intensity with altitude. in a linear way up to 5 kilometers and then begin catenary regressions to the values that remain unchanged to 60 kilometers. There is clearly something very important about the 5 kilometer level. (this figure and comment were changed 11/27 due to my spreadsheet error) If we assume this net radiance is isotropic, CO2 only would appear to cool the planet as its net radiance exceeds 340 W/m2 for most of the curve.


MODTRAN addresses the IR part of the spectrum from wavenumbers 100-1500. The sun, at 5800K does not radiate significantly in these spectra. All the energy in the MODTRAN part of the spectrum harks from the surface of the planet. The atmosphere is initially radiated and otherwise heated from the bottom up.

With this in mind it seems interesting to subtract the values of “background” upselling radiation for progressively higher altitudes from the lowest altitude in MODTRAN (half a meter). We might expect that background upwelling would diminish with altitude, leading to a steady increase in the differences. This exercise was performed for CO2 only and water only. Below are the results for the first kilometer.


CO2 was done first and the very surprising result was that CO2 differences were “quantized”. The data points were chosen to parse out the “quanta”. The stepwise progression in energy is very regular and appears to be a model increment. The stepwise progression in altitude is very irregular. The same data points were used for water. There could be quanta for water too if its data were systematically parsed at meter scale, but they would not match those of CO2. This effort will focus on CO2 and use water only for a frame of reference.

The other surprising result is that the background radiance is higher than the surface leading to negative differences for both CO2 only and water only. The background does not drop back to surface (half meter) levels until maybe 370 meters for water and 400 meters for CO2.

How can this be? How can “background” radiation that emanates from the “ground” increase with altitude?

It is very interesting in this regard that the 400 meter level  where background upwelling values higher than the surface end for CO2 onlyis the level we begin to see a signature of specific CO2 absorption.


Four hundred meters is the top of the “brick” for CO2 only.

Energy can  be neither created nor destroyed. Neither can it be amplified without borrowing. Greenhouse gasses do not create energy. They can absorb it and transform it, but there is always a processing loss to other forms of energy. The increase in radiation above the surface could rely on energy “borrowed” in the form of downwelling from higher levels.

We can evaluate this possibility with the downwelling difference from the surface shown below.


It can be seen that downwelling differences from the surface are very linear and decline in energy with altitude. This is not promising. One would expect an increase in downwelling in the same 0-400 meter zone if it were the cause of upwelling values higher than the surface.

How can CO2 only radiation be stepwise?

The stepwise altitude progression of CO2 shows very equal increments in energy (.31 W/m2) but altitude increments ranging from 14 to 386 meters. These could represent layers with different properties, but the layers would apply to CO2 and not water.


We have painted a picture of CO2 acting strangely in the atmosphere. It appears to radiate in only one direction below 300 meters; its Planck intensity up and down always matches; it shows a pronounced stepwise altitude variance with the surface; and its upward radiance is higher than the surface up to 400 meters.

What could account for this strange behavior?

Possibly unique spectral properties.

Orders of CO2 Transitions

Carbon dioxide is unique among the major greenhouse gasses in having what is called a “Q branch” in the arcane. The HITRAN  image next below (thanks to Barrett Bellamy) shows what this means. The central spike is the “Q” branch corresponding to the 667.4 fundamental bend. The “P”  are rotational transitions gained along with the bend that reduce the energy of the molecule. The “R” are rotational transitions also gained with the bend that increase the energy. It is a package deal. You do not get the P and R without the Q, and the P and R substantially cancel out.

Similarly, in the image above it can be seen that all of the CO2 transitions are dependent on the fundamental bend or Q branch. They all spring from the first transition rather than the ground state. Another package deal. Without the fundamental bend the other transitions do not exist.

The image above also illustrates the preponderance of total energy represented by the fundamental bend. This indicates very powerful absorption. It also shows how the second order transitions are much weaker, and that all of these transitions are saturated at 280 ppm pre industrial levels of atmospheric CO2.

No light in either the first or second order bands has made it from the surface to the tropopause since the industrial revolution.

The third order transitions are in turn entirely dependent on the second order, and they are orders of magnitude weaker. Only these third order bands are unsaturated, and only these can further warm the “planet”. The saturated bands are already completely absorbed, and increasing CO2 in the atmosphere merely moves the complete absorption level closer to the surface .


The image above also shows the absorption fraction in the fundamental bend at 400 ppm CO2.It can be seen that the Q branch, on which the P and R depend, is very nearly completely absorbed in one meter.

The resolution of MODTRAN is one meter.

This may help explain the kissing conundrum and how there could be no distinct signature (the brick) looking down and a clear signal of powerful absorption looking up. The Q branch spikes “kiss” looking up and down because this CO2 band is an extremely powerful absorber in both directions; and, the entire CO2 absorption spectrum rests on its shoulders. With one meter resolution, MODTRAN can be seeing different molecules looking down and looking up, and nearly the complete absorption section for the fundamental 667.4 bend is contained in the same meter. We could be seeing downwelling energy warming the brick in the upper part of the one meter section.

The stepwise altitude progression of CO2 upwelling differences from the surface appear to describe atmospheric layers with no change in CO2 only upward background radiance. The deepest such layer extends from 14 meters to 400 in altitude. The following layer is also 14 meters to 414. Above this the steps are more regular, ranging from 69 to 88 meters. I have no explanation to offer.

The increase in upward background radiance above the surface for both CO2 only and water only is also difficult to explain. The top of this excess radiance seems to match the top of the “brick” for CO2 only, but the top of the brick for water only is much higher than its top of excess radiance, closer to the level at 5 kilometers. where both CO2 only and water only reach their maximum net radiance. CO2 and water differ in many aspects of their greenhouse behavior. Water never “kisses” up and down, for example. This excess energy phenomenon may have something to do with the intense radiative exchange between the surface and the lowest atmosphere described as the photon food fight. Kevin Trenberth’s energy budget gives this as 398 W/m2 upwelling and 340 W/m2  (same as TSI) downwelling taking place simultaneously. These numbers  square fairly well with MODTRAN, which gives a background upwelling value of 417 W/m2 and composite downwelling of 348 W/m2 at one meter. Perhaps the photon food fight acts like a reflector oven.

It is widely known that global temperature is not currently following the model script. Temperature controls the variation around the trend of increasing atmospheric CO2 even today.It is  widely acknowledged that CO2 follows temperature in the ice and benthic cores extending back at least several million years. CO2 does not appear to control temperature at any time scale.

The obvious and unaddressed question is, “Why not?” Hopefully this exercise provides the partial answer that we still have a lot to learn about how CO2 works in the atmosphere.

However one choses to interpret these investigations, it should be clear that the “heat absorbing” properties of CO2 (and water) in the atmosphere are not “well known” or  “well understood.”




Posted in Climate, MODTRAN | Leave a comment

Modtran Up and Down VI

This series is getting long, but there is one more avenue to explore before it can be finished. We left off with the kissing conundrum that was not completely resolved. That’s ok. We may not resolve this problem either. The kissing problem is that up and down looking Modtran intensity/blackbody values for CO2 spectra always kiss, even below the 1 km altitude where CO2 spectra are indistinguishable from blackbody in Modtran.

What we must explore to finish this series is what changes take place at a kilometer altitude to allow CO2 to begin radiating? To be clear, CO2 must radiate below a kilometer altitude to some extent, but it is absorbed so quickly that the Fourier Transform spectrometers used to gather the data for Modtran  see the atmosphere below one kilometer as a brick. A brick where individual molecular spectra are not apparent. A brick that radiates very close to the blackbody curve.

4Modtran up and down 1km

Above you can see the first hints of deviation from the blackbody spectrum looking down at one kilometer in the red curve, and the very strong deviation in the blue curve looking up.

3Modtran up and down 100m

This can be contrasted with the 100 meter or 1000 MB level where no deviation from the blackbody is seen in the CO2 bands looking down.

What has changed?



This is what has changed. A new source of energy from the release of enthalpy of vaporization becomes available at the condensation level at about 1 kilometer altitude. This is the typical altitude of the bottom of the cloud deck.

We can summarize this entire Modtran up and down exercise with the following image.


The background image credit to the Cloud Appreciation Society.

The principal greenhouse gasses do not absorb or radiate separately from the mass of the atmosphere below one kilometer. This is partly a pressure effect. All outgoing IR radiation is completely thermalized below this level and energy transport takes place by conduction and convection.

Above one kilometer we begin to see distinct outgoing radiation in the CO2 bands. Above two kilometers we begin to see distinct outgoing radiation in the water bands.

Outgoing radiation in CO2 bands increases continuously from one kilometer until it reaches a maximum about 17 kilometers. Outgoing radiation in water bands increases steadily from two kilometers and also reaches a maximum  about 17 kilometers.



That they both should reach maxima at the same altitude is very interesting. Who knows? We may have do do another post in this series some day.







Posted in Climate | Leave a comment

MODTRAN Up and Down V, The Kissing Conundrum


We left off the last post with the puzzlement that Modtran looking up and down from the same altitude sees CO2 deviation from the blackbody spectrum, evidence of CO2 absorption and radiance, that matches almost exactly in intensity.

4Modtran up and down 10km

This is illustrated well here from 10 kilometers.


When we zoom in a bit we can see that what is really kissing is the wavenumber 667.4 “spike”. The rotational “shoulders” of the fundamental bending spike are sloped to their respective Planck temperatures. The red upwelling (looking down) shoulders are at 237 K, the lapse temperature for 10 kilometers. This can be found in Modtran by “mouse over” in the lapse graph to the right. The mouse over gives you a pointer dot but no values in the main panel. Using my eyechrometer, I declare that the Planck temperature of the blue downwelling (looking up) shoulders is about 235 K. This corresponds to an altitude about 300 meters higher.

We can summarize that our theoretical spectrometer looking up and down in the CO2 bands at 10 kilometers sees radiation at the blackbody temperature for the altitude of the instrument for the rotational shoulders looking down, but sees the same radiation from a slightly lower temperature and higher altitude than the instrument looking up.The kissing spikes at 667.4 match nearly perfectly.

To get to the bottom of this we need to delve into the rotational shoulders.


This is a HITRAN plot of the absorption fraction in one meter for the CO2 molecule (credit Barret Bellamy). The Q branch, which is almost entirely absorbed within one meter of atmosphere with 380 ppm CO2, is vibrational only and corresponds to wavenumber 667.4. The P and R bands to the left and right represent situations where the 0 to 1  quantum vibrational transition is complicated by part of the energy being diverted to rotation. The P rotations decrease the overall energy of the molecule and the R rotations increase the overall energy. The rotational spectra largely cancel out with a slight net energy increase due to a slightly stronger R branch.

The important point here is this entire spectrum must be viewed as a unit entirely dependent on the fundamental 667.4 “bending” (vibration).


This is what our fundamental 667.4 unit looks like plotted against a bunch of stuff, notably various CO2 spectra seen from space and our 220 K Planck curve. It can be seen that the 667.4 unit defines a gap of zero transmission to the tropopause from the surface, and that this same 667.4 unit (and gap) corresponds to the inclined shoulders of the CO2 spectra the satellites see from space.

With this context, the conundrum we seek to explain is why this 667.4 unit when seen looking down, and inverted looking up, and inclined to the Planck curve; always shows the 667.4 Q branch kissing. This phenomenon is all the more amazing as the Q branch does not appear as a noticeable spike until 10 kilometers at Modtran detail, yet the up and down values for the 667.4 unit match even below 1 kilometer where the upwelling (looking down) CO2 spectra are no different than the Planck curve.

To summarize:

1Modtran up and down 1m

From 1 meter,

4Modtran up and down 30km

to 30 kilometers, the values are ALWAYS equal looking up and down at wavenumber 667.4 for CO2 only. This behavior is unique to CO2 among the main greenhouse gasses.

There is some fundamental limit to the total up and down radiation at 667.4 between the surface and 30 kilometers. My suspicion is this relates to the extraordinarily short (1 meter at surface pressure) extinction path for this wavelength in the 2 good greenhouse gas. Furthermore, since CO2 is a linear molecule with a dipole moment of zero, the zero to one quantum transition at 667.4 is the basis of ALL subsequent vibrational and rotational transitions.

Orders of CO2 Transitions

Not only are the rotational shoulders of 667.4 a “unit”, but the entire CO2 spectrum is essentially a unit. Whatever limits 667.4 limits everything else.

While this seems a good start, exactly how this would limit the total up and down radiation seen by Modtran at 667.4 is not clear to me. Perhaps this is a new level of meaning for “saturation”.

Posted in Climate | 2 Comments

MODTRAN Up and Down IV, Moving Higher

Time to make good on the promise to progress upwards. In the first post in this series we established that Modtran sees no significant deviation from the blackbody spectrum looking down in the CO2 bands until about one kilometer altitude. Looking up, Modtran sees classical deviation from the blackbody spectrum, indicating absorption and radiation by CO2 at every altitude.

In the second post in this series we established that essentially the same is true for water, except that the altitude of first radiation in water bands looking down is four or five kilometers altitude.

In the third post in this series, we surmise that the reason Modtran can see no radiation looking down in the CO2 and water bands below their respective first altitudes, yet can see strong signals all the way to the surface looking up; is that the higher energy quantized vibrational states of the molecules at lower altitudes are transparent to lower energy radiation from above.

Where we left off at one kilometer, CO2 was just beginning to show it’s stuff looking down for the first time above the surface.

4Modtran up and down 1kmHere it is. Let’s go to five kilometers.

4Modtran up and down 5kmBig difference. Ten kilometers.

4Modtran up and down 10kmThe upwelling signal now has greater intensity that the radiation from above.

4Modtran up and down 16kmAt sixteen kilometers we are at the tropical tropopause at the beginning of the little “vertical spot” that extends to 17 kilometers. Within this kilometer, the atmosphere basically does not change temperature as you climb. Above 17 kilometers the atmosphere begins to warm with increasing altitude.

4Modtran up and down 30km

The trend beginning at one kilometer has been increasing upwelling and decreasing downwelling radiation. The really remarkable thing is that even from one meter in the first post, the temperature/intensity of the down and up looking radiation have been the same. The down and up looking values “kiss”. This trend begins to break down at 30 kilometers above, with only the 667.4 band from above stretching mightily for the kiss.


At 50 kilometers the lovers are finally separated.

What does this mean? The down and up looking values for water never match.

Up and down Water only 5km

Above is water only up and down at its heyday altitude of 5 kilometers. No hanky panky here.

We will explore the remarkable equality of up and down CO2 values in the next post.


Posted in Climate | 1 Comment


Further delaying a repeated promise to progress upwards in prior posts in this series, it seems important to address an obvious question: why, when you look up and down from say 10 meters, can you see deviations from the blackbody spectrum indicating radiation from greenhouse gasses looking up? In other words, if we are going to say that no signal is seen looking down because such radiation is instantly and completely absorbed and thermalized, why doesn’t this happen to the radiation from above?

Always in the back of your mind there is the possibility that Modtran is just wrong, or somehow inadequately designed for this up and down exercise.

modtran-up-and-down-full-bore-10-meterHere is Modran full bore, as I call it, with all the greenhouse gasses; looking down with the red background run and up with the blue looking up at ten meters. At first glance one is inclined to say the signal from above is all water. We can check that.


What we have done here is leave the full bore signal looking up from ten meters in red as the background run compared to water only looking up from the same level in blue.

Interesting. In the CO2 bands from wave numbers 600-750, the water only signal comes from a much higher and colder altitude. This information is obscured in the full bore view because the virtual spectrometer registers the higher energy near surface thermalized radiation along the Planck curve to about WN 700 and then the “tops” of the water bands.

Through the atmospheric window the energy and altitude differences are much smaller and the blue water only signal barely “pokes out” below the full bore signal except in the ozone bands where the full bore lines “jump up”.


Using the same comparison between full bore and ozone you get the idea…ozone behaves peculiarly because there is significant surface ozone besides the better known stratospheric zone.


Just to complete the train of thought we compare full bore with CO2 only. CO2 agrees with water in the deviation from blackbody in the ozone bands.

After all this, how do we answer the initial question: why is radiation from above not absorbed and thermalized just as radiation from below? Why do we see a deviation from the blackbody temperature looking up but not down?


Getting back to the first graphic of full bore up and down from 10 meters, the answer is that the radiation from above is coming in at a lower energy than the radiation from below. The vibrations that dominate the molecular excitations in the infrared spectrum are quantized. The higher quanta/ higher energy molecules both below 10 meters and above are transparent to lower energy photons from above. They just sail right through.



Posted in Climate | 1 Comment