If you don’t like Fourier transforms, you won’t like this post – indeed, I’m only writing it for myself but comments are still welcome.A single point
A single sample is achieved by multiplying a waveform by what is known as a delta function. That is to say, by a function which has no value (or zero) at every point except the single point that is being sampled.
Such a function has a frequency response which is flat across the entire frequency domain but whose angle varies according to the offset of the point from zero. In other words, it looks rather like White noise.
A sample of data
A sample of data is a period during which samples are taken. As such it is the multiplication of the waveform being sampled by two step functions. One is zero at all points except those after the start, the other is zero at all points except those after then end.
Step functions have 1/f type frequency response, so each one imposes a 1/f type response. But the offset between the two “twists” the response to produce a “ring” whose periodicity is determined by the difference in time between the two steps.
In the frequency domain, this is a sinx/x function where the frequency of sinx is determined by the length of time of the data. (Here I’m assuming data is over a time, but other variables are as valid).
So, sampling imposes a 1/f type function (albeit sinx/x) onto the waveform. In other words, sampling reduces the scale of long-term noise so that the longer the period of change, the less of it can be seen (it’s not rocket science!).
Now a trend is just the integral of a step function. And a step function has a 1/f type frequency response, so the integral will be a 1/f2 type response imposed on the data. However, we never sample a trend over all time (except in theory), so it tends to be a sinx/x3 type of response – averaged (??) . However, if we look at very low frequency data (much longer than sample period) this becomes 1/f2
And if we impose this in 1/fn type noise (0→n→2), the effect of taking trends is to change this into 1/f(n+2) where (2→n→4). So really boosting low-frequency, long period, long-term change.
In other words, by taking the trend of 1/f noise, we are taking a noise source that is already very heavily biased toward long period noise and exaggerating that long period noise. So, for climate, whereas we might see a decadal trend, what we may really be doing is exaggerating the century-scale noise so that it now exceeds any decadal (presumed induced) trend.
So, what is a trend?
Producing a trend is achieved by a multiplying by a trend function. If we then produce a “trend over time”, this is the convolution of the trend function in time-space which is the same as multiplying by the fourier transform of the trend function in frequency space.
In other words, the removal of short-period noise. The rational for doing this is that it is assumed that any “signal” is a step function or “something applied”, which has a frequency response of 1/f. So, taking the trend, is in effect is a very strong low-pass filter. As such it has the effect of removing the short-term noise which has a low intensity for a step-wise signal but is large for white noise and increasing the long-term frequencies where a step-wise signal is stronger than the noise because of its 1/f frequency response will be larger than any “white noise” which has the same amplitude at all frequencies. So, this in effect is just dramatically increasing the ratio of signal to noise.
However, it all falls apart if the noise is 1/fn type where n>1, because now a step wise input “causing” a change, will have a 1/f type signal which becomes increasingly smaller relative to the noise, the lower the frequency (longer period). So, whereas for white noise, taking a trend significantly removes the noise to leave the signal in contrast where we have 1/fn type noise (where n>1), taking the trend increases the noise to (step) signal effectively removing the signal!
In other words, we cannot use a trend (or other low pass filter) to see “climate change” because the step-wise “change” has a 1/f type noise.
Techniques like trends are only valid when the signal is expected to have a frequency response whereby the low frequencies exceed so of the noise. So, e.g. if climate has a 1/f3/2 type noise, then we do not increase the signal to noise of a “change” (step) which has a 1/f type noise function, but we do increase the signal to noise if the signal is a trend with a 1/f2 type frequency profile.
This is why I’m so horrified by all these assertions from climate academics about “knowing” the climate is changing. Because you cannot possibly talk about the significance of climate “change” without first being very clear about the natural variability and frequency profile thereof, of the climate.
Standard statistical tests assume a white noise response where the noise has constant power over all frequencies. If however, the noise has a 1/f type response, using a low pass filter will increase low-frequencies so, the noise will increase to appear as if it is signal. So, the statistical threshold must also be increased.
But the amount the threshold must be increased is dependant on the type of noise. And unless we accurately know the type of noise, we do not know how much we need to increase the threshold of the statistical test.
But climate is a real pickle. Because the only reliable data contains both signal and noise (the suggested change has been occurring to varying degrees throughout the instrumentational record). So, we have no period of “normal” variation from which to determine the frequency profile of normal background climate variation.
Statistical Phase test
However, we do know when the data signal was applied. Therefore, the phase of the 1/f noise of the signal would be deterministic, whereas that of the noise would be indeterministic (random).
This might provide the basis of a meaningful test which does not rely on knowing the frequency profile of the noise.
So, whilst the amplitude tells us very little because the signal and noise are both 1/f type and very hard to distinguish, they have very different phases. So, the appropriate test should focus predominantly on the phase and not the amplitude.
In common speak, the test of whether climate is “trending” (within a limited sample containing both signal and noise where the noise is known to be 1/f type) must be based not on the scale of any change, but on the timing of any change.