So, according to the 1964 paper you cited, there is no reason to stop at the second derivative.
Correct.
According to your reading of the paper we should also be able to extract the third, fourth and fifth derivative......from a single application?
Correct, but I may be misinterpreting what you mean by "a single application". I'm interpreting that to mean a single pass over the raw time series data. For more detail, including the definition of "a single pass" as that phrase is used by mathematicians and computer scientists, see my responses to
femr2 below.
According to your understanding, is there any reason the authors can't just keep going past the second derivative?
Savitzky and Golay use third derivatives in several of their examples, provide tables for computing first through fifth derivatives, and derive the equations for arbitrarily high-order derivatives in their Appendix I.
Having asked me these questions, I assume you won't mind if, in the spirit of public service, I respond (succinctly) to some of the other posts in this thread.
LOL. Exactly how are you going to get a velocity plot
and an acceleration plot by running the filter once (without recoding it so that multiple derivative orders are output) ?
Let N be the degree of the best-fit polynomial, and let W be the number of points in each window.
Code:
for each window:
for i = 1 to N:
compute derivative i
That's one pass over the time series data, with N W-sized convolutions at each data point.
If both N and W are small, those N W-sized convolutions will probably take less time than reading a single data point.
If N is small but W is large, it might be faster to find the best-fit polynomial via standard minimization of the sum of the square errors, followed by evaluation (at the center of the window) of every non-trivial derivative of the best-fit polynomial.
That's what my implementation does. It delivers exactly the same results as standard Savitzky-Golay (up to roundoff error).
With modern computers, and time series as short as the ones we've been discussing in this subforum, it's beyond silly for
femr2 to worry about the speed of the Savitzky-Golay algorithm.
You can run the filter on displacement data and get velocity data. You can run the filter on displacement data and get acceleration data. We want both.
Easily done. Run the Savitzky-Golay algorithm over the displacement data, generating both velocity and acceleration data in that one pass (as described above).
Implementations vary of course, but there should theoretically be no limit upon which derivative is returned,
Correct.
...derivative order cannot exceed polynomial order,
Incorrect.
Polynomial functions are C
∞ (
smooth, infinitely differentiable).
Of course taking the second derivative also forces you to use a poly order of at least 2.
Incorrect.
(The second derivatives of linear and constant-valued functions aren't very interesting, but they do exist.)
Derivative order cannot exceed polynomial order, for obvious reasons.
Incorrect.
That's not the point here. The point is... You can't generate third order derivative, from a second order polynomial.
Incorrect.
To answer your question, the result of two successive applications of SG (that being apply SG to displacement data and generate 1st derivative, then apply SG to velocity data and generate 1st derivative) is not the same as applying SG once and generating the second derivative.
Correct.
Whether that is desirable is subjective.
Correct.
Iirc single pass was far too noisy for my purposes,
We can't know whether that's correct until
femr2 explains his purposes.
That bare statement does not in any way demonstrate that you could not achieve that additional smoothing (or a better one from some theoretical point of view [1]) by applying one SG filter with different parameters, including a higher degree polynomial, to the same data. Which was my point.
Footnote [1]: I mean based on some objective theory, such as uniformity of the attenuation of the spectrum, etc., which is not to say that for certain applications the result could be better with two passes than with one pass, but you're far from proving it is.
Correct.
The Savitzky-Golay algorithm is noted for its flat frequency response, which derives from its rectangular convoluting function. In
pgimeno's next paragraph, quoted below, he sketches a proof that two iterations of the Savitzky-Golay algorithm can't be equivalent (on all inputs) to using a rectangular window of convolution. That means iterated SG doesn't have the same frequency response as the Savitzky-Golay algorithm. If there are objective reasons for wanting a flat response, as can happen, then iterated SG would be objectively worse than Savitzky-Golay differentiation.
Correct me if I'm wrong, but by performing two passes, you're NOT minimizing window width: in the first pass, each data point will be influenced by points at the window width distance (or more properly, at (window width - 1)/2 distance, if I understand what the window width parameter means here exactly); in the second pass, each data point will be influenced by points that were already influenced by the window width, so the effective window width is doubled, though with different weights, and thus with the possibility of introducing artifacts.
Correct.
(To simplify his explanation,
pgimeno assumed the window widths were the same for both iterations, but his point would hold even without that assumption.)
No, the second application of the filter is working with an entiely new set of data, each point of which is influenced by data from its own window of the original data.
pgimeno's point appears to have sailed over
femr2's head.
By all means use a second order derivative SG pass on displacement data. If you use a fairly wide window and fairly low poly order, you'll end up with fairly similar trend results.
Correct (for this particular time series).
And, yes, each point in the second pass will have been influenced by all source points in its window. Similarly for data smoothed twice by any other method, such as running average with symmetric differencing.
Again,
femr2 seems to
have missed the point.
Even if acceleration data is generated from displacement data with a second derivative SG pass, to produce velocity data and acceleration data still requires running the SG filter twice, albeit with both passes using the same source dataset. Once to generate the velocity data, and once to produce the acceleration data.
You only get one dataset output per pass, not two.
Incorrect.
As
Wikipedia explains:
In computing, a one-pass algorithm is one which reads its input exactly once, in order, without unbounded buffering.
With that definition, which is the definition used by mathematicians and computer scientists,
femr2's assertion is objectively incorrect.
Rather than waste more of my time engaging in fairly tedious ping-pong with opinion based assertions from others, and as I'm a thoroughly super chap, have
a peek.
Once again,
femr2 has failed to describe his parameters and methods in enough detail for others to replicate his results.