Merged Discussion of femr's video data analysis

tfk · Aug 13, 2010

I've been playing with femr's data from WTC7 for a while.

Specifically, I've been looking at the calculations of 1st & 2nd derivatives of the data.

The really interesting part (to me) has been the enormous impact on the calculated velocity & acceleration that results from the simple act of increasing the sampling rate.

People have said before that "computers are the fast way known to generate errors." This may be the purest implementation of that adage that I've ever seen.

I'd like to request that everyone (including me) keep this thread to the engineering. There are plenty of other places that we can express our dissatisfaction with each of our perceptions of the other side's politics, agendas, etc.

Here, let's stick to engineering, please.

Femr, one of the critical questions that we've argued about is your claimed "sub-pixel resolution", as produced by the SynthEyes program.

Here's a simple test.

Please pick some points on some other buildings (not WTC7) that have definable points that are comparable (in size, distance, spatial frequency & contrast) to the points on WTC7 that you've tracked, and provide tracings of those feature for several seconds. A couple of tracings of near-to-each-other features (one on WTC7 and one nearby in the frame, but on another building) would be very useful.

Please make sure that all scaling factors are set the same as in your WTC7 traces.

Presumably, that point on that building will be stationary. If there are no "sub-pixel" jumping up & down in those traces, it'll go a long way to showing that the bouncing signal are at least associated with WTC7. Perhaps schlieren effects from heat, if not actual motion of the building.

However, if the signal from a non-impacted building is bouncing just like the WTC7 signal is, then it's pretty clear that source of the trace noise is from some other source. Perhaps the software algorithms.

tom

DGM · Aug 13, 2010

tfk said:
However, if the signal from a non-impacted building is bouncing just like the WTC7 signal is, then it's pretty clear that source of the trace noise is from some other source. Perhaps the software algorithms.

tom

Or maybe very small movements of the camera itself? I've often wondered how far you could push the resolution of a video taken from a camera that was not attached to an absolutely stable platform. Also how much could "video stabilization" effect these calculations and how would you know?

beachnut · Aug 13, 2010

atmosphere errors
lens errors
frame rate error
and more errors
sub pixel resolution?
Is there a claim you can get more resolution from a set of pixels, than the pixels present?

femr2 · Aug 13, 2010

tfk said:
The really interesting part (to me) has been the enormous impact on the calculated velocity & acceleration that results from the simple act of increasing the sampling rate.

There's zero impact by increasing the sampling rate at all.

What does change is the way that data must be treated.

People have said before that "computers are the fast way known to generate errors." This may be the purest implementation of that adage that I've ever seen.

Computers just do what they are told to do. They don't really generate errors in this context, but rather reveal errors in procedure rapidly.

I'd like to request that everyone (including me) keep this thread to the engineering. There are plenty of other places that we can express our dissatisfaction with each of our perceptions of the other side's politics, agendas, etc.

No problem, though we're not really discussing engineering. I'll stay technical for sure.

Femr, one of the critical questions that we've argued about is your claimed "sub-pixel resolution", as produced by the SynthEyes program.

Indeed. Had hoped that had been clarified by now, especially given the pure tests provided, but we can go over it until you're satisfied.

Please pick some points on some other buildings (not WTC7) that have definable points that are comparable (in size, distance, spatial frequency & contrast) to the points on WTC7 that you've tracked, and provide tracings of those feature for several seconds.

No problem, though am not sure if you mean on the WTC7 footage or a separate piece of footage. Let me know what footage you want to test.

We've previously discussed the process of static point extraction, which is the tracing of features on separate buildings and extraction of that data from moving feature traces to eliminate frame factors such as frame deinterlace jitter and camera shake. Static point data is in the Cam 3 spreadsheet...
Download

A couple of tracings of near-to-each-other features (one on WTC7 and one nearby in the frame, but on another building) would be very useful.

Aha. On the WTC 7 footage. See previous comment.

Please make sure that all scaling factors are set the same as in your WTC7 traces.

Haven't included scaling in the cam#3 data yet. It's all in pixels. Will be sorting per-feature scaling metrics over the weekend hopefully.

Wouldn't be able to use the same scaling for features closer or further from the camera though. Would have to make some attempt at accounting for frame position and relative camera orientation.

Presumably, that point on that building will be stationary.

Fairly, yes.

If there are no "sub-pixel" jumping up & down in those traces, it'll go a long way to showing that the bouncing signal are at least associated with WTC7. Perhaps schlieren effects from heat, if not actual motion of the building.

Aha. Sounds like you are looking at the 59.94fps Dan Rather data.

That has been deinterlaced. It is an absolute MUST to take account of deinterlace jitter when using the data. We've been through that before, but can do so again. The alternate frame *bouncing* is not a problem with tracing technique, it's a direct effect resulting from deinterlacing. Not a problem, just has to be handled properly.

First simple step is to use a two sample running average on the data, though there are better methods.

However, if the signal from a non-impacted building is bouncing just like the WTC7 signal is, then it's pretty clear that source of the trace noise is from some other source. Perhaps the software algorithms.

Assuming you've been looking at the deinterlace jitter, then no, it's not a software algorithm problem, it's data knowledge.

femr2 · Aug 13, 2010

beachnut said:
atmosphere errors

Possible.

lens errors

Possible, though traces reveal inter-frame changes in position, minimising the effect of lens distortion. There are other features of SynthEyes which can take account of lens distortion, but from testing there's little point in actually using it for the kind of footage being looked at. Not using it helps other use alternate techniques to replicate the data, as the data accounting for lens distortion is rather complex to handle without actually using SynthEyes.

frame rate error

Possible, but rather unlikely, even for hardware back in the olden days of 2001.

and more errors

The most significant factor I'd suggest.

sub pixel resolution?

Not specifically in this context.

Sub-pixel feature position identification.

Is there a claim you can get more resolution from a set of pixels, than the pixels present?

See previous comment.

The posts here may be instructive...
http://www.internationalskeptics.com/forums/showpost.php?p=6114713&postcount=232

femr2 · Aug 13, 2010

DGM said:
Or maybe very small movements of the camera itself?

There are of course (very) minor variations in camera position, even for footage taken on a tripod. Footage such as Sauret has allowed the technique of static point extraction to be tested with very good results.

I've often wondered how far you could push the resolution of a video taken from a camera that was not attached to an absolutely stable platform. Also how much could "video stabilization" effect these calculations and how would you know?

Important to make the distinction between resolution and feature position. It's possible to extract additional resolution using algorithms such as those within SuperResolution video plugins, but that's not what we're doing. We're peforming sub-pixel accurate feature position tracing, which in my case utilises *8 upscaling and LancZos3 filtering, followed by area-based pattern matching to zero-in on specified features.

Worth reading pages 4-8 of the Physics toolkit thread, rather than going over it all again here.

beachnut · Aug 13, 2010

femr2 said:
...The posts here may be instructive...
http://www.internationalskeptics.com/forums/showpost.php?p=6114713&postcount=232

If only you had sources and references for your claims. Books; degrees, education, who you studied with. Are you making up your stuff? Then source it.

femr2 · Aug 14, 2010

A quick repost of the basic pure sub-pixel tracing example...

I generated the following animation of a simple circle moving in a circle itself...

I then downscaled it until the circle is a small blob...

If we look at a blow-up of the shrunken image, such that we can see the individual pixels, it looks like this...

Sub-pixel feature tracking essentially works by enlarging the area around the feature you want to track, and applying an interpolation filter to it. Lots of different filters can be used with varying results.

Applying a Lanczos3 filter to the previous GIF, to smooth the colour information between each pixel, results in the following...

I think you will see that there will be no problem for a computer to locate the centre of the circle quite accurately in that smoothed GIF, even though the circle in the original tiny image was simply a random looking collection of pixels. This process of upscaling and filtering generates arguably more accurate results than simply looking at inter-pixel intensities.

The resulting position determined is therefore clearly sub-pixel when translated back into the units of the original tiny source.

It is a side effect of aliasing that small movements of the object cause slight variation in inter-pixel intensity, saturation and colour.

Tracing the position of the small blob on the tiny version results in the following...

The raw data is here...
http://femr2.ucoz.com/SubPixelTracing.xls

The graph shows accurate (not perfect) sub-pixel location data for the position of the small blob.

I could go into more detail, but hope that clarifies.

Another test using exactly the same small blob rescale, but extended such that it takes 1000 frames to perform the circular movement. This results in the amount of movement being much much smaller between frames. This will give you an idea of how accurate the method can be...

(click to zoom)

Would you believe it eh.

Here's the first few samples...

Code:

0	0
-0.01	0
-0.024	-0.001
-0.039	-0.001
-0.057	-0.001
-0.08	-0.002
-0.106	-0.002
-0.136	-0.002
-0.167	-0.002
-0.194	-0.004
-0.214	-0.005
-0.234	-0.005
-0.251	-0.007
-0.269	-0.008
-0.289	-0.009
-0.31	-0.009
-0.337	-0.012
-0.365	-0.014
-0.402	-0.015
-0.431	-0.018
-0.455	-0.019
-0.48	-0.02

For this example, I'll quite confidently state that the 3rd decimal place is required, as accuracy under 0.01 pixels is clear. There are other sources of distortion, such as the little wobbles in the trace, which are caused by side-effects of the smoothing and upscaling when pixels cross certain boundaries. This reduces the *effective* accuracy. Can be quantified, by graphing the difference between *perfect* and the trace location, but not sure how much it matters.

Now, obviously this level of accuracy does not directly apply to the video traces, as they contain all manner of other noise and distortion sources.

tfk · Aug 14, 2010

femr,

femr2 said:
There's zero impact by increasing the sampling rate at all.

And I'd say "nonsense" to that.

Using difference equations,

v = (h2 - h1) / (t2 - t1).

the error in t2 - t1 is insignificant compared to the height error. (a good, real approximation, in this case).

If you have two points that are taken 1 second apart, and they measure equal heights, with an error of ±1 foot, then the error in the calculated velocity is

v = 0 ft/sec
V error = ± 1 ft/sec

If you maintain the very same height error, but your sampling rate is now 60 frames per second, then then

v = 0 ft/sec
V error = (±1 foot) / .0167 sec = ± 60 ft/sec.

And the propagating errors into acceleration are going to go as about the square of this.

This is inescapable, and it results EXCLUSIVELY from the fact that you've increased the DAQ rate & you are taking time derivatives of your data.

It is independent of the techniques that one chooses to employ to mitigate those errors.

The lesson that I've learned from this is that, when you are taking derivatives of discrete data, it is crucially important that you decrease measurement errors by a multiplier that is equal to, or greater than, the multiplier at which you increase your acquisition rate.

This is precisely the aspect of this problem that I found to be interesting.

femr2 said:
What does change is the way that data must be treated.

The treatment of the data is what this thread will be all about.

femr2 said:
Computers just do what they are told to do. They don't really generate errors in this context, but rather reveal errors in procedure rapidly.

There is no such thing as "computer error". It is all "human error".

femr2 said:
No problem, though we're not really discussing engineering. I'll stay technical for sure.

Error analysis & data filtering are going to be front & center. Those are engineering disciplines.

femr2 said:
Indeed. Had hoped [sub-pixel resolution] had been clarified by now, especially given the pure tests provided, but we can go over it until you're satisfied.

"Asserted", yes.
"Clarified", no.
"Proven", heck no.

And the information that i've been massaging suggests that you do not have sub-pixel accuracy. That is the whole purpose of this thread & discussion.

But hopefully, this thread will help us all find out what is correct. That's what we're all after, right?

femr2 said:
No problem, though am not sure if you mean on the WTC7 footage or a separate piece of footage. Let me know what footage you want to test.

The Dan Rather video is, by far, the best video for this purpose. The near perpendicular perspective eliminates all kinds of perspective effects. That's the one we should use.

femr2 said:
We've previously discussed the process of static point extraction, which is the tracing of features on separate buildings and extraction of that data from moving feature traces to eliminate frame factors such as frame deinterlace jitter and camera shake. Static point data is in the Cam 3 spreadsheet...
Download

Let's get one from the Dan Rather video.

The Camera 3 video can be a calibration standard that'll help answer the question if we can't get good data from the DR video. (although I don't know any reason that we should not be able to do so.

Also, let's make sure to get some static points RIGHT THRU the collapse, too. This will help rule out camera motion effects, atmospheric effects that are synchronous with the collapse.

femr2 said:
Haven't included scaling in the cam#3 data yet. It's all in pixels. Will be sorting per-feature scaling metrics over the weekend hopefully.

Let's stick with the DR Video.

I have seen another video, taken from the north, almost street level, that shows much more of the lower section of WTC7 from the north.
http://video.google.com/videoplay?docid=3664073116607499063#
Do you have a better version of this?

femr2 said:
Wouldn't be able to use the same scaling for features closer or further from the camera though. Would have to make some attempt at accounting for frame position and relative camera orientation.

You should (I think) be able to "bracket" the building. And just work in pixels.

With the DR tape, there are several buildings to the right of WTC7 that are prime candidates. The low, white building to the right with the stair stepped roof. One corner that is immediately adjacent to WTC7 maintains a slight gap throughout the collapse. that point would be ideal.

The narrow tall black building to the right of the towers would be excellent.

femr2 said:
Sounds like you are looking at the 59.94fps Dan Rather data.

Yes/

femr2 said:
That has been deinterlaced. It is an absolute MUST to take account of deinterlace jitter when using the data. We've been through that before, but can do so again. The alternate frame *bouncing* is not a problem with tracing technique, it's a direct effect resulting from deinterlacing. Not a problem, just has to be handled properly.

I am using the data in your files. I assume that you have eliminated the interlace jitter from these numbers.

There is no interlace jitter left in the numbers. If there were, there would be a statistical tendency for every other frame to be either above or below, left or right of every alternate one. This does not appear in the data.

femr2 said:
First simple step is to use a two sample running average on the data, though there are better methods.

It sounds here like you are saying that there jitter has not been removed.

Here is a graph of the raw data before the collapse. Every other data point has been shown as a red or blue dot.

If there were interlace jitter, there would be a pattern. There isn't one.

femr2 said:
Assuming you've been looking at the deinterlace jitter, then no, it's not a software algorithm problem, it's data knowledge.

Nope.

tom

tfk · Aug 14, 2010

I've seen this before. There are several reasons that I believe that it does not prove "sub-pixel resolution" in the wtc videos.

Let's get some of that static position data on the DR video, and see what it shows.

tom

femr2 · Aug 14, 2010

tfk said:
And I'd say "nonsense" to that.

We've been through this before, with additional clarification and agreement by W.D.Clinger.

A few reference posts...
#205
#248
#253 (W.D.Clinger)

The accuracy of each individual positional location does not change when increasing the sample rate, it is purely how the resultant data is subsequently treated that has any effect upon derived metrics.

Performing first and second order derivations from noisy data using near-adjacent samples will, of course, result in extreme amplification of that noise.

it results EXCLUSIVELY from the fact that you've increased the DAQ rate & you are taking time derivatives of your data.

Using the same methods as used upon samples a second apart, of course, but as I hope we've already established, that's not a smart (or at all appropriate) thing to do.

It is independent of the techniques that one chooses to employ to mitigate those errors.

Incorrect. Very incorrect Tom.

The treatment of the data is what this thread will be all about.

That's great, but these repeated fundamental errors you keep making must be sorted out first.

I'll upload a spreadsheet with my local Dan Rather data soon (I'll make it a bit more presentable first).

It includes...

* NW Corner Trace
* Static Point extraction
* Deinterlace Jitter treatment
* Velocity and Acceleration derivatives by both wide-band symmetric differencing, and least squares.
* A global single-cell scaling metric, so you can modify the scaling as you wish.
* Various graphs of the above.

There is no such thing as "computer error". It is all "human error".

Agreed. Always a problem between keyboard and seat.

Error analysis & data filtering are going to be front & center.

Fine.

And the information that i've been massaging suggests that you do not have sub-pixel accuracy. That is the whole purpose of this thread & discussion.

Am not at all sure what data you are using. The graph you uploaded tonight seems to have waaay too few points, seeing as the sample rate is 59.94fps.

Here's a copy of my position/time data for Dan Rather (zoomed)...

But hopefully, this thread will help us all find out what is correct. That's what we're all after, right?

You mean whether the data is validly sub-pixel accurate on position, yes ? If so, sure. Would be more useful for me if noise itself can be quantified, as I'm more than happy with the tracing methods already. Willing to justify the data again of course.

The Dan Rather video is, by far, the best video for this purpose. The near perpendicular perspective eliminates all kinds of perspective effects. That's the one we should use.

I don't agree, but have no problem using it. The quality of the Cam#3 footage is better, but hey-ho. We'll focus on Dan Rather, though I suggest you ensure we're both using the same data once I upload my local spreadsheet copy of the Dan Rather data.

Let's get one (static point) from the Dan Rather video.

Already done. May already be extracted from the data you are using. Not sure.

The Camera 3 video can be a calibration standard that'll help answer the question if we can't get good data from the DR video. (although I don't know any reason that we should not be able to do so.

Not sure I understand.

Also, let's make sure to get some static points RIGHT THRU the collapse, too.

Static point traces are always full clip-length. There's nothing to stop them being so.

This will help rule out camera motion effects, atmospheric effects that are synchronous with the collapse.

Aiii.

I have seen another video, taken from the north, almost street level, that shows much more of the lower section of WTC7 from the north.
http://video.google.com/videoplay?docid=3664073116607499063#
Do you have a better version of this?

What time position ? No use if the camera is not static though. Have to do all sorts of jiggery-pokery to stabilise the footage it the camera is not essentially static.

You should (I think) be able to "bracket" the building. And just work in pixels.

Fine. Is what I;ve done for the Dan Rather static point at this juncture.

With the DR tape, there are several buildings to the right of WTC7 that are prime candidates. The low, white building to the right with the stair stepped roof. One corner that is immediately adjacent to WTC7 maintains a slight gap throughout the collapse. that point would be ideal.

Will find out what point I used and post an image.

I am using the data in your files. I assume that you have eliminated the interlace jitter from these numbers.

From the look of your graph, yes, though the graph looks very odd and have no idea what data you are using. Too few points from what I can see (tho of course I have to go to your profile to see your images. Don't suppose you fancy using a normal image upload service rather than JREF album ?)

tfk · Aug 14, 2010

femr,

Look, let's try to avoid walking down our usual path, please.

First thing, please use your own arguments. WD will have every opportunity to express his own opinions here. You're misstating his.

Second, please respond to what I say, not a twisted up version of what I say.

femr2 said:
We've been through this before, with additional clarification and agreement by W.D.Clinger.

A few reference posts...
#205
#248
#253 (W.D.Clinger)

The accuracy of each individual positional location does not change when increasing the sample rate, it is purely how the resultant data is subsequently treated that has any effect upon derived metrics.

I didn't say that the position error changed when you speed the sampling rate. I said the exact opposite. I specifically said "If you maintain the very same height error..."

I specifically said that the error ripples into the velocity & acceleration calculations.

Please respond to what I actually say.

femr2 said:
Performing first and second order derivations from noisy data using near-adjacent samples will, of course, result in extreme amplification of that noise.

It doesn't "amplify the noise" in the slightest. "Amplifying the noise" would mean "more noise in the position vs time graph".

It amplifies the error in the calculated instantaneous velocity & acceleration. Exactly what I said.

femr2 said:
Using the same methods as used upon samples a second apart, of course, but as I hope we've already established, that's not a smart (or at all appropriate) thing to do.

I gave you the illustration of the source of the error. I didn't say that was the right way to do something. We'll get to that the best way to massage the data eventually.

femr2 said:
tfk said:

[The source of the error in the time derivatives] is independent of the techniques that one chooses to employ to mitigate those errors.

Click to expand...

Incorrect. Very incorrect Tom.

tfk said:

The treatment of the data is what this thread will be all about.

Click to expand...

That's great, but these repeated fundamental errors you keep making must be sorted out first.

Please...

In spite of the chubby it gives you to say it, what I said above is not "incorrect, very incorrect, or laced with fundamental errors".

If you'd kindly respond to what I really say, not your misinterpretation of what I say, you'd find that, magically, "I'll suddenly be making far fewer errors"...

femr2 said:
Am not at all sure what data you are using. The graph you uploaded tonight seems to have waaay too few points, seeing as the sample rate is 59.94fps.

Here's a copy of my position/time data for Dan Rather (zoomed)...
http://femr2.ucoz.com/_ph/7/981070076.png

If you look at the axes, you'll find that the data I posted is identical to the data that you just posted. Just zoomed to a slightly different time domain.

femr2 said:
You mean whether the data is validly sub-pixel accurate on position, yes ? If so, sure. Would be more useful for me if noise itself can be quantified, as I'm more than happy with the tracing methods already. Willing to justify the data again of course.

Quantifying the noise level is exactly where we will be at the end.

femr2 said:
I don't agree, but have no problem using it. The quality of the Cam#3 footage is better, but hey-ho. We'll focus on Dan Rather, though I suggest you ensure we're both using the same data once I upload my local spreadsheet copy of the Dan Rather data.

Fine.

Please include a brief text file that outlines what each data set is, and describes any processing that was done to the data.

Please make sure that there is at least one clearly identified data set of the raw data that comes out of the SnythEyes, without any massaging done to it.

femr2 said:
tfk said:

Let's get one (static point) from the Dan Rather video.

Click to expand...

Already done. May already be extracted from the data you are using. Not sure.

I have been using the data that you posted as being "from the Dan Rather video". It produces the same curves that you post (as the one above).

It starts & ends with:
Time(s) Drop (ft)
0.01668335 0.207940299
0.0333667 0.120656716
0.05005005 0.272119403
...
9.59292626 -332.2381095
9.60960961 -333.7407562
9.62629296 -335.8449751

It is a difficulty when I don't know all the massaging that you may have done (i.e., deinterlacing, motion removal, etc). So that is why I'd like to get a clear description of the various data sets.

femr2 said:
tfk said:

The Camera 3 video can be a calibration standard that'll help answer the question if we can't get good data from the DR video. (although I don't know any reason that we should not be able to do so.

Click to expand...

Not sure I understand.

We're getting a baseline reference signal to try to measure noise. We need to use a reference point that is not on the building, because points on the building might be moving.

The baseline tells us about noise in the processing, camera motions, atmospheric effects, etc. All motions that might be from a source other than WTC7 motion.

femr2 said:
tfk said:

I have seen another video, taken from the north, almost street level, that shows much more of the lower section of WTC7 from the north.
http://video.google.com/videoplay?do...3116607499063#
Do you have a better version of this?

Click to expand...

What time position ?

From about 1:04 of that video. But it looks like the resolution is pretty lousy. The've electronically zoomed so far that the building resolution is too coarse.

femr2 said:
tfk said:

I am using the data in your files. I assume that you have eliminated the interlace jitter from these numbers.

Click to expand...

From the look of your graph, yes, though the graph looks very odd and have no idea what data you are using. Too few points from what I can see (tho of course I have to go to your profile to see your images. Don't suppose you fancy using a normal image upload service rather than JREF album ?)

Is anyone else having problems seeing the images I've posted?

Is this a mac/windows thing? File name problem?

Or is this in femr's set-up?
femr, is it just my images you have trouble with, or everyone's? Or some people & not others?

tom

femr2 · Aug 14, 2010

tfk said:
Look, let's try to avoid walking down our usual path, please.

No problem. Was just highlighting that when you said *the enormous impact on the calculated velocity & acceleration that results from the simple act of increasing the sampling rate*, that it's how the data is treated that is important.

I assume we agree that generation of derived metrics must not use near-adjacent samples.

what I said above is not "incorrect, very incorrect, or laced with fundamental errors".

To say *It is independent of the techniques that one chooses to employ to mitigate those errors.* is in my opinion wrong. The techniques used are critical. Using, say, 19 sample wide adjacent differencing does not result in such extreme noise amplification.

It's fine to disagree.

Quantifying the noise level is exactly where we will be at the end.

More what I was intending was to quantify the *source* of noise. I'm happy that the tracing techniques are sub-pixel accurate, but there are of course other sources of position variation within the available WTC 7 footage.

One point I think it's worth making at this juncture is that we're basically looking at determining an end-result *effective* positional accuracy, yes ? If the underlying positional accuracy was only integer pixel accurate, then our *effective* accuracy would obviously be +/- a number of pixels, rather than, say, +/- 0.5 pixels. Agreed ?

Please include a brief text file that outlines what each data set is, and describes any processing that was done to the data.

No probs.

Please make sure that there is at least one clearly identified data set of the raw data that comes out of the SnythEyes, without any massaging done to it.

Aiii.

I have been using the data that you posted as being "from the Dan Rather video". It produces the same curves that you post (as the one above).

It starts & ends with:
Time(s) Drop (ft)
0.01668335 0.207940299
0.0333667 0.120656716
0.05005005 0.272119403
...
9.59292626 -332.2381095
9.60960961 -333.7407562
9.62629296 -335.8449751

OK.

It is a difficulty when I don't know all the massaging that you may have done (i.e., deinterlacing, motion removal, etc). So that is why I'd like to get a clear description of the various data sets.

Format of the spreadsheet is similar to the Cam3 data. Raw pixel data comes first. Individual processes are applied over subsequent columns.

We're getting a baseline reference signal to try to measure noise. We need to use a reference point that is not on the building, because points on the building might be moving.

The baseline tells us about noise in the processing, camera motions, atmospheric effects, etc. All motions that might be from a source other than WTC7 motion.

Yes, static point data.

femr, is it just my images you have trouble with, or everyone's? Or some people & not others?

Just yours from what I've seen. I don't use cookies, so my JREF session is through URL session ID. I think that may be the problem, but I just don't *do* cookies

W.D.Clinger · Aug 14, 2010

I see a lot of violent agreement in this thread.

tfk said:
The really interesting part (to me) has been the enormous impact on the calculated velocity & acceleration that results from the simple act of increasing the sampling rate.

The mathematical reasons for this are covered in the opening pages, specificially sections 1.3 and 1.4, of

R W Hamming^WP. Numerical Methods for Scientists and Engineers. McGraw-Hill, 1962.

Hamming won the Turing Award^WP in 1968, and the book cited above is a standard reference.

femr2 said:
There's zero impact by increasing the sampling rate at all.

What does change is the way that data must be treated.

I agree with the second sentence, but I'd quibble with the first: Increasing the sampling rate creates a trap for the unwary. A naive analyst might calculate derivatives via differencing at the full resolution of the sampled data. As tfk observed:

tfk said:
And the propagating errors into acceleration are going to go as about the square of this.

But the following is untrue:

tfk said:
This is inescapable, and it results EXCLUSIVELY from the fact that you've increased the DAQ rate & you are taking time derivatives of your data.

It is independent of the techniques that one chooses to employ to mitigate those errors.

The lesson that I've learned from this is that, when you are taking derivatives of discrete data, it is crucially important that you decrease measurement errors by a multiplier that is equal to, or greater than, the multiplier at which you increase your acquisition rate.

This is precisely the aspect of this problem that I found to be interesting.

No, the increased error can be avoided by downsampling^WP and related techniques. Downsampling is counter-intuitive, of course. Evaluating the tradeoff between error and frequency response takes some sophistication.

femr2 said:
We've been through this before, with additional clarification and agreement by W.D.Clinger.

A few reference posts...
#205
#248
#253 (W.D.Clinger)

The accuracy of each individual positional location does not change when increasing the sample rate, it is purely how the resultant data is subsequently treated that has any effect upon derived metrics.

Performing first and second order derivations from noisy data using near-adjacent samples will, of course, result in extreme amplification of that noise.

Mostly correct, but the accuracy of sampled data does tend to degrade somewhat when the sampling rate is pushed too high. I think that's a legitimate concern with femr2's data: Since we're going to have to ignore at least some of the extra resolution to keep quantization error (amplified by differencing) within reasonable bounds, does the higher sampling rate really help?

tfk said:
First thing, please use your own arguments. WD will have every opportunity to express his own opinions here. You're misstating his.

Not too badly, but I took advantage of this opportunity to clarify my views.

femr2 said:
No problem. Was just highlighting that when you said *the enormous impact on the calculated velocity & acceleration that results from the simple act of increasing the sampling rate*, that it's how the data is treated that is important.

I assume we agree that generation of derived metrics must not use near-adjacent samples.

I certainly agree with that.

femr2 · Aug 14, 2010

W.D.Clinger said:
I see a lot of violent agreement in this thread.

Hamming won the Turing Award^WP in 1968, and the book cited above is a standard reference.

Will keep it to hand.

Increasing the sampling rate creates a trap for the unwary.

Fair enough.

Since we're going to have to ignore at least some of the extra resolution to keep quantization error (amplified by differencing) within reasonable bounds, does the higher sampling rate really help?

I'd suggest one thing that helps is that, even with downsamplig of some kind, an averaged value could be used, or a median, or a...we know more about what happens between any points we may *ignore*, and so can determine a *better* value to use at any particular point.

The averaging techniques I use are pretty basic, but am all ears for more appropriate methods.

We're not looking for *jolts* in this context, so there's quite a lot of legroom (headroom) in terms of smoothing the data without negatively affecting the results too much.

Not quite sure how to address Toms intention to verify sub-pixel accuracy. The actual tracing points are certainly sub-pixel (It's easy to simply eyeball attachment to the desired feature in SE, and it's very solid). It's the sources of noise in the video data itself that need quantifying, and it's those that result in variation of the traced position. Dan Rather footage is pretty poor quality, even though it has a good camera location. Hmm...

femr2 · Aug 14, 2010

A question...

To generate the 59.94fps data, it is necessary to combine deinterlaced video footage into *bob doubled* format.

There are inherent differences between alternate frames as a result of this, and I use simple averaging to account for it.

There are two options available...

We can:

a) Use the 59.94fps data, and be aware of the inherent deinterlace jitter

-or-

b) Use the TWO sets of deinterlaced trace data at 29.97fps

The latter will allow the generation of two separate drop/velocity/accel curves, with the caveat that they are 1/59.94 seconds apart, but the removal of the jitter.

Can do both if y'like...

tfk · Aug 14, 2010

WD,

You & I know calculus, derivatives & error analysis. We know what the source of the problem is. But we're talking by each other.

W.D.Clinger said:
I see a lot of violent agreement in this thread.

A bunch of "Conan the Librarians" around here...

Hardly violent.

W.D.Clinger said:
The mathematical reasons for this are covered in the opening pages, specificially sections 1.3 and 1.4, of

R W Hamming^WP. Numerical Methods for Scientists and Engineers. McGraw-Hill, 1962.

Thanks for the reference.

This problem brought up the issue of doing error analyses with differential equations. I've not run into that before, and started looking around. I didn't find much.

The simplest way would be to use standard "static equation" techniques with the discrete difference equations. Not exact, but close enough.

And probably more appropriate for discrete data.

W.D.Clinger said:
But the following is untrue:

tfk said:

This is inescapable, and it results EXCLUSIVELY from the fact that you've increased the DAQ rate & you are taking time derivatives of your data.

It is independent of the techniques that one chooses to employ to mitigate those errors.

The lesson that I've learned from this is that, when you are taking derivatives of discrete data, it is crucially important that you decrease measurement errors by a multiplier that is equal to, or greater than, the multiplier at which you increase your acquisition rate.

This is precisely the aspect of this problem that I found to be interesting.

Click to expand...

No, the increased error can be avoided by downsampling^WP and related techniques. Downsampling is counter-intuitive, of course. Evaluating the tradeoff between error and frequency response takes some sophistication.

I said "the error results from simply decreasing the time interval, because the instantaneous velocity is calculated by the slope".

I meant that the SOURCE of the error was simply decreasing the time interval while the positional error stayed constant. And I stand by that.

I didn't say that there were no ways to compensate for it. Or that you had to live with reporting adjacent data points.

Filtering, smoothing & (I came to appreciate) data decimation are all techniques.

The SOLUTION to the problem are all the techniques that you've mentioned.
___

The interesting part of the issue is that one is rarely faced with the "problem" of "too much data".

Mostly this happens because people who set up the data acquisition tune the DAQ rate to the frequency of the effect that they're looking for. Make sure that you've got sufficient rate to meet Nyquist requirements & call it a day.

Having a problem like this arise merely from an embarrassment of data riches is unusual and somewhat fascinating. I'm trying to think of other places where one might run into a similar problem.

Can't think of one at the moment.

tom

femr2 · Aug 14, 2010

To save some time, I've uploaded a simple copy of the latest Dan Rather Data.

I imagine we'll be using totally different processing techniques on the data, so best not to pollute the given data with too much of mine...

Download

Columns:

C/D - Raw pixel data for NW corner

F/G - Raw pixel data for static building

I/J - Pixel data for normalised NW corner (Static point data subtracted)

L/M - Pixel data for de-jittered NW corner (simple two point rolling average)

S/T - De-jittered NW corner data in feet (Uses variable scaling metric source values)

The scaling metrics can be modified, but are based on the building width and the visible distance between NW corner and the building it becomes obscured by.

The vertical distance is based upon NIST values, and I'm inclined to spend some more time on it. It's simply a matter of changing the scalar so I've uploaded the data now. We can agree on refined scaling metrics shortly.

femr2 · Aug 14, 2010

A question that has been raised is whether the trace data is accurate sub-pixel.

Here is a graph of the horizontal and vertical position traces for the NW corner from the Dan Rather data supplied in the previous post...

The vertical axis is in pixels and spans +/- 1 pixel, and the horizontal axis spans 13 seconds.

Is there any remaining doubt that the positional data is accurate sub-pixel ?

femr2 · Aug 15, 2010

For reference, here are the trace locations for the data provided...

The image is in deinterlaced form, showing both fields.

The size of the box indicates the feature search area.

The large search area used for the static feature trace allows for slightly increased accuracy.

femr2 · Aug 15, 2010

The original source video can be found here...

Download

Bear in mind it is in interlaced form, and if you choose to use it, the first thing to do is unfold each interlace field.

Any processing of the video should use a lossless codec such as RAW or HuffYUV.

femr2 · Aug 15, 2010

A very *loose* curve fit.

The initial polynomial is simply done with excel, then a quick 2nd order plot in Maple...

Probably useless, but a *start point*. I *very* rarely use Maple, so will dig in and see whether I can use the raw data rather than the poxy excel poly.

tfk · Aug 15, 2010

femr2 said:
A very *loose* curve fit.

The initial polynomial is simply done with excel, then a quick 2nd order plot in Maple...

http://femr2.ucoz.com/_ph/7/761812080.png

Probably useless, but a *start point*. I *very* rarely use Maple, so will dig in and see whether I can use the raw data rather than the poxy excel poly.

What value is that equation supposed to represent?

Vertical position vs. time?

Over what domain?

tom

femr2 · Aug 15, 2010

tfk said:
What value is that equation supposed to represent?

Vertical position vs. time?

Over what domain?

tom

Initial equation is polynomial curve fit of position/time data within excel.

Graph is a second order curve - Acceleration.

x - time (s)
y - ft/s^2

Time is 10 to 17 seconds in the supplied data.

tfk · Aug 15, 2010

femr2 said:
Initial equation is polynomial curve fit of position/time data within excel.

Graph is a second order curve - Acceleration.

x - time (s)
y - ft/s^2

Time is 10 to 17 seconds in the supplied data.

That's what I thought.

That doesn't demonstrate a particularly astute understanding of curve fitting & function range (+30' to -30' to +30' ?) or domain ...

Overlay your polynomial onto your raw data & you'll see that they look nothing alike. So your empirical equation is pretty useless to giving you velocity & then acceleration values. Which, of course, would be one of the principle reasons for generating that curve in the first place.

BTW, it's a 5th order poly fit. Not a 2nd.

tom

tfk · Aug 16, 2010

femr2 said:
For reference, here are the trace locations for the data provided...

http://femr2.ucoz.com/_ph/7/2/759212128.jpg

The image is in deinterlaced form, showing both fields.

The size of the box indicates the feature search area.

The large search area used for the static feature trace allows for slightly increased accuracy.

Thanks.

What is the point on the building within large green rectangle that is your static point? (The rectangle's too big to tell.)

Could you do the points that are inside the red boxes as well.

Uploaded with ImageShack.us

Also, what the heck is the solid black line that rises up the right side & roof line of WTC7?

Is that in the original CBS video? Why does it look like it is completely fake. Like it's been added after the fact?

tom

femr2 · Aug 16, 2010

tfk said:
That's what I thought.

That doesn't demonstrate a particularly astute understanding of curve fitting & function range (+30' to -30' to +30' ?) or domain ...

Overlay your polynomial onto your raw data & you'll see that they look nothing alike. So your empirical equation is pretty useless to giving you velocity & then acceleration values. Which, of course, would be one of the principle reasons for generating that curve in the first place.

tom

I said it's a very loose curve (though it's not *that* far off - have you plotted the equation itself ?), and the graph is probably useless (as it's very low order due to max-ing out on the poxy excel poly fit. Am looking at a order 56 plot at the moment, but there's no rush). lol. Bit of a lack of curve fitting tools in this building, and we've done it all before anyway. Call it a prompt to produce...whatever...

As far as my understanding goes, the intention of this thread was your intention to prove claims of sub pixel position accuracy wrong. Post containing the +/- 1 pixel graph a couple of posts ago could do with a response.

btw - Yes, the initial equation is a 5th order poly, however, the graph is a second derivation of it (the diff commands in Maple). If you thought that was a graph of the initial 5th order equation, there's no wonder you're saying it's not a good fit to the position/time data. Note it comes out around the 32ft/s^2 mark at peak

I included the maple commands to make sure that was clear. I might have to include some additional verbage to make sure my posts are simply clear, but please take a little extra time looking at them, rather than assume I'm doing something *stoooopid* eh.

femr2 · Aug 16, 2010

tfk said:
Could you do the points that are inside the red boxes as well.

Can do.

Uploaded with ImageShack.us

Ta. Can see it no problem.

Also, what the heck is the solid black line that rises up the right side & roof line of WTC7?

It's simply a high contrast region. It's not an overlay. Have a look at the linked mpeg video zoomed. As I said, the Dan Rather footage is great from a perspective, er, perspective, but the quality is not great. Video artefacts galore.

Is that in the original CBS video?

Yep.

Why does it look like completely fake. Like it's been added after the fact?

Video CCD's do lot's of odd things like that. Am sure the CBS overlay titling doesn't help things either.

femr2 · Aug 16, 2010

tfk said:
Could you do the points that are inside the red boxes as well.

A draft vertical trace...

There is not enough contrast on the background building, and the trace drifts quite badly. So I've omitted it.

tfk · Aug 16, 2010

Re: My misreading of your chart explanation: My apologies. It demonstrated my failure to read to the end of your post.

Acceleration makes more sense, but the results are still way off. And this, ultimately, is going to be the cornerstone of my point in this thread: Obtaining a reasonable, plausible result in velocity & acceleration is one of the few objective checks that one can employ to get a sense of accuracy & error in the position vs. time graph.

The extended +30g values at the beginning & end of your acceleration graph don't make any sense whatsoever.

And now, I'm going to jump to the end and may lose you. It's better to go step by step, but I'll post a peek at where we're going.

There are folks who are "leveraging" the calculation of acceleration from position data to sell "controlled demolition".

The motivations are irrelevant to this conversation. What is relevant is not "what results can I get from the data and the analysis?"

What is relevant is "What results are really justifiable & defensible from the data & the analysis?"

In order to do this right, it is imperative to have a sense of the "sensitivity" of the analysis. That is, how sensitive the answer is to different variables.

When one is taking few data points, then the issue of "smoothing the data" (i.e., frequency filtering) is irrelevant. As soon as one starts gathering data higher speeds, then there is a metric that is going to be of the form ∂y * ƒ (where ∂y = position error & ƒ = frequency) which sets a limit on the known accuracy of your results.
___

[Aside]
It's fascinating that this is virtually identical to a statement of the Heisenberg Uncertainty Principle, which of course applies only to atomic scale objects. But the math may well apply in both areas...

WD, I'm certain that there is some fundamental theorem in Data Acquisition that addresses this issue directly. I suspect that your reference in Numerical Methods probably addresses it. Can you put a name to it?
[/Aside]
___

The end result of all of this is, right now, "art". As soon as someone can point us to the theory, it'll become engineering, where we can quantify the relationship between DAQ ("Data Acquisition") rate and measurement error.

The engineering says that there is error in all measurements and therefore in all calculations. The art says that there are acceptable & unacceptable levels of error.

The acceleration graph that you produced has a clearly unacceptable level of error. It shows the building having an average of about +10G over the first 1.2 seconds or so. Which, from a zero initial velocity, means that the building would have jumped upwards about 180 feet.

I trust that you agree that this is an unacceptable level of error...

I've found out, by playing with these equations, that it is pointless to bother trying to fit any polynomial to the position data if you are going to do the differentiating for velocity & position. If you set the degree of the polynomial high enough to capture the various characteristics of the position over the whole time interval of the fall, you will inevitably be introducing pure artifacts that will blow up when you calculate the velocity and (especially) acceleration.

Instead you have to use "interpolating functions". Both Mathematica (the program that I use) & Maple have them. These don't try to fit the same polynomial over the whole range, but string a bunch of them together, making sure to match values and derivatives at all "connections" between the curve segments.

You can choose polynomial interpolation (for which you can specify the polynomial order) or spline interpolation.

The spline interpolation is designed to give you continuous 2nd derivatives (no kinks). While splines seem like a natural choice, Mathematica will not integrate spline based interpolation functions. This is a good reality check on your results. Integrate twice back up to get total drop, and make sure that it agrees with the original data.

Getting the right drop is a necessary - not sufficient - check on your acceleration results. You can have higher accelerations for shorter periods of time or lower accelerations for longer periods of time, and achieve the same displacement

Here are the curves that I got with your old Camera 2 (NIST ID, the Dan Rather) video. You said that you couldn't see them before.

This curve is your data (smoothed & decimated) vs. NIST's data after I've adjusted time offset and applied a 0.93 scale factor to your descent. It matches quite well. Small differences at the start of the collapse. Not surprising because you & NIST used different points on the roof line.

I got this scale factor by comparing NIST's drop to your drop over a central (i.e., "constant acceleration") section of the curve. From about 25' drop to about 170' drop). By staying in the middle of the range, I was able to avoid the "where is t0?" question & various dynamics at the end of the drop too.

This is puzzling, and deserves an examination. There is no reason that I can see that your scale factor should be off. Could you re-check it against known points on the building, please.

FEMR's drop vs time
Smoothed & decimated

This curve shows just your data, 21 sample smoothed & decimated (every 7th point), along with the interpolation curve that fits the data. You can see that the fit is excellent. Much better than any single polynomial will give you.

Velocity (smoothed) vs. time:
From Smoothed drop data

Acceleration (smoothed) vs. time
From smoothed velocity data

At the end of the day, we're gonna find out that the acceleration results are very sensitive to the filtering & decimation values used. This makes one nervous that your results might not be objective, but merely a reflection of your prejudices regarding "acceptable values".

Which is precisely why I started this thread. To get some feedback from some of the EEs with background in DAQ and/or signal processing. To see if there is a more objective way to determine proper filtering of the data.

Nonetheless, I believe this acceleration data to be "reasonable", which gives a "sanity check" on the filtering.

Final point: You'll see that the curve I've posted only covers 5 seconds, whereas the one you posted covered 7.

BTW: small point. You'll find that (being 2nd derivative of 5th order eqn) that the curve your plotted is a 3rd order poly. (Not particularly important.)

That's enough for now.

More later.

tom

tfk · Aug 16, 2010

femr,

Could you post the numerical data on the static points for those other 3 buildings, please.

It's immediately clear that there is a very high correlation between the jitter of all three of those static points. This goes a long way to suggest that the jitter is really in the video, and not in the software algorithms.

Btw, was this data taken from interlaced video? Did you do any "jitter correction" on it?

tom

Do you have a url reference back to an original of the CBS video? (BTW, are you "xenomorph"? I'm just asking if the origins of all those videos is you or someone else.)

T.A.M. · Aug 16, 2010

First thread in a while with a plethora of educational material in it. Bump to keep it a top...not that it needs it at this point.

TAM

femr2 · Aug 16, 2010

tfk said:
Acceleration makes more sense, but the results are still way off.

As the initial equation is a 5th order poly, the accel derivation is only order 3, which is why it's the shape it is.

And this, ultimately, is going to be the cornerstone of my point in this thread:

OK.

Obtaining a reasonable, plausible result in velocity & acceleration is one of the few objective checks that one can employ to get a sense of accuracy & error in the position vs. time graph.

Not sure I agree with that. I think I've shown that the positional accuracy is pretty hot, but certainly finding techniques to reduce the noise level for subsequent derivations will be productive.

The extended +30g values at the beginning & end of your acceleration graph don't make any sense whatsoever.

They wouldn't. The vertical axis is in ft/s^2, not G. As I indicated, the *peak* is about 32ft/s^2, which is why I included the graph. Using higher order initial polys still results in over-g accel, but I'm looking at various ways of eliminating noise before I post further accel graphs.

What is relevant is "What results are really justifiable & defensible from the data & the analysis?"

My viewpoint is even simpler...what does the data actually show. Thus far the tracing process for WTC7 has shown that movement spans a wide timeframe, building corner release points can be quantified, that sort of thing...

The acceleration graph that you produced has a clearly unacceptable level of error. It shows the building having an average of about +10G over the first 1.2 seconds or so. Which, from a zero initial velocity, means that the building would have jumped upwards about 180 feet.

See prior comments. ft/s^2, not G.

I've found out, by playing with these equations, that it is pointless to bother trying to fit any polynomial to the position data if you are going to do the differentiating for velocity & position. If you set the degree of the polynomial high enough to capture the various characteristics of the position over the whole time interval of the fall, you will inevitably be introducing pure artifacts that will blow up when you calculate the velocity and (especially) acceleration.

It's a problem, yes. Am looking at the effect of sequential passes of downsampling and interpolation at the moment to smooth it out.

While splines seem like a natural choice, Mathematica will not integrate spline based interpolation functions.

Route forward there would be to generate a new data-set from the interpolation function. Match it to the original sample rate, then use the more basic symmetric differencing on that data perhaps ?

This is a good reality check on your results. Integrate twice back up to get total drop, and make sure that it agrees with the original data.

Here are the curves that I got with your old Camera 2 (NIST ID, the Dan Rather) video. You said that you couldn't see them before.

I see piccys.

There is no reason that I can see that your scale factor should be off. Could you re-check it against known points on the building, please.

Sure, though there is very scant information available on building size metrics.

At the end of the day, we're gonna find out that the acceleration results are very sensitive to the filtering & decimation values used.

Absolutely.

This makes one nervous that your results might not be objective, but merely a reflection of your prejudices regarding "acceptable values".

Oi. Enough of that.

Which is precisely why I started this thread. To get some feedback from some of the EEs with background in DAQ and/or signal processing. To see if there is a more objective way to determine proper filtering of the data.

OK.

BTW: small point. You'll find that (being 2nd derivative of 5th order eqn) that the curve your plotted is a 3rd order poly. (Not particularly important.)

Aiii.

femr2 · Aug 16, 2010

tfk said:
femr,

Could you post the numerical data on the static points for those other 3 buildings, please.

Sure. Will upload after I've been t'pub. Thirsty

It's immediately clear that there is a very high correlation between the jitter of all three of those static points. This goes a long way to suggest that the jitter is really in the video, and not in the software algorithms.

For sure. The pure noiseless video testing (the little rotating blob) allows quantifying the actual tracing algorithm variances. There are many sources of noise in the video itself.

Btw, was this data taken from interlaced video?

No. Unfolded video in the form of the image posted earlier, so there are two traces for each point which are then combined to form the 59.94 sample per second data-set.

Did you do any "jitter correction" on it?

Yes. (n1+n2)/2.

Do you have a url reference back to an original of the CBS video?

No, that's the best I have access to.

are you "xenomorph"?

No.

femr2 · Aug 16, 2010

femr2 said:
Will upload after I've been t'pub. Thirsty

Maybe not such a great idea

Here y'are, before I go...

Extra Static Point Data

tfk · Aug 16, 2010

T.A.M. said:
First thread in a while with a plethora of educational material in it. Bump to keep it a top...not that it needs it at this point.

TAM

In coarse, Hollywood, over-the-top Hispanic accent:
"Do ju know wha' a "plethora" ees?"

Name that movie ...
(& show your age ...)

tom

T.A.M. · Aug 16, 2010

I dunno, but i would guess cheech and chong....something or other....lol

As for my age, i'll be 40 in 6 weeks.

TAM

tfk · Aug 16, 2010

T.A.M. said:
I dunno, but i would guess cheech and chong....something or other....lol

As for my age, i'll be 40 in 6 weeks.

TAM

Youngster...!

http://www.daveandthomas.net/2010/03/12/yes-el-guapo-a-plethora-of-pinatas/

tom

DGM · Aug 16, 2010

tfk said:
Youngster...!

http://www.daveandthomas.net/2010/03/12/yes-el-guapo-a-plethora-of-pinatas/

tom

That's not an old movie!

It came out ten years after I graduated HS!

W.D.Clinger · Aug 16, 2010

tfk said:
What is relevant is "What results are really justifiable & defensible from the data & the analysis?"

In order to do this right, it is imperative to have a sense of the "sensitivity" of the analysis. That is, how sensitive the answer is to different variables.

When one is taking few data points, then the issue of "smoothing the data" (i.e., frequency filtering) is irrelevant. As soon as one starts gathering data higher speeds, then there is a metric that is going to be of the form ∂y * ƒ (where ∂y = position error & ƒ = frequency) which sets a limit on the known accuracy of your results.
___

[Aside]
It's fascinating that this is virtually identical to a statement of the Heisenberg Uncertainty Principle, which of course applies only to atomic scale objects. But the math may well apply in both areas...

WD, I'm certain that there is some fundamental theorem in Data Acquisition that addresses this issue directly. I suspect that your reference in Numerical Methods probably addresses it. Can you put a name to it?
[/Aside]
___

The name I would give to it is "forward error analysis^WP of quantization error^WP". Quantization error behaves somewhat like roundoff error^WP, but tends to be much larger in simple calculations such as ours.

Unfortunately, most numerical analysis^WP textbooks concentrate on algorithmic error (aka formula error) and algorithmic stability, discuss roundoff error only in passing, and don't mention quantization error at all. The reason for this, I suspect, is that numerical analysts have historically been less concerned with solving problems given by noisy data than with solving problems expressed by mathematical formulas. The textbooks and research literature of numerical analysis have not fully caught up with the data revolution.

Numerical differentiation^WP via differencing is ill-conditioned^WP. That much is a well-known fact:

Wikipedia said:
An important consideration in practice when the function is approximated using floating point arithmetic is how small a value of h to choose. If chosen too small, the subtraction will yield a large rounding error and in fact all the finite difference formulae are ill-conditioned...

What isn't so widely appreciated is that the ill-conditioning of numerical differentiation amplifies quantization error much as it amplifies roundoff error.

Bottom line: For our purposes, calculating accelerations from positions by taking second differences, the error in the accelerations is linear in the position error but quadratic in the sampling rate.

That means femr2's subpixel resolution is helpful, but the higher sampling rates are substantially less helpful.

tfk said:
The end result of all of this is, right now, "art". As soon as someone can point us to the theory, it'll become engineering, where we can quantify the relationship between DAQ ("Data Acquisition") rate and measurement error.

The engineering says that there is error in all measurements and therefore in all calculations. The art says that there are acceptable & unacceptable levels of error.

I think I've explained the theory, and pointed you guys toward some references. Unfortunately, you'll probably have to study up on conditioning and roundoff error, and then figure out how to apply those principles to quantization error.

Merged Discussion of femr's video data analysis

Illuminator

Skeptic not Atheist

Penultimate Amazing

Illuminator

Illuminator

Illuminator

Penultimate Amazing

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Philosopher

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Illuminator

Penultimate Amazing

Illuminator

Illuminator

Illuminator

Illuminator

Penultimate Amazing

Illuminator

Skeptic not Atheist

Philosopher