Lane Bias Review

General track discussions.
Post Reply
Sylvar
Pine Head
Pine Head
Posts: 97
Joined: Mon Mar 01, 2004 8:22 pm

Lane Bias Review

Post by Sylvar »

Hello all,
My pack just completed our derby and a great time was had by all. This was a year of many firsts for us. First year with an electronic judge. First year using a Final standing method instead of double elim. First year with a newly refurbished track. I wrote a race manager for this years Derby and slotted the cars according the guidelines that Darin McGrew describes on his site. Everyone was very happy with this method since all boys raced raced an equal number of times on each lane and the standard deviation of head to head matchups was less than 1. Now I am poring over the results looking for signs of lane bias in the new track. At this point I am reasonably sure I know the answers, but I was surprised that some of my original thoughts on how to determine lane bias were completely wrong!

First thought: the lane with the most last place finishes is the slowest lane.

Fact: The lane that seems to be the slowest (based on head to head lane reversal results) Actually had the LOWEST number of last place finishes...It also had the highest number of second place finishes

Second thought : If you total up the finish positions of all the heats run in the lane the lowest scoring lane would be the fastest, next lowest the second fastest etc...

Fact : Due to the high number of second place finishes in lane 1 it looked like the second fastest lane, coming within just a few points of the fastest lane.

Are there any accepted metrics for tracking lane bias? I am happy that the lanes seem to be fairly closely matched, but closely matched cars make the bias more obvious. Are there any good methods for adjusting a lane to eleviate bias?

I have talked too long! Sorry,

Shane Morris
pack 189
Amanda, Oh
User avatar
Cory
Master Pine Head
Master Pine Head
Posts: 358
Joined: Fri Jul 25, 2003 7:18 am
Location: Chantilly, VA
Contact:

Re: Lane Bias Review

Post by Cory »

Sylvar wrote: Are there any good methods for adjusting a lane to eleviate bias?
Not sure how valid this is, but I like to take two lanes at a time and compare them.

Given a chart with perfect lane equity (PPN), each of two lanes has an equal chance of receiving the faster car in a given race, which makes each race between two lanes like a "coin flip".

E.g suppose you run a 4-lane race and the finish order is 4-2-1-3. Then you have six results to tally:

Lane 1 lost to Lane 2
Lane 1 lost to Lane 3
Lane 1 lost to Lane 4
Lane 2 lost to Lane 3
Lane 2 beat Lane 4
Lane 3 beat Lane 4

(I tally these in Excel using IF()....e.g. =IF(A1<A2,1,0)...then I use SUM() to add the results for each pair of lanes.)

Now focus in on Lanes 1 and 2. Suppose you had 200 races, and that Lane 1 won only 95 out of 200 races versus Lane 2. Are the two lanes uneven???

If you enter the following formula into Excel...

=BINOMDIST(95,200,0.5,TRUE)

...it will give the probability of getting 95 or fewer heads in 200 coin flips. The answer is over 25%, so this is not that unlikely an event -- it is not reason enough to reject the hypothesis that the two lanes are equal.

On the other hand, 85 or fewer wins for Lane 1 occurs only 2% of the time, so if that occurred you'd likely conclude that the lanes aren't even.

Further, assuming track setups from year-to-year are independent, you can multiply probabilities. If you got 95 wins or fewer from Lane 1 versus Lane 2 for three consecutive years, the chances of this are less than 2% so you'd likely conclude that the pair of lanes is uneven.
Sylvar wrote: Are there any good methods for adjusting a lane to eleviate bias?
On our old wooden track, we had some bad joints. I did a lot of sanding until cars would go over all the joints smoothly, then repainted, and it made a big difference.
User avatar
dknowles67
Master Pine Head
Master Pine Head
Posts: 163
Joined: Fri Jan 16, 2004 6:21 am
Location: Melbourne, Florida

Re: Lane Bias Review

Post by dknowles67 »

It seems like you must be talking about a very small lane bias.
I would wonder if the lane bias isn't insignificant in comparison to the staging at the top of the track, and the release mechanism. Maybe you have got all these things down to very minor differences.

I believe our track has a much more noticable difference in lanes.
I have never had an opportunity to perform this experiement, but I witnessed it during the races:
Say you want to test 2 lanes, to see which is faster -
Take two closely matched cars, and race one in each lane. Note which car finishes first, and by how much (e.g. 1/2 car length, 1 car length, fraction of an inch, etc). Then swap lanes, and race again. Repeat until you convince yourself that you are getting repeatable results, or give up.
If the faster car wins by the same amount in both lanes, then you have good lane equity.
If the faster car wins by a longer distance in one lane than it does in the other, then that lane is slightly faster, but you have fair lane equity.
If the winning car is determined by which lane it races in, then you have poor lane equity.
If the results seem random, and no pattern emerges, either the two cars are too equally matched, or the lanes are too rough to perform this experiment.
Sylvar
Pine Head
Pine Head
Posts: 97
Joined: Mon Mar 01, 2004 8:22 pm

Re: Lane Bias Review

Post by Sylvar »

It seems like you must be talking about a very small lane bias.
Hmmmm..... Small, yet significant. When I look thru the results at cars that were evenly matches (based on total score) I find instances where those cars went head to head with one in the suspect lane and again on more closely matched lanes and the outcome changes.
I agree that the staging of the car, done by the boys in our case, is a huge variable that makes my task harder. I wish we had timings, but we couldn't afford that upgrade. Maybe next year!

Cory,
I like your idea for extrapolateing more data out of the results. Let me give that a try. I will report my results.

Thanks!

Shane
Sylvar
Pine Head
Pine Head
Posts: 97
Joined: Mon Mar 01, 2004 8:22 pm

Re: Lane Bias Review

Post by Sylvar »

Ok, here are the scores based on all 94 heats

1 vs 2...1 vs 3...1 vs 4...2 vs 3...2 vs 4...3 vs 4
...48.........45........49........52.........49........48


Pretty even over all.


OK, Lets pull out heat that the top car ran in. In our case the top car in each age group went undefeated, therefore it was strong enough to overcome any lane bias. That leaves us 71 data points

1 vs 2...1 vs 3...1 vs 4...2 vs 3...2 vs 4...3 vs 4
..34.........31........35........39........38.........34


lets round off and call the middle 35. Each point from the middle being around 1.5% difference? Is that right or would it be 3%? at any rate we see this:
2 is 1.5% better than 1
3 is 6% better than 1
1 and 4 are equal
2 is 6% better than 3
2 is 4.5% better than 4
4 is 1.5% better than 3

Still looks good, but if 3 is better than 1 and 2 is better than 3 then why isn't the difference between 1 and 2 greater?

We are going to have to set this track up somewhere and do controlled trials aren't we.

Shane
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4926
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Lane Bias Review

Post by gpraceman »

An analysis of lane bias would be easier to do with times, as opposed to finish order. Continuous data is easier to apply statistical methods than discrete data.

If you had a set of times for each lane, you can compute the average and standard deviation for each lane. You should look at both since you can have lanes that have the same average, but a significantly different variation or the other way around.

Determining a "Significant" Difference in Standard Deviation
If Largest Std Dev / Smallest Std Dev > 2.72 then the difference is statistically significant, with a 95% confidence.

There is a little more accurate of a formula, but the above one is a good rule of thumb.

Determining a "Significant" Difference in Average
This is a lot more dicey. You can use a t-test forumla, but you'd need a stats book with a significance table to help you determine the significance of the resulting t value. This site, http://trochim.human.cornell.edu/kb/stat_t.htm has a decent explanation of the t-test. The visuals are helpful for underestanding why you need to look at both avearge and standard deviation.

Without all of the hair-raising calculations, you can put two data sets into this Student's t-test applet and it will do the analysis for you (must have a Java enabled browser).
Last edited by gpraceman on Wed Mar 03, 2004 9:18 am, edited 1 time in total.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
Sylvar
Pine Head
Pine Head
Posts: 97
Joined: Mon Mar 01, 2004 8:22 pm

Re: Lane Bias Review

Post by Sylvar »

Unfortunately, we could not afford that upgrade to our line judge. We don't have any information on the actual timing of the races. I agree, this analysis would be MUCH easier with that info. I like that site though! I will definately tuck that away in my bag of tricks!
User avatar
Cory
Master Pine Head
Master Pine Head
Posts: 358
Joined: Fri Jul 25, 2003 7:18 am
Location: Chantilly, VA
Contact:

Re: Lane Bias Review

Post by Cory »

gpraceman wrote: Without all of the hair-raising calculations, you can put two data sets into this Student's t-test applet and it will do the analysis for you (must have a Java enabled browser).
Given that many people don't or can't record times (or like me they question the consistency of recorded times over an entire day of racing), would it make sense to enter lane places into the applet, two lanes at a time? E.g...

Lane_1 Lane_2
3 1
2 4
3 2
4 2
1 2
3 4
3 4
. .
. .
. .
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4926
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Lane Bias Review

Post by gpraceman »

Cory wrote:Given that many people don't or can't record times (or like me they question the consistency of recorded times over an entire day of racing), would it make sense to enter lane places into the applet, two lanes at a time?
You can try it, but since it is "discrete data", you probably will not get a determiniation of "statistical significance". If I recall right, you may have to apply a different set of equations and it is only effective with a high number of data points. A easier and more accurate analysis is done with continuous data.

You may question consistency for the use in determining the final standings of a race, but for this analysis you are looking at the whole population of measurements, not small samples (a racer's set of heat times) so it really should not be an issue.

Besides if you are worried about the effect of bumping the track or some other event that you feel would shift the results, you can use this analysis to determine whether or not the event produced a "statistically significant" effect on the outcome. A very simple (Mark 1 Mod 0 Eyeball) comparison of averages or standard deviations will not, in most cases, tell you if the difference is statistically significant unless the differences are extreme. The statistical tests that I mentioned should really be performed if you want a difinitive answer.

If you have a timing system that can interface to a computer, you can use it to collect this data even if you are scoring by a points system. If you have commercial software package that will display times you can get the data from there. My GrandPrix Race Manager software will record times, even if points scoring is used (just switch to times scoring to view the data). If you do not have commercial software that will give times, then with serial timers you can use HyperTerminal to view the data stream and with a simple parallel port sensor system, there is free software available.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
Cory
Master Pine Head
Master Pine Head
Posts: 358
Joined: Fri Jul 25, 2003 7:18 am
Location: Chantilly, VA
Contact:

Re: Lane Bias Review

Post by Cory »

You can try it, but since it is "discrete data", you probably will not get a determiniation of "statistical significance".
I'll try it with our recent district data, it had 228 races.
Besides if you are worried about the effect of bumping the track or some other event that you feel would shift the results...
The applet assumes that each set of data comes from a single population. Given what I've seen happen to elapsed times when a tracks gets bumped, I'm worried that one set of data might end up coming from multiple populations, and that I can't be sure when the "shift" from one population to another occurred.

Thanks, though, this is very helpful!
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4926
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Lane Bias Review

Post by gpraceman »

gpraceman wrote:You can try it, but since it is "discrete data", you probably will not get a determiniation of "statistical significance". If I recall right, you may have to apply a different set of equations and it is only effective with a high number of data points. A easier and more accurate analysis is done with continuous data.
I'll correct myself. The other equations I was referring to are those pertaining to "attribute" data (Yes/No, Good/Bad, High/Low, etc.). With discrete data, you can use the same set of formulas, but you will probably need more data points to discern the statistical significance. It is a lot harder when your data is out to zero decimal places as opposed to out to 3 or 4.
Cory wrote:The applet assumes that each set of data comes from a single population. Given what I've seen happen to elapsed times when a tracks gets bumped, I'm worried that one set of data might end up coming from multiple populations, and that I can't be sure when the "shift" from one population to another occurred.
This should be shown by plotting the average of each set of heat times (each heat average is a separate data point) versus the heat number. If there was a shift in the average (up or down), you should be able to see it in that plot.

You could also run a controlled test. Get a set of data, then bump the track and get another set. Then analyze the two data sets to see if it really had a significant effect.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4926
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Lane Bias Review

Post by gpraceman »

Out of curiosity, I did a quick check of the data from our last race.

Analysis of Average
Here is what the Student's t-test applet analysis returned when comparing the worst lane versus the best lane (based on average of all times in those lanes):
Student's t-Test for Comparing Things

Java Implementation by Bryan Lewis, Kent State University, 1997

This test determines if there is a significant difference between the averages of the two samples above.

Mean of first data set : 2.6518964912280696
Standard deviation of first data set : 0.1809653735277583
Number of observations in the first set : 57.0

Mean of second data set : 2.6689157894736857
Standard deviation of second data set : 0.19031912371954995
Number of observations in the second set : 57.0

Degrees of freedom : 112.0
t Value (one-tailed) : 0.4892710082412615
P(x>t) : 0.3128011264062487

There is not a significant difference in the sample means.

A difference could not be detected due to large variability, small sample size, or both. Of course, it's possible that the samples really are from the same population!

An F test on the sample variances indicates that they could be from the same population, (alpha level of 0.005).

Accordingly, the t-test was set up for samples with equal population variance.
What at first glance may seem to be significant difference in the averages (2.6519 vs. 2.6689 or over 2 inches at the finish line) turns out not to be statistically significant.

Analysis of Standard Deviation
When comparing the best and worst standard deviation, the rule of thumb calculation ends up being:

0.19204 / 0.17687 = 1.08580 which is less than the 2.72 value, so there is not a significant difference.

The same lane was the "bad" lane, but there was a different "good" lane than for the average times analysis.

Conclusion
There just isn't enough evidence at this point to get a conviction against the "bad" lane, so it will live to race another year. :lol: That is not to say that I will not be looking at that lane with a closer eye to see how it might be brought even closer to the other lanes' average and to reduce the variation.

It was a good drill. This analysis would also be good for anyone undertaking experiments and wants to draw a difinitive conclusion about one data set when compared to another. Hint Hint
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
MathGuy
Master Pine Head
Master Pine Head
Posts: 178
Joined: Tue Dec 16, 2003 11:33 am
Location: NW burb of Chicago

Re: Lane Bias Review

Post by MathGuy »

I can't believe I missed this thread. I am actually more of STATGUY, then a pure MATHGUY.

There is a flaw with the analysis I am thinking. The standard deviation used in the T-Test was did not account for one key variable, IE Cars. The standard deviation are made up of two components.

SD(RACE)² = SD(CARS)² + SD(ERROR)²
IE Total Variance = Variance of Hypothecal means + Process Variance.

The times that you get out of the Lane times would be SD(RACE). The relevent standard deviation for lane total test is the SD(ERROR) for the track. I am going to go get the Feb WHIRL results for a sample to sample signicance of that.

BOTTOM LINE, IF CARS RACE IN EVERY LANE EQUALLY, YOU CAN MEASURE THE LANE EFFECT BASED ON THE AVERAGES, BECAUSE THE THE SAMPLES ARE UNBIASED IN CARS SELECTION, AND TIMER ISSUES (IE THE NUMBER OF SLOW STARTS ARE GOING TO BE THE SAME FOR ALL LANES)
Fun for one, Fun for all.
Rob D
User avatar
MathGuy
Master Pine Head
Master Pine Head
Posts: 178
Joined: Tue Dec 16, 2003 11:33 am
Location: NW burb of Chicago

Re: Lane Bias Review

Post by MathGuy »

SUMMARY OUTPUT
Regression Statistics
Multiple R 0.11660283
R Square 0.013596222
Adjusted R Square -0.023105106
Standard Error 0.014981781
Observations 84

ANOVA
d SS MS F Significance F
Regression 3 0.000250597 8.35322E-05 0.372157928 0.773299772
Residual 81 0.018180755 0.000224454
Total 84 0.018431352
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0%
Intercept 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A
X Variable 1 1.00151 0.00283 353.72977 0.00000 0.99588 1.00715 0.99588 1.00715
X Variable 2 0.99758 0.00283 352.34199 0.00000 0.99195 1.00322 0.99195 1.00322
X Variable 3 1.00091 0.00283 353.51573 0.00000 0.99527 1.00654 0.99527 1.00654
Above is the posted output of the Feb WIRL Races comparing the 3 lanes. I normalized the car's data, and regressed only on X1, X2, X3 (Lane 1,2,3). I tossed out the last place car runs, because there was some extreme runs there. As you can see, (Maybe), the signicance os the lanes here is not measurable to a degree of signicance. Lane 2 indicated to run 0.24% faster but the 95% CI is .99195 to 1.00322 Thus who knows.

But for another track, it might be measureable.
Fun for one, Fun for all.
Rob D
Post Reply