Timer Problems impact fairness of District Race

Discussions on race planning, preparations and how to run a "fair" and fun race.
User avatar
birddog
Master Pine Head
Master Pine Head
Posts: 307
Joined: Thu Apr 26, 2012 12:40 pm
Location: Denver, CO

Timer Problems impact fairness of District Race

Post by birddog »

Folks-

My son joined Tiger Scouts this year, so this is my first year participating in Scouts since I was a kid. We had a wonderful time working together on our PWD car this year and are already looking forward to next year.

I've volunteered to be the race coordinator for our Pack race for the next 4 years while my son is in Cub scouts.

We did very well at our Pack race. Took first place. Our times were very consistent across all the races (within about 0.03 seconds or so).

At the district races, I was video taping much of the action when my son's car was running, so I didn't get to pay attention to all the times. I do know his first time was one of the top 3 or 4 times of all the cars. He won all 4 of his "heat" races, but I noticed at least one of his times was different by a full 0.2 seconds. This happened to every other car in that heat as well.

I mentioned the problem to the race officials and they said they were aware of the timer problems and would manually scan the data looking for problems and re-run any heats that needed re-running.

Unfortunately, although a couple heats were re run at the end, none of the heats my son ran in were re-run as they should have been.

This caused him to place in 31st place as opposed to 3rd or 4th place where he probably should have been. He knows nothing about this, I've just told him that he raced slower cars in his heats and didn't get a chance to go head to head with any fast cars.

When I got home that night, I pulled out the video tape and found out what had happened. The track was using a laser or infra red timer to start the timing system. When the first car crosses the laser, the light goes out and starts the timer. Sometimes, the timer would actually go out before any cars crossed the path due the the fragility of the timing system as the starter manually moved the starting gate, causing the timer to trip early. I was able to measure 6 video frames of the timer starting too early on one of his heats. 6 frames at 30 frames per second accounts for the 0.2 seconds we saw in the measured time.

The race was using the Grand Prix Race Management software package, which worked great. What I'm wondering is this:

Would it make any sense to add an auditing option to the GPRM software program where it could audit the race data for problems such as this and throw a dialog box up if it sees something amiss?
With the data from this race, I'll bet the software designer could easily come up with an audit that would prevent any other race from suffering from an inconsistent timer malfunction such as what occurred at this race.

I'm trying to get the race by race data from the race coordinator, but he isn't interested in providing it. Personally, I'd like to learn from this so that when I run my races for the next 4 years I can be sure this is one problem I'll know how to handle.

Interested to hear what the experts on this forum think.

thanks.
User avatar
FatSebastian
Pine Head Legend
Pine Head Legend
Posts: 2788
Joined: Wed Jun 17, 2009 2:49 pm
Location: Boogerton, PA

Re: Timer Problems impact fairness of District Race

Post by FatSebastian »

:welcome: birddog!
birddog wrote: Interested to hear what the experts on this forum think.
Laser gates have a reputation for causing weird results and some operational hassles. See this discussion of drawbacks Too bad your son's standings may have suffered.
birddog wrote:I'd like to learn from this so that when I run my races for the next 4 years I can be sure this is one problem I'll know how to handle.
The best way to handle may be to replace the laser starting gate with a mechanical switch.
User avatar
Darin McGrew
Pine Head Legend
Pine Head Legend
Posts: 1825
Joined: Fri Jul 18, 2003 1:23 pm
Location: Knoxville, TN
Contact:

Re: Timer Problems impact fairness of District Race

Post by Darin McGrew »

If you trust the finish order but not the actual times, then another approach is to switch to a point-based system based only on the finish order.
User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6856
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope »

"Scanning the data" looking for anomolies is difficult to do, and more so under the time pressure of the event. Some extra tools would be helpful.

One such tool is to compare results when the racers are ranked by time and when ranked by points. (This capability is built into GPRM, I think.) Discrepencies in those rankings suggest racer's whose heats should be inspected in detail.

The "inspection" is
1. Compare the times in that racer's heats to his average.
2. For each "much slower" heat, look at each car in that heat, comparing its time in that heat to its average. If all show "much slower" then that heat's times are bad.

This is an inspection that GPRM could perform on its own to tell the staff that they "have problems."
Stan
"If it's not for the boys, it's for the birds!"
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4919
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Timer Problems impact fairness of District Race

Post by gpraceman »

birddog wrote:When I got home that night, I pulled out the video tape and found out what had happened. The track was using a laser or infra red timer to start the timing system. When the first car crosses the laser, the light goes out and starts the timer. Sometimes, the timer would actually go out before any cars crossed the path due the the fragility of the timing system as the starter manually moved the starting gate, causing the timer to trip early. I was able to measure 6 video frames of the timer starting too early on one of his heats. 6 frames at 30 frames per second accounts for the 0.2 seconds we saw in the measured time.
The laser sensor start can be problematic, as FS mentioned. Alignment of the beam is critical. It sounds like the vibration of the gate opening or the operator maybe bumping the track when tripping the gate caused the beam to get enough out of line with the sensor to trip it. I am not a fan of the laser start sensor and recommend using the mechanical switch that Micro Wizard provides. It has several advantages over the laser sensor, especially consistency.
birddog wrote:Would it make any sense to add an auditing option to the GPRM software program where it could audit the race data for problems such as this and throw a dialog box up if it sees something amiss?
With the data from this race, I'll bet the software designer could easily come up with an audit that would prevent any other race from suffering from an inconsistent timer malfunction such as what occurred at this race.
Such auditing may be difficult. Trying to detect an anomaly with any given racer's times can be difficult since you might only have 3 or 4 data points, at some events, to compare. That is just not enough data to make a statistical comparison of. An anomaly could easily be due to differences in staging or a change to the car, like damage from a crash.

Trying to compare time averages for each heat can also be problematic. You do have more data points for comparison (usually) but there can still be things that can cause anomalies which do not relate to the timer. At many races, you do have some pretty slow cars, so that can throw off the heat average for the heats they are in.

A concern that I have would be to falsely indicate a possible problem with the results. If the race crew is not able to immediately explain the possible anomaly, they may rerun races unnecessarily. A car's performance can get better or worse as the race proceeds, so a rerun might just change the end results.

As Stan pointed out, GPRM does already provide some lane and individual stats that the race crew can review if they suspect that there is a problem. They can also scan down the heat results looking for something askew.

If someone can think of a check(s) that won't result in a bunch of false warnings, I'd consider it for the next version of the software.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
psycaz
Master Pine Head
Master Pine Head
Posts: 667
Joined: Wed Apr 13, 2005 9:21 am
Location: Somewhere, US

Re: Timer Problems impact fairness of District Race

Post by psycaz »

Randy,

Would it work to be able to input a time of the user's choosing at which any time slower would generate a warning box?

That way each user could select a time that would correspond to their track.

EX, The track we used the slowest time was 3.250. If we could be able to tell the software to notify us of any times 3.450 or slower, it should help spot suspect times.
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4919
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Timer Problems impact fairness of District Race

Post by gpraceman »

psycaz wrote:Would it work to be able to input a time of the user's choosing at which any time slower would generate a warning box?
That might work for competitive races, but if there any pretty slow cars that can skew things leading to many false warnings.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
psycaz
Master Pine Head
Master Pine Head
Posts: 667
Joined: Wed Apr 13, 2005 9:21 am
Location: Somewhere, US

Re: Timer Problems impact fairness of District Race

Post by psycaz »

gpraceman wrote:
psycaz wrote:Would it work to be able to input a time of the user's choosing at which any time slower would generate a warning box?
That might work for competitive races, but if there any pretty slow cars that can skew things leading to many false warnings.
Maybe it could be adjustable during the race?

It would be something. I would presume it could be turns on and off as well so that if it generated too many false warnings it could be tuned off for the race.
User avatar
birddog
Master Pine Head
Master Pine Head
Posts: 307
Joined: Thu Apr 26, 2012 12:40 pm
Location: Denver, CO

Re: Timer Problems impact fairness of District Race

Post by birddog »

Some good ideas here. I agree, the last thing you want to have happen is to post warnings to race managers that are not really problems.

I've got some data from our pack race and in looking at that data, I can see a wide range of times for cars. I would say about 80% of the cars are fairly competitive and 20% are running times that are fairly slow and inconsistent.

What if the race manager could enter what he/she believes is a "competitive" time for their track. All cars with times at or below that mark would be eligible for a "consistency check/audit". The race manager could also enter what he/she believes would be a variance that would trigger a pop up.

For example, in our Pack race, where we have more of a range of competition, I would enter in a time of 3.15 seconds. Any cars with a time that was that fast, would be eligible for the audit. I would also enter in a time of 0.2 seconds of variance. Then, the audit would tell me if I had any cars that ran a time of 3.15 or less and had a variance of 0.2 in their times. I suspect this would be auditing only about 80% of the cars (the competitive ones).

For the district race, where the competition is higher, I would tighten up the numbers. For example, assuming use of the same track, I might enter in a time of 3.00 seconds as the competitive time and 0.12 as the variance time. Although I do not have the data, I suspect almost every car in the district race would have been in the audit.

Of course, you could also choose to just turn the audits off if you are certain of the reliability of your hardware.

Our Pack race uses a BestTrack with a Champion timer that has a mechanical timer start, thus no lasers to get out of alignment there, but I guess you never know what kinds of problems may occur, and I want to be ready for anything.

I'm also volunteering to offer our Pack's new track and timer for next years district race (and to help run it), if they'll have me.

Pretty neat that the owner of GPRM software actually responded to this. Thanks Randy. I plan to purchase your software for our Pack races next year!
rpcarpe
Master Pine Head
Master Pine Head
Posts: 736
Joined: Wed Mar 17, 2010 11:58 am
Location: Huntsville, Alabama

Re: Timer Problems impact fairness of District Race

Post by rpcarpe »

We used GPRM in our pack races and District races, mechanical switches for both starting systems.
During set-up/test, we ran fast cars from previous years to see how well the system worked.
Any races that showed 'weird' times, we just made a note of the heat and re-ran it at the end.
This requires the computer operator to have a good idea of what the good times should be.
Not tough to do... you'd probably do fine.
My wife started a new support group... Widows of the Pinewood Derby.
User avatar
gpraceman
Site Admin
Site Admin
Posts: 4919
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Timer Problems impact fairness of District Race

Post by gpraceman »

Stan Pope wrote:1. Compare the times in that racer's heats to his average.
2. For each "much slower" heat, look at each car in that heat, comparing its time in that heat to its average. If all show "much slower" then that heat's times are bad.
Looking back at Stan's post, I like this approach best so far. Though, trying to do that early on in a race would be problematic, since you need enough data to establish averages for the racers. So, maybe wait till the last heat is completed to perform an automated audit and then report any heat(s) that the race crew should review the data for.

I do also think that an audit should look for a "much faster" time scenario as well as "much slower". I can think of a couple of scenarios where times can be faster than usual.
Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.
User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6856
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope »

Here is a different algoritm which does away with the ambiguous "much slower" criterion:

After scheduled heats are complete, but before results are finalized:

1. Tabulate the slowest heat for each racer, noting its heat number.

2. Count the number of times each heat number is listed as 'slowest heat'.

3. If no heats have as many slowest heat tabs as there are lanes, then continue at #4,
else for each heat in which the heat is listed as "slowest heat" for each racer in the heat (i.e. has a high probabbility of being defective):
a. schedule that heat for a rerun
b. remove that heat's data from the table of results
c. repeat from #1

4. Tabluate the fastest heat for each racer, noting its heat number

5. Count the number of times each heat number is listed as "fastest heat."

6. If no heats have as many fastest heat tabs as there are lanes, then stop analysis and report results as okay,
else for each heat in which the heat is listed as "fastest heat" for each racer in the heat:
a. schedule that heat for a rerun,
b. remove that heat's data from the table of results
c. repeat from #4.

The reason for the iteration in 3c and 6c is that one or more racers might participate in more than one defective heat ... the iteration catches those.

The probability of false hits is pretty low and declines as the number of lanes and the number of heats per racer increase. The probability of false hits increases as the number of racers in the competition increases.

I think that the following probabilities are correctly expressed, but someone with a younger brain should check me on 'em.

P(a specific good heat is not flagged as defective by chance) = 1 - (1 / (# of heats per racer)) ^ (# of lanes).
For a typical 3 lane, twice in each lane race, this would be 1-(1/6)^3 ~= 0.9954 or 99.5%

P(no good heats are flagged as defective by chance) = P(a specific good heat is flagged as defective by chance) ^ (# of heats in competition).
If 30 racers were entered (60 total heats), the P(no heats are flagged as defective by chance) = P(a specific heat is flagged as defective by chance) ^ 60 = 0.7570 or 75.7%

In other words, in about 1/4 of the events one or more heats will be rerun needlessly.
Stan
"If it's not for the boys, it's for the birds!"
dna1990
Master Pine Head
Master Pine Head
Posts: 360
Joined: Wed Jan 09, 2008 8:16 pm
Location: Houston, TX

Re: Timer Problems impact fairness of District Race

Post by dna1990 »

I do several apporaches like Stan describes in the week "after" the race, with many more samples points to work with - looking for lane or heat bias. Even then one has to real-world that to certain cars or places where they have low stddev. I do not think cars made from wood, running on plastic wheels and nails - for only 3, 4, or even 8 runs is enough sample points to make clear, something as small as 0.02.

Why not just throw out the bottom time for each racer? GPRM already has this function, we love it.

And what happens when you run such an audit, and announce re-runs (many needless as Stan points out). You will be hammered with questions and doubt. And lastly part of PWD is about losing, even thru no fault of your own. Had some high school robotics kids this week the World Championship...epic issues in the final rounds with technical communications the bots use to connect to the field. While many will work to understand and correct that in the future, many students had to swallow this 'bad luck' at the most inopportune time. And they did so graciously. Well, ok, pretty graciously.
User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6856
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope »

If a malfunction is detectable in a timely fashion, it should be detected and corrected.
dna1990 wrote: Why not just throw out the bottom time for each racer? GPRM already has this function, we love it.
This feature throws out a lot of good runs along with the first malfunction experienced by a racer, but it fails to discard the 2nd malfunction experienced by a racer.

Even if there are no equipment malfunctions, discarding each racer's slowest time biases the competition in favor of the less consistent racer. If any times are to be discarded, both fastest and slowest should be scrapped.
dna1990 wrote:And what happens when you run such an audit, and announce re-runs (many needless as Stan points out).
This is not a correct interpretation of my statements. A correct interpretation would be "A few are needless in some events."
Stan
"If it's not for the boys, it's for the birds!"
User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6856
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope »

The next step that someone trained in probability might take is to have the software identify the minimum count of "slowest times in same heat" occurrences (based on # of lanes, # of racers, and # of runs per racer) at which it is 95% (or some other percentage) certain that heat glitches occurred. Then always report if, at end of racing, that threshhold was exceeded and which heats were involved.
Stan
"If it's not for the boys, it's for the birds!"
Post Reply