Timer Problems impact fairness of District Race

Discussions on race planning, preparations and how to run a "fair" and fun race.
User avatar
birddog
Master Pine Head
Master Pine Head
Posts: 307
Joined: Thu Apr 26, 2012 12:40 pm
Location: Denver, CO

Re: Timer Problems impact fairness of District Race

Post by birddog » Thu May 03, 2012 2:29 pm

I can say that in the district race in which we had these timer problems, they did throw out the lowest time for everyone (that was part of their race program as defined from the start).

Problem is, the timer had so many problems that went un noticed, that some cars had more than 1 heat impacted. In our case, it was at least 2 heats, most likely 3 (out of 4 total run).

I have no problems losing the race (taught us to be good losers as well as good winners, I hope), just thinking that if software can help other races from running into this problem in the future, why not try to correct it?



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Thu May 03, 2012 2:43 pm

birddog wrote:I can say that in the district race in which we had these timer problems, they did throw out the lowest time for everyone (that was part of their race program as defined from the start).

Problem is, the timer had so many problems that went un noticed, that some cars had more than 1 heat impacted. In our case, it was at least 2 heats, most likely 3 (out of 4 total run).

I have no problems losing the race (taught us to be good losers as well as good winners, I hope), just thinking that if software can help other races from running into this problem in the future, why not try to correct it?
I wonder if your organizers might be willing to send the data to Randy if Randy were looking to develop software that would detect and correct such equipment issues ... would be worth a try.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
birddog
Master Pine Head
Master Pine Head
Posts: 307
Joined: Thu Apr 26, 2012 12:40 pm
Location: Denver, CO

Re: Timer Problems impact fairness of District Race

Post by birddog » Thu May 03, 2012 9:29 pm

I'll ask them. I've already asked them for a copy for myself, but haven't seen a response. I suspect they just want this to fade away at this point, and to be honest, I don't blame them.

I really don't want to be a burden on them, but at the same time, if we could make some intelligent mods to the software such that other race managers in the future never have to run into this particular problem again, it seems worth the effort to me.

I'll email out a request tonight and post again if I get a response.



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Thu May 03, 2012 9:36 pm

Yes, I think that the organizers may be much more willing to send Randy the data for purposes of improving the software.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Fri May 04, 2012 9:52 pm

Well, there it is, Randy ... Can you be the first to include Verification of a Quality Racing Environment along with your already Quality Schedules? BTW, if you work both ends of the time spectrum, you can catch some other occasional equipment problems:
1. Marginally slow manual gate operation.
2. Sticky microswitch in start gate.

Regardless of your plans, my plan is to finish the documentation of my algorithm and the related statistical basis and then post it on my web site in hopes that it will be incorporated into all race management software.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
FatSebastian
Pine Head Legend
Pine Head Legend
Posts: 2646
Joined: Wed Jun 17, 2009 2:49 pm
Location: Boogerton, PA

Re: Timer Problems impact fairness of District Race

Post by FatSebastian » Fri May 04, 2012 10:21 pm

Stan Pope wrote:Here is a different algorithm which does away with the ambiguous "much slower" criterion: [...] 3. If no heats have as many slowest heat tabs as there are lanes, then continue
:thinking: I wonder about the situation where some cars are unfortunate enough to encounter a slow heat more than once. That is, suppose 3 cars on a 4-lane track have their slowest time on a defective heat, but one unfortunate car has a slower heat still, such that the heat is not flagged. Are there scenarios (perhaps pathological) where the flagging of bad heats would break down, particularly if malfunctions were common or there were some cars involved with variable performance.

Is there special processing required for DNF's?



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Sat May 05, 2012 1:10 am

FatSebastian wrote:
Stan Pope wrote:Here is a different algorithm which does away with the ambiguous "much slower" criterion: [...] 3. If no heats have as many slowest heat tabs as there are lanes, then continue
:thinking: I wonder about the situation where some cars are unfortunate enough to encounter a slow heat more than once. That is, suppose 3 cars on a 4-lane track have their slowest time on a defective heat, but one unfortunate car has a slower heat still, such that the heat is not flagged. Are there scenarios (perhaps pathological) where the flagging of bad heats would break down, particularly if malfunctions were common or there were some cars involved with variable performance.

Is there special processing required for DNF's?
Good insight!

Iteration covers most of the problem cases, but DNF creates a special situation that must be dealt with if the algorithm is to be successful. Off the top of my head, DNF's must be excluded from the analysis as they are not (probably) a result of the bad times causes.

Of course, the algorithm applies only to malfunctions which affect all of the lanes. A single lane equipment malfunction would not fall to this algorithm. (We had one of those a few years ago, and it would have gone undetected had a knowledgeable fellow in the audience not persisted in his requests of the track staff to look at the data again!)


Stan
"If it's not for the boys, it's for the birds!"

User avatar
gpraceman
Site Admin
Site Admin
Posts: 4696
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Timer Problems impact fairness of District Race

Post by gpraceman » Mon May 07, 2012 11:19 am

Stan Pope wrote:Well, there it is, Randy ... Can you be the first to include Verification of a Quality Racing Environment along with your already Quality Schedules?
I'll definitely consider some type of audit system for the next version of GPRM. I will just want to make sure that false positives are rare.


Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Mon May 07, 2012 2:00 pm

gpraceman wrote:
Stan Pope wrote:Well, there it is, Randy ... Can you be the first to include Verification of a Quality Racing Environment along with your already Quality Schedules?
I'll definitely consider some type of audit system for the next version of GPRM. I will just want to make sure that false positives are rare.
I'd expect nothing less!

You've been so quiet the past few days that I thought you might have been secluded, working out the details and prepping it for release! :)


Stan
"If it's not for the boys, it's for the birds!"

User avatar
FatSebastian
Pine Head Legend
Pine Head Legend
Posts: 2646
Joined: Wed Jun 17, 2009 2:49 pm
Location: Boogerton, PA

Re: Timer Problems impact fairness of District Race

Post by FatSebastian » Mon May 07, 2012 3:40 pm

gpraceman wrote:I'll definitely consider some type of audit system for the next version of GPRM.
If one is willing to wait until a larger sample population is generated, such as with Stan's proposed algorithm, I wonder if an outlier identification process based on a statistical analysis (not unlike your proposal for testing for a slow lane) might also be a worthwhile approach to investigate. (Given your black-belt training, you might be comfortable with such an approach.)



User avatar
gpraceman
Site Admin
Site Admin
Posts: 4696
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Timer Problems impact fairness of District Race

Post by gpraceman » Tue May 08, 2012 9:45 am

Stan Pope wrote:You've been so quiet the past few days that I thought you might have been secluded, working out the details and prepping it for release! :)
Naw. I've been doing a bunch of projects around the home to get it ready to sell. Looking to downsize as our kids are leaving the nest.
FatSebastian wrote:(Given your black-belt training, you might be comfortable with such an approach.)
I'd have to do some dusting off of my stats training, since I got out of manufacturing many years ago.


Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Wed May 09, 2012 11:49 pm

Here is an interesting tidbit, assuming that my computations are correct.

If I conduct a timed race on a 3-lane track with 30 competitors each running twice in each lane, then, absent equipment malfunction, the probability is 0.76 that there will be no occurrences where a heat's participants register their slowest times of the day . The probability is 0.97 that there will be 1 or fewer. Virtually no "false positives!

Since I really want to catch and correct occurrences of equipment malfunction rather than avoid a comparable number of unnecessary reruns, I would probably elect to set thresholds at the 0.75 (or even less) probability than at the psychologists' favorite 0.95.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Sun May 13, 2012 11:04 am

Here is the "beta" of my paper on the subject: http://www.stanpope.net/timeerror.pdf" target="_blank. I think that it is sound. Critiques welcome via email (please NO PM.)


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Timer Problems impact fairness of District Race

Post by Stan Pope » Sun May 13, 2012 4:33 pm

I think that the most interesting conclusion one can draw from my paper is that for the most commonly used PPN schedules (3 lanes with each racer running twice in each lane and 4 lanes with each racer running once or twice in each lane) and group sizes (15 to 40 racers), it only takes one heat in which each racer registers his slowest (or fastest) run of the day to trigger the alarm bells that there is an equipment problem.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
FatSebastian
Pine Head Legend
Pine Head Legend
Posts: 2646
Joined: Wed Jun 17, 2009 2:49 pm
Location: Boogerton, PA

Re: Timer Problems impact fairness of District Race

Post by FatSebastian » Mon May 14, 2012 6:24 pm

Stan Pope wrote:in about 1/4 of the events one or more heats will be rerun needlessly.
Stan's algorithm is impressive; however, I wonder if most might still want to keep the bar set relatively high (say, 95% or more) for “needless reruns”. Reruns cast a shadow on all race outcomes, not just those outcomes being replaced. A race operator may be uncomfortable clobbering outcomes that have a high likelihood of changing the race standings, such that a high confidence threshold seems warranted.
Stan Pope wrote:Here is the "beta" of my paper on the subject...
If I understand the algorithm, this proposed approach flags those runs where the lowest (or highest) times of a group of racers all occurred in the same run, and then the statistical test is iterated by re-evaluating the smaller population of remaining "good heats".

If so, I wonder if it is feasible to flag situations where, say, "one of the [two?] lowest/highest" times of each racer all occurred against each other, and then re-test (at a high level of confidence of, say, 95% or more) after clobbering those rejected runs with "good" reruns. By not limiting the test to considering only the absolutely lowest/highest times, there remains an ability to detect cases where a racer was affected by more than one malfunction, and perhaps avoid an iterative procedure that might break down?



Post Reply