Minimum vs. Cumulative vs. Average times for outcome

Debates and discussions on the various race scheduling methods that can be used and their fairness and accuracy in determining the winners.
User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Sat May 09, 2009 10:53 am

Stan Pope wrote:But, remember the context of the discussion ... summarize as "how to minimize/eliminate the influence of bad tracks on the race outcome." The post seems to be ignore that constraint and say "that's racing."
I seem to be on track, with the fact that out of the 6 lanes everyone is far more likely to find at least one lane they run well in... rather than using all the times for all the lanes which will cause more uneven results if the track favors or one type of car over another... if the race is structured properly using semis and finals you get the fastest cars running for all the potato chips in the lanes they do best in which minimizes the influence of bad tracks and makes for a great race at the end of the day.
Stan Pope wrote:In one of three runs, if I counted them correctly. It finished 2nd in the other two.
Oh and if you count the final event that would be 2 out of 4 races... hmm that would mean that 2 equal cars both won the same number of races on an uneven track... I think that was our goal, right?
Stan Pope wrote:So, in the example that I gave with two equally prepared cars go at it, what was the most important determining factor in who got the first place trophy? I think that it depends on how each builder chose which wheel to make dominant ... inside info? or chance? (HTA or PDL?)
As for this I will try one more time: If the cars were equally prepared PDL has nothing to do with it unless you are racing blocks right out of the box... likely a lot of work and skill went into the builds and the chance of 2 car being prepared by 2 different people and being dead even is so improbable that the PDL factor would only work into the equation about as often as one of the cars being struck by lightning just before the end of the race blowing the car to kingdomcome... As for the HTA, using your example, you might know that most of the lanes on the track favored the one DFW over the other and would build a car to maximize your potential of a good run by choosing this set up... in the end the car with this set up lost... making the HTA a loser.



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Stan Pope » Sat May 09, 2009 2:16 pm

Rod Turnbull wrote:Oh and if you count the final event that would be 2 out of 4 races... hmm that would mean that 2 equal cars both won the same number of races on an uneven track
There was no criterion posited for running the final race. How can we predict how it would come out?

All that aside, what we have seen from these discussions is that sometimes one method produces more accurate results, and sometimes other methods produce more accurate results. When a savvy organizer encounters such a decision, he looks outside to find "the best solution". Resources include personal experience, organization experience, and theoretical or statistical analysis (including game theory and simulation).

There are several aspects to consider, including:
1. What does "fastest entry" really mean?
2. If all the equipment works correctly, which method has the highest probability of identifying the fastest entry?
3. If there are undetected glitches in the equipment, which method has the highest probability of identifying the fastest entry?
4. Which method(s) allow interested parties to see for themselves (without relying on external authorities like timers) that the result was correct?

Personally, I think #4 is pretty important. Also, I like to include both organization experience and statistical analysis. Ya gotta be careful ... Stearns came highly recommended and looked really good until I started doing accuracy simulations on it! :(

YMMV.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Sun May 10, 2009 3:41 am

Stan Pope wrote:There was no criterion posited for running the final race. How can we predict how it would come out?
Rod Turnbull wrote:My professional opinion would be to run for fastest ET and use the times as qualifying for a final event and let the top 6 pick their lane (in order of fast ET time) and have one final race to determine the top 6 positions.
Stan Pope wrote:Since the question bear on lane equality, consider this track which penalizes RR cars as follows:


Code:
Lane: 1 2 3
L DFW: -0.03 -0.10 -0.10
R DFW: -0.10 -0.04 -0.05
Now since the L DFW car has lane choice, assuming someones not a complete more-on, they would pick lane 1 to run in... I'll let you figure out from here what happen.
Stan Pope wrote:what we have seen from these discussions is that...
I think you should only speak for yourself rather than lumping everyone in what a general term like 'we'... It's likely that others actually understood this concept.
Stan Pope wrote:1. What does "fastest entry" really mean?
Was the word 'fastest' or 'entry' is confusing you?
Stan Pope wrote:2. If all the equipment works correctly, which method has the highest probability of identifying the fastest entry?
Rod Turnbull wrote:run for fastest ET and use the times as qualifying for a final event and let the top 6 pick their lane (in order of fast ET time) and have one final race to determine the top 6 positions
Stan Pope wrote:3. If there are undetected glitches in the equipment, which method has the highest probability of identifying the fastest entry?
Rod Turnbull wrote:run for fastest ET and use the times as qualifying for a final event and let the top 6 pick their lane (in order of fast ET time) and have one final race to determine the top 6 positions
Stan Pope wrote:4. Which method(s) allow interested parties to see for themselves (without relying on external authorities like timers) that the result was correct?
Rod Turnbull wrote:Take a photo/video of the cars at the line in the final race and the timers don't even matter... there is no way to hand out a trophy to the wrong person.
I think we are starting to see some redundancy here...



User avatar
gpraceman
Site Admin
Site Admin
Posts: 4607
Joined: Fri Jun 20, 2003 12:46 am
Location: Highlands Ranch, CO
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by gpraceman » Sun May 10, 2009 1:08 pm

Rod Turnbull wrote:
Stan Pope wrote:1. What does "fastest entry" really mean?
Was the word 'fastest' or 'entry' is confusing you?
Being a little antagonistic? Hopefully, not.

That is a quite valid question. One person's definition of "fastest" may not be the same for someone else. Is it the single fastest time? Best average? Most points awarded? and so on. Each places vaules something different. They each have their pros and cons. It is really up to what an organization places their importance on. Not one system will work for everyone.


Randy Lisano
Romans 5:8

Awana Grand Prix and Pinewood Derby racing - Where a child, an adult and a small block of wood combine for a lot of fun and memories.

User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Sun May 10, 2009 3:16 pm

gpraceman wrote:Being a little antagonistic? Hopefully, not.
Me? :idk: Never, LOL... Besides, Stan started it! ;)
gpraceman wrote:One person's definition of "fastest" may not be the same for someone else.
Well actually if you break it down like I did it makes it more simple to answer yet preserves the dynamic quality you speak of. 'Entry' infers that you have or are entering a specific event, regardless of what scoring you want to attach to the process... 'fastest' is the car that wins the event according to the rule set out prior to the start of the event.

Therefor fastest entry = event winner

So the question should have been phrased: How do you best determine the 'fastest entry'? Not; What does 'fastest entry' really mean?

Even: What does 'fastest car' really mean? would work since no criteria is defined as is in an 'event'.



User avatar
Darin McGrew
Pine Head Legend
Pine Head Legend
Posts: 1736
Joined: Fri Jul 18, 2003 1:23 pm
Location: Mountain View, CA
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Darin McGrew » Sun May 10, 2009 4:10 pm

Rod Turnbull wrote:So the question should have been phrased: How do you best determine the 'fastest entry'?
I don't think that avoids the problem at all. You really need to decide what "fastest entry" means before you can "best determine" it.

If it's the first to cross the finish line, then you need head-to-head races between the fastest cars.

If it's the lowest average time, then you need a timer, and you need to set up the track and timer in a way that assures that times from different runs will be comparable.

And so on. You can decide what you think "fastest entry" means and then choose the appropriate race method, or you can pick a race method and let the race method determine what "fastest entry" means for your derby. But you can't discuss how to "best determine" something unless you first decide what that something is.



User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Sun May 10, 2009 9:14 pm

Darin McGrew wrote:You really need to decide what "fastest entry" means before you can "best determine" it.
You can't decide what fastest entry is... it is not something that is open for interpretation... I will break it down and make it even more simple:

fast - in this case we are looking at PWD cars... fast refers to the operation of the car, particularly it's ability to move quickly.

est - The superlative is the form of an adjective or adverb that shows which thing has that quality above or below the level of the others... hence fast'est' in this case would mean the PWD car that displayed the quality (fast) above the level of the other cars.

entry - this word further describes the car as one that is entered in an event or contest... in this case a PWD race.

Fastest Entry is the car that best displays the ability to move quickly in a particular PWD race.

Frankly it doesn't matter how you measure the quality, the definition of fastest entry never changes.
Rod Turnbull wrote:So the question should have been phrased: How do you best determine the 'fastest entry'?
Last edited by Rod Turnbull on Sun May 10, 2009 9:52 pm, edited 1 time in total.



User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Sun May 10, 2009 9:23 pm

You know it's the off season when an argument over semantics and grammar gets over 1400 views :O

Start working on next years cars people... or maybe even something constructive around the house. Sounds like a good idea for a new post. :scratching:


.
..._/""\__........~_o__............../""""\__.....~_o\__...........//[3].[4].[1].[2]\\.......MASTER
_"O----O-"____"O""-O"~_____"-O----O"___-O----O"____[]____________[]___PINEHEAD

User avatar
Darin McGrew
Pine Head Legend
Pine Head Legend
Posts: 1736
Joined: Fri Jul 18, 2003 1:23 pm
Location: Mountain View, CA
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Darin McGrew » Mon May 11, 2009 12:10 am

Rod Turnbull wrote:You can't decide what fastest entry is... it is not something that is open for interpretation...
Let's try a thought experiment. Say we have three cars. Each runs four races. Their times are:

Car A: 3.215 3.214 3.215 3.216 (average: 3.215, min: 3.214)
Car B: 3.230 3.210 3.231 3.229 (average: 3.225, min: 3.210)
Car C: 3.224 3.226 3.223 3.215 (average: 3.222, min: 3.215)

So Car A has the lowest average time, and Car B has the lowest minimum time. Just to complicate things, the three cars did not actually race against each other until the last time listed for each of them, when Car C crossed the finish line first.

Which one is fastest car? Don't you think it might be open to interpretation, just a little?



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Stan Pope » Mon May 11, 2009 12:30 am

Rod Turnbull wrote:Besides, Stan started it!
If you felt that I said something impolite, glib, flippant, demeaning, denigrating, impugning, or intellectually dishonest, then I sincerely apologize. There is/was no such intent.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Mon May 11, 2009 4:10 am

No, don't worry about it Stan, I was only kidding... you know how kids point fingers, I'm just mimicing one of the ankle biters... feel free to debate anything I post, it was the only interesting thing going on here for the past few days now that most of the racing action is over.
Darin McGrew wrote:Which one is fastest car? Don't you think it might be open to interpretation, just a little?
Rod Turnbull wrote:You can't decide what fastest entry is... it is not something that is open for interpretation...
Seems you are comparing apples to oranges Darin... 'fastest car' is not the same as 'fastest entry'... the fastest car can not be determined until you tell me what event you are entering it in and I know what the rules for the event are. The fastest entry (in a race of speed) is always the winner... the fastest car, not so much so. Just because apples come in different colors that doesn't change the fact that the oranges in my kitchen are in fact orange. LOL, just some food for your thought experiment... :thinking:
gpraceman wrote:That is a quite valid question. One person's definition of "fastest" may not be the same for someone else.
Can you see now where I disagree? It has nothing to do with what people view as fastest... it's the fact that the word 'entry' was attached to it, which implies that someone has already set a criteria for determining what constitutes or defines what fastest means.
Rod Turnbull wrote:What does 'fastest car' really mean?
Like I said... I belive this is how the question should have been posed to make it a valid one.



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Stan Pope » Mon May 11, 2009 11:04 am

A couple of questions, first about CMA's process for selection of race method ...

When a member suggests a competition method that appears to be worthwhile, what process does CMA use to decide whether to try it is an actual competition? Is there a formal evaluation process? If so, is the process available as a resource to others and how might I access it?

One of the method tests implicit in my 1997 paper (http://www.stanpope.net/pwmeval.html), based on my work from 1989, summarizes as follows: "If the event winner (or another of the "high finishers") had not competed, would the order of finish of the remaining entries change?" It looks like the presence of two or three "one lane wonders" would cause distortion in this test's results by vaulting a non-finalist into a high position in the final standings. I'd have to go through a thorough evaluation to tell if this is a frequent issue, affecting the overall accuracy rating of the method, or if it just an occasional artifact. None of my tools handle the method at this time, so an thorough evaluation is a few "round tuits" away, which leads me back to the first question. What has CMA's analysis results said about that issue?


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Darin McGrew
Pine Head Legend
Pine Head Legend
Posts: 1736
Joined: Fri Jul 18, 2003 1:23 pm
Location: Mountain View, CA
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Darin McGrew » Mon May 11, 2009 11:08 am

Ah, yes. I didn't catch your distinction between "entry" (where the rules are already defined by the event it is entered into) and "car" (where the rules are not necessarily defined yet). I was looking at the context of this thread, which is discussing what the best way to run a derby is, to properly reward the "fastest" car.



User avatar
Rod Turnbull
Master Pine Head
Master Pine Head
Posts: 241
Joined: Wed Dec 17, 2008 11:49 pm
Location: Ontario, Canada
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Rod Turnbull » Mon May 11, 2009 3:41 pm

The CMA stuff will not help you much as the races, events and skill competitions deal with issues that are not specific to PWD races... the rules are more of a frame work than anything else. As for the process, don't get me started, it is the reason I quit. If anyone is interest in a specific type of motorcycle racing and how it is score I could dig up some info.

As for the one lane wonders and questions as to how things would have turned out if a competitor had been excluded from the event... this would be a kin to a top ranked rider or riders falling in the heat races or qualifying races and not making the finals... which ultimately comes down to 'that's just racing'... they may have been the fastest rider with the fastest bike but according to the event rules if you are lying on the track when everyone crosses the finish line in a qualifying round... you obviously were not the fastest for that event. Some events have consolation races to give a rider a second chance to make the finals for just such a reason. There are a lot of things a race schedule depends on, usually there are fewer riders that you would see entries in a PWD event so a lot of the motorcycle stuff get put together on the fly just before each event depending on the given factors at the moment... size of the track, number of competitors, track conditions (which can change as the event progresses), safety concerns, weather, rules specific to the level of competition... organizers and officials must come up with the best way to determine a winner for each event while following the rules and factoring in the current race conditions... so we could be having a similar debate to this thread subject each week. A method which I feel could be used in PWD racing... I have excluded a lane and restructured a PWD race because I felt that the one lane did not perform in a consistent manor... maybe we should not be so ridged and work with problems as they arise... maybe this is the key to running a successful race and coming up with a better way to determine fastest car. I've seen a dozen or more methods for running a PWD event... some may work better than others depending on different conditions. For a perfect track I would feel good just using an average or a cumulative time for the race but as the quality of the track or location diminished I would definitely be scrambling for something different involving some heads up races with lane choices for determining the top positions.

Have you looked into using a mean average or removing outliers when doing statistical analysis of the different methods of running a race? I don't have time to look at your paper today (work sucks) but I will look at it soon.



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Minimum vs. Cumulative vs. Average times for outcome

Post by Stan Pope » Mon May 11, 2009 11:29 pm

Does CMA have a set of competition methods from which local organizers are encouraged or required to select for sanctioned events? Or does each local organizer get to "do his own thing" in selecting or concocting a scheme? If CMA has recommended or required methods, how do they go about deciding which methods should be included in the list? Is there a formal evaluation procedure for deciding if a method should be added to the list? ...
Rod Turnbull wrote:As for the one lane wonders and questions as to how things would have turned out if a competitor had been excluded from the event... this would be a kin to a top ranked rider or riders falling in the heat races or qualifying races and not making the finals... which ultimately comes down to 'that's just racing'...
Paragraph has good info, but answers a different question.
Rod Turnbull wrote:Have you looked into using a mean average or removing outliers when doing statistical analysis of the different methods of running a race? I don't have time to look at your paper today (work sucks) but I will look at it soon.
For analyzing actual competition results, I have worked with both arithmetic means and standard deviations with and without including the fliers. I usually only work actual race data to show whether or not there are abberations in the equipment or to develop "typical population data" for input to simulations. Actual race data tells you "one instance" of applying the method, usually for an unknown lane and racer population. Getting method accuracy measures requires a few thousand such races with lane and racer population distributions that reflect the real world (means and standard deviations). Even if I could get quality data, I don't have enough time to develop accuracy info that way.

Instead, the paper describes an evaluation concept involving simulation, using a statistical model of the racing environment (track and racers). It runs as many trials as are needed to stabilize the accuracy averages. For each trial a "racing environment" (including track and participants) is generated randomly, then the program applies the racing method to the environment. Each randomly generated sample contains "racers of known speeds on a perfect track" so the placement results from using the race method for a trial can be compared against the known rankings to tell the accuracy for the specific trial. After a few thousand such iterations, you can compute some accuracy averages for the method, including such things as the percent of time the Nth fastest racer finished in Nth place, and some other esoteric numbers.

Cory Young took the accuracy concepts from the paper and added/developed a good general case model of tracks and of racers and then built a nice simulation program dealing with charted points or timed racing and another that dealt with general N-loss elimination racing. My workbench has simulation programs for other racing methods in development, but I am lacking enough "round tuits" to finish 'em up!

Someone somewhere may have done something comparable, but I've not found 'em anywhere. So far as I've been able to tell, Cory's and my respective efforts are original and unique.


Stan
"If it's not for the boys, it's for the birds!"

Post Reply