# The Statistical Case for Combined Finals



## Ravi (Aug 8, 2010)

I have long been of the opinion that a five-solve trimmed average measures a speedcuber’s skill too unreliably to decide the winner of a competition. Recent events have shown this observation to be correct, especially after the tightening of the top group of competitors that has taken place over the last several years. One case that comes to mind is Breandan Vallance’s spectacular yet improbable 10.74 average in last year’s world championship. Another is last year’s US Nationals: although Rowe did have the best overall performance, his victory must be attributed at least partly to chance, as the three rounds of the competition actually produced three different winners. As I write this, the 2010 edition of US Nationals promises to be even more so, as none of the top three cubers in the first round managed to return to the podium in round 2.

I recently calculated some statistics to test just how uncertain averages can be. I started by assuming that any given speedcuber’s times are modeled fairly accurately by a normal distribution (bell curve)--this ignores factors such as lucky cases, DNFs, nerves, and mechanical slips*, but it should be close enough for our purposes. Then any speedcuber can be represented essentially by two numbers: his or her mean and standard deviation. The purpose of taking averages, of course, is to decrease the standard deviation while preserving the mean. The question: How much do standard deviations decrease when averages are used?

There are two ways to find out, and I somewhat regrettably chose the one that didn’t involve solving Rubik’s Cubes all night. First, I programmed the trimmed-averaging process into Mathematica (the input is a list of arbitrary length):

trim[x_] := Drop[Drop[Sort[x], -1], 1]
avg[x_] := Mean[trim[x]]

Now it’s time for some number-crunching. I had Mathematica calculate a million random averages using the default mean of 0 and standard deviation of 1 (a translation and a dilation away from our ideal cuber’s distribution), then calculate the standard deviation of the averages.

StandardDeviation[Table[avg[RandomReal[NormalDistribution[0, 1], 5]], {i, 1000000}]]

I tried this several times, generally getting results around 0.4765. This means that the standard deviation of your averages is slightly less than half of the standard deviation of your raw times; that is, if you ordinarily have a standard deviation of 1.4 seconds, the standard deviation of your averages will likely be around 0.7 seconds. For comparison, I tried a few other averaging schemes, with the approximate coefficients listed below:

single solve (mean of 1): 1.000
avg (median) of 3: 0.670
mean of 3: 0.577
mean of 4: 0.500**
avg of 5: 0.477
mean of 5: 0.447
mean of 10: 0.316
avg of 12: 0.295
mean of 3 avgs of 5: 0.275***
avg of 15: 0.262
mean of 15: 0.258
mean of 20: 0.224

You can experiment with this by adjusting the “5” in the code above and, for untrimmed means, changing “avg” to “Mean”.

What do these statistics mean? Well, averages of 5 are certainly more precise than single solves, but it is not difficult to improve further. An average of 12, for example, theoretically has only 62% of the standard deviation of an average of 5. Moreover, most major competitions already have three rounds of five solves each, so using combined finals for 3x3 could cut standard deviations nearly in half without requiring any additional time in the schedule. Of course, if more than five solves were used to calculate averages, it may be necessary to allow, say, two dropped times instead of one for the odd DNF. Or, in the interest of maintaining compatibility with existing average records, it might make more sense to simply rank competitors based on the mean of their averages throughout the competition, in the same way that a golf tournament is scored. This would have no adverse effects on scheduling, compatibility with past records, or (in my opinion) suspense, but it would make our competition results much more precise (coefficient = 0.275).

In more practical terms, let’s imagine a pair of hypothetical competitors, A and B. A and B both have standard deviations of exactly one second, but A is half a second faster than B on average. If they compete with a single solve in a one-on-one competition, then A has a 63.8% chance of winning. With trimmed averages of 5, A has a 77.1% chance of winning. With a mean of three averages of 5, as I suggested above, A has a 90.1% chance of winning****.

In the next few hours, the final round of this year’s US Nationals will take place. If, as I expect, no single person emerges as a clear champion, then I believe it will be time to seriously consider a more precise averaging scheme for our competitions.

May the best cuber win.




* I suspect that all of these factors, especially nerves, would mildly increase unpredictability and therefore all standard deviations. Also, the distribution is somewhat skewed in real life, since (for example) a speedcuber who averages 11 seconds is much more likely to screw up somehow and solve in 18 seconds than to get lucky enough to solve in 4 seconds. A generalized extreme value distribution is likely to provide a better model, but only slightly so.

** You may notice that the coefficient for a mean of n is 1/sqrt(n). This follows from the fact that for independent X and Y, stdev(X+Y) = sqrt[stdev(X)^2 + stdev(Y)^2]. Exact values for the standard deviations of trimmed averages are less obvious.

*** Mathematica code:
StandardDeviation[Table[Mean[Table[avg[RandomReal[NormalDistribution[0, 1], 5]], {i, 3}]], {i, 
1000000}]]

**** Mathematica code:
N[CDF[NormalDistribution[0, Sqrt[2]], 0.5]]
N[CDF[NormalDistribution[0, Sqrt[2]*0.4765], 0.5]]
N[CDF[NormalDistribution[0, Sqrt[2]*0.275], 0.5]]
The factor of sqrt(2) comes from the fact that I did the calculation by subtracting two normal distributions centered on the origin and checking the probability of the signed difference being less than 0.5. Standard deviations of differences of normal distributions are the same as those of sums--see the double-asterisked item above.


----------



## Kirjava (Aug 8, 2010)

I'd rather that the person who set the best average at the comp won, no matter what round it was in.


----------



## Carrot (Aug 8, 2010)

Kirjava said:


> I'd rather that the person who set the best average at the comp won, no matter what round it was in.



GO JAPAN!!


----------



## riffz (Aug 8, 2010)

Kirjava said:


> I'd rather that the person who set the best average at the comp won, no matter what round it was in.



I've long been of this opinion. Moving on to the next round would be important because you would have more chances to set a better average.


----------



## Ravi (Aug 8, 2010)

By the way, Rowe Hessler won in much the same way as last year: he did have the best overall performance, but could have lost, if his fourth-place finish in round 1 is any indication.

http://www.cubingusa.com/usnationals2010/results/index.php#7


----------



## Lucas Garron (Aug 9, 2010)

An important fact to consider: In the earlier rounds of a tough competition, cubers often try to make it to the next round rather than a fast average.
Two DNFs are unacceptable, adding 2 seconds to your average can be.

In the finals, however, it's about the maximum speed and all its risks.


----------



## Carrot (Aug 9, 2010)

Lucas Garron said:


> An important fact to consider: In the earlier rounds of a tough competition, cubers often try to make it to the next round rather than a fast average.
> Two DNFs are unacceptable, adding 2 seconds to your average can be.
> 
> In the finals, however, it's about the maximum speed and all its risks.



GOGO GARRON!


----------



## Ravi (Aug 9, 2010)

Lucas Garron said:


> An important fact to consider: In the earlier rounds of a tough competition, cubers often try to make it to the next round rather than a fast average.
> Two DNFs are unacceptable, adding 2 seconds to your average can be.
> 
> In the finals, however, it's about the maximum speed and all its risks.



You have a point: It would be (and is) unfortunate for a serious competitor to be eliminated early because of DNFs. Of course, the same can happen in a final, but combined finals would increase the probability. At the same time, though, is there any reason for one average to be "worth more" than another in terms of ranking? I would argue that someone who averages 12, 11, and 13 is a better speedcuber than one who averages 14, 13, and 12.

Kir and riffz also have an interesting point, but I can't say I'm totally in favor of it. I think a great speedcuber is defined more by consistent skill than by occasional spikes of incredible speed--to use WC2009 as an example again, I'm more impressed by Tomasz Zolnowski's 11.09, 11.89, and 11.64 than by Breandan's 12.80, 12.20, and 10.74. Also, I would assume that a combined final would statistically point to the fastest competitor much more reliably than a "best of"-type format. (Of course, what you're talking about is far better than the straight best-of-3 used in WC1982. But the speedcubing world of 1982 was a totally different place.)

What do you guys think about the stats though? Does a theoretical improvement of 77.1% to 90.1% accuracy merit changing the rules?


----------



## qqwref (Aug 9, 2010)

Ravi, there are two important factors that you are not considering. First: there is always a chance of a very nice time from a skip or a very bad time from a pop or similar problem. This even extends into averages as it is always possible in theory to get two DNFs (or bad times) or two skips (or very nice solves). Very bad times are more likely in a competition due to pressure, especially in a final like the one at Nationals 2010 where every competitor did their five solves in front of hundreds of people, the only one on stage in a silent and huge room. So the standard deviation approach doesn't do well. You can tell because your numbers give an average of 5 as varying LESS than a trimmed average of 5; this isn't true in real life as many competition averages are strongly hurt or helped by the removal of the best and worst times. So considering times to fall in a strict bell curve is an approach that may be easier but will not give a good model.

Second: people's time distributions will change depending on a number of factors, such as tiredness (either from cubing all day or from just waking up), being warmed up, nervousness, how much they've been practicing that puzzle in the past few days, what cube they're using, whether they are distracted, and so on. However, not everyone is affected the same way, and not everyone will have the same factors affecting their average in a given round anyway. Because of this it's not accurate to talk about someone's mean and SD as a constant throughout the competition.



Kirjava said:


> I'd rather that the person who set the best average at the comp won, no matter what round it was in.


This.

It's for the last reason that I don't think it's quite fair to only rank people by their final round average. Taking a mean of a few averages could be good, but some people might be very negatively affected in one of the rounds and that would basically negate their chance of winning even if they totally destroyed in other rounds. So I like the idea of ranking by the best average obtained in the competition. That would allow people to do their best when they feel best, rather than expecting everyone to do their best at a given point; it would make previous rounds matter, instead of just being a "get under <x> seconds" task which is trivial for someone who could win; and it would make sure that anyone who leaves the competition with a world record average (and is thus, officially, the best ever at that event) is guaranteed to also have won that event.


----------



## blade740 (Aug 10, 2010)

One thing i'd like to point out is that while cubers may get different scrambles in preliminary rounds, all finalists have the same scrambles. Using the best of 3 rounds might mean that an event with easy scrambles in only one group of the first round might disadvantage cubers who are otherwise faster. 

I really like the idea of using themean of your averages. The problem I stated still exists but is mitigated by averaging.

Ultimately, prizes of any sort, and the "winner" of a competition, are at the discretion of the organizers. I hope this will be taken into account in the future but I see no reason for a rule change.


----------



## blade740 (Aug 10, 2010)

One thing i'd like to point out is that while cubers may get different scrambles in preliminary rounds, all finalists have the same scrambles. Using the best of 3 rounds might mean that an event with easy scrambles in only one group of the first round might disadvantage cubers who are otherwise faster. 

I really like the idea of using themean of your averages. The problem I stated still exists but is mitigated by averaging.

Ultimately, prizes of any sort, and the "winner" of a competition, are at the discretion of the organizers. I hope this will be taken into account in the future but I see no reason for a rule change.


----------



## Ravi (Aug 10, 2010)

qqwref said:


> many competition averages are strongly hurt or helped by the removal of the best and worst times. So considering times to fall in a strict bell curve is an approach that may be easier but will not give a good model.



I think you are saying, to use the language of http://en.wikipedia.org/wiki/Kurtosis#Terminology_and_examples , that competition times tend to be leptokurtic (aka positive excess kurtosis): that most of the variance comes from occasional outliers rather than small, frequent variations. This is true to a point, but let's refer again to the statistics. I entered into Mathematica all of the statistics for everyone who made the finals (20 solves for each person, except for the two who DNFed earlier), and calculated kurtosis values, which are listed below in the order of final rankings:

1. Rowe Hessler -1.32
2. Phillip Espinoza -0.81
3. Andy Tsao -0.19
4. Andrew Kang +0.14
5. Michal Pleskowicz -1.18
6. Mitchell Stern -0.84
7. John Tamanas +0.59
8. Justin Adsuara +0.61
9. Weston Mizumoto +3.72
10. Patricia Li -0.71
11. Lucas Garron +2.72
12. Emily Wang -0.07
13. Erik Johnson +0.29
14. Timothy Sun +3.72
15. Dan Dzoan +11.23
16. Anthony Brooks +0.50

So nine of the sixteen did indeed have positive excess kurtosis as you predicted, while the other seven had negative excess kurtosis. For all of the four competitors with significantly positive (>1) excess kurtosis, it is easy to point to one or two solves which threw off the statistics massively--Dan's 34 in the final, Tim's 25 in the third round, Weston's 20 in the first round and 18 in the third, and Lucas's 22 in the first round--but which did not count in their respective averages. Thus it is only barely true that, as you argued, most real speedcubers' distributions in competition are more leptokurtic than a normal distribution, and that effect is almost entirely negated and even reversed in the process of trimming averages. Normal distributions aren't perfect models, but they're surprisingly accurate, especially when we're trimming the averages anyway.


----------



## incessantcheese (Aug 10, 2010)

i apologize that i'm entering into your extremely analytical thread with more of an opinion/feel thing but -- trying to get rid of the risk/statistical variance in determining the winner isn't always a good thing. i think it's far more exciting to have it be possible for any person who's reasonably close to a winning average time to have a brilliant stroke of luck and nerves and win a competition - rather than it be a grind of who most deserves it. of course this will lead to some "undeserved" or "unfair" wins, but i think the winner of an event is just that - whoever managed to get into the last round and set a great average when things mattered the most.


----------



## cmhardw (Aug 10, 2010)

Kirjava said:


> I'd rather that the person who set the best average at the comp won, no matter what round it was in.



I'm not convinced that this identifies the best cuber at the tournament better than some other proposed methods in this thread.

For Fridrich solvers there is a \( \frac{15263}{15552} \) probability of a full step LL solve assuming no special techniques are used during LL beyond OLL and PLL. So the chance of getting at least two solves in a trimmed 5 cube average that are some form of LL skip is:

\( 1 - \frac{15263}{15552}^5 - 5\times \frac{15263}{15552}^4\times \frac{289}{15552} \approx 0.0033 \) or roughly 1/3% chance to get such an average.

Now, if you're good enough to make it through 3 rounds of 3x3x3 then the chance that at least one of your averages has at least 2 skips is approximately:

\( 1-(1-0.0033)^3 \approx 0.0099 \)

So each cuber who makes it to the final, and gets to do 3 averages, has an approximately 1% chance that one of their 3 averages has at least two skips. With 16 competitors making it to the final, the probability that at least one of those competitors gets at least one average with at least two skips is:

\( 1-(1-0.0099)^{16} \approx 0.147 \)

So this competition has an approximately 15% chance that the winner wins by luck, and not necessarily by skill.

A major assumption I am making here is that all of the 16 people in the final are good enough that getting at least two skips in a trimmed 5 cube average gives them a significant enough advantage to outclass the other cubers who would also make it to the final and not get such an average. I think this assumption, even if it might not be completely true now, will not be unreasonable within 5 years.

Another assumption is, of course, that other methods will contribute to this chance as well - some possibly more than this and some possibly less. This also does not include the chance for F2L pair skips, and the fact that people use advanced techniques to increase their chance of a skip in the LL. The real percentage of competitions won by luck would, I propose, be higher than this estimated chance.

Chris


----------



## Pedro (Aug 10, 2010)

Yes, averages of 5 are not a really good indicator of people's skills. That's why we usually compare averages of 12 at home, since you have many many chances (as many as you want/can do) or even averages of 100.

But in competition you get at most 20 solves (usually 15 if you are a reasonably fast cuber) and giving people more solves would take too much time. Our competitions already take a lot of time (US Nationals was held over 4 days, but we usually have 2 and venue limitations and people who need to catch a flight and all that). Giving greater averages as rounds go on (5, then 7, then 9 solves or whatever) would be unfair to people who didn't qualify for, let's say, 0.01 and won't have the chance of a "better" measurement of their skills.

I agree with the point that, in competitions, it's about performing when it matters. In a World Cup of soccer, you can win all matches until the final by 5-0. If you lose the final by 0-1, you're not the champion. Maybe it wasn't "fair", but you knew that you should perform at that especific match and you won the other matches to qualify for that one. You don't get to play the final match if you fail to qualify, so who gets there made his way.

And since cubing, as other sports, takes into account pressure and nerves, I think it's fine to give the title to who performed best when they *had* to perform best.

The way it is now, people prepare themselves (or try to, at least) for the final, to do their best when it matters, so if we were to change the rules and use all solves, it would require a different preparation/way of thinking.

Also there is the problem that, for example, someone does really well in the 1st round but not so well in the 2nd, and doesn't qualify for the final. Suppose person A gets 10.00 and 14.00, respectively. Person B gets 11.00 and 12.00 and qualifies, but in the final he/she is nervous/tired/whatever and the average is 13.90. He/she will have a greater global average than person A, but he/she had more solves, so more chances to fail/succeed, and the comparison is not really valid in my view.

So how would we weight that? Who wins? Someone with a 12.00 average over 10 solves or someone with 12.01 average over 15 solves?


----------



## Ravi (Aug 10, 2010)

qqwref said:


> First: there is always a chance of a very nice time from a skip or a very bad time from a pop or similar problem. This even extends into averages as it is always possible in theory to get two DNFs (or bad times) or two skips (or very nice solves). Very bad times are more likely in a competition due to pressure, especially in a final like the one at Nationals 2010 where every competitor did their five solves in front of hundreds of people, the only one on stage in a silent and huge room.



True in theory, again, but let's look at the statistics. Throughout this year's US Nationals, four people were eliminated because of multiple DNFs:

-	one person with a DNF and four DNSes in the first round;
-	one person with 22, 24, 27, and two DNFs in the first round;
-	one person (Nicholas Sia) with 15, 16, 19, and two DNFs in the second round after a 14.23 average in the first; and
-	one person (Toby Mao) with 15, 16, and three DNFs in the second round after a 15.78 average in the first.

Of the 80 solves in the final round, zero were DNFs (although Dan Dzoan’s 34.21 might as well have been one). Of the 160 solves in the third round, there was one DNF (by 31st-placer Chris Dzoan), and a total of four sup-20 solves. Of the sixteen finalists, only Dan Dzoan (in the first round) and Mitchell Stern (in the second) had any DNFs throughout the competition. That’s two out of 320. Conclusion: DNFs really aren't as much of an issue as they are usually imagined to be.

As I argued previously, catastrophically slow times aren't a whole lot worse, especially when averages are trimmed. And, of course, lucky/unusually fast times are even less of an issue in general. Looking at the 318 solves collectively completed by the 16 finalists through the competition, we see that seven finalists suffered a single solve at some point which was over five seconds slower than their overall untrimmed average, but none had a solve over five seconds faster than average--the closest being Tim Sun's 9.40 compared to his 13.93 untrimmed (and therefore inflated by his third-round 25.33 solve) average.



qqwref said:


> your numbers give an average of 5 as varying LESS than a trimmed average of 5; this isn't true in real life



Well, you have a point. I'm beginning to wear myself out with all this statistical stuff (lol), but I'll let it suffice to say that real-life stats probably reflect your claim. Averages of 5 do vary slightly less than trimmed averages of 5 assuming perfect bell-curve distributions, presumably because the increased sample size (5 vs. 3) outweighs the likelihood of trimming a lopsidedly outlying time. However, this would probably change with the occasional twenties and thirties that happen in real competitions. There are very rarely two outliers in the same round, though--partly because of the rarity of the outliers, but also because people are more careful after having one--so using trimmed averages per round should fix nearly all major discrepancies. (This is part of the reason I suggested a mean of averages rather than a single trimmed average of 15 or 20.)


----------



## Dene (Aug 10, 2010)

I don't think it should be changed.
The first reason is that people strategically cube to ensure they get to the next round (already mentioned).
The second reason is that one can mess up, but still make it to the next round. In usnats about a 16s average would get you into round 2. What if Rowe screwed up with a 16s average? Or a 14.5 average in the second round, and a 14 average in the third round? He was on a bad solve streak. But we all know that he is the best 3x3 cuber in the US, and he won and I think we all agree that he clearly deserved to win. If we did it your way, he wouldn't stand a chance.
The third reason is that it would be inconsistent with other speed sports. If Usain Bolt doesn't perform in the olympic finals then he misses out.


----------



## Ravi (Aug 10, 2010)

qqwref said:


> Taking a mean of a few averages could be good, but some people might be very negatively affected in one of the rounds and that would basically negate their chance of winning even if they totally destroyed in other rounds.



It sounds to me like "very negatively affected in one of the rounds" is a euphemism for "not a consistently fast speedcuber." Am I missing something?



qqwref said:


> make sure that anyone who leaves the competition with a world record average (and is thus, officially, the best ever at that event) is guaranteed to also have won that event.



Personally, I have a little bit of scorn for world records--especially single solve records, but averages to an extent as well. Let's say there are two hypothetical speedcubers. One has been to dozens and dozens of competitions, and usually averages in the 11-12 range, but once had, say, two PLL skips in the same round which, combined with a streak of brilliance on the other two solves, gave him/her an 8.46 average and 6.70 single. The other hasn't competed in years, but shows up at one competition and averages 8.91, 9.10, and 9.02 in the three rounds, with a best single of 8.06. Which one would you say is the better speedcuber?

Yes, of course, averages don't generally vary that much. But they do vary. In my first post, I pointed out that taking averages reduces standard deviations (yes, assuming normal distributions) to 47.7% of those of single solves. Applying a "best of 3" format gives a coefficient of 74.8%; combining the two, we have a roughly 35.7% coefficient for "best of three averages of 5." This may be compared to 27.5% for "mean of three averages of 5" or 29.9% for "median of three averages of 5." (All of these are trimmed enough to remove the effects of nearly all outliers.) I also believe that (1) within reason, every solve should count, and count equally; and (2) best performance is less meaningful than average performance--I'm curious what the speedcubing community's consensus would be on these topics.


----------



## qqwref (Aug 10, 2010)

Ravi said:


> I think you are saying [...] that competition times tend to be leptokurtic [...]
> 
> So nine of the sixteen did indeed have positive excess kurtosis as you predicted, while the other seven had negative excess kurtosis. [...] Thus it is only barely true that, as you argued, most real speedcubers' distributions in competition are more leptokurtic than a normal distribution, and that effect is almost entirely negated and even reversed in the process of trimming averages.


You're overanalyzing. With such a small sample size (20 solves) you can't expect most of the people to get a really bad or a really good solve, and that would explain why many of the people had a moderately negative calculated kurtosis. Instead of just looking at how many people had a positive kurtosis, and saying that 9 of 16 is indistinguishable from 50%, what if we analyzed all 16 people together and took the average kurtosis? I get +1.15, supporting my conclusion that a typical person would have a higher-than-average kurtosis. My whole point was that outliers are more common than a normal distribution would imply (but still pretty rare in most cases) so you can't expect every sample of 20 solves to include one of them.



cmhardw said:


> With 16 competitors making it to the final, the probability that at least one of those competitors gets at least one average with at least two skips is [...] .147
> 
> So this competition has an approximately 15% chance that the winner wins by luck, and not necessarily by skill.


Interesting point. But there's always luck involved in cubing, and nobody has enough luck on a competitive event to win - or even place - without already having a considerable amount of skill. As Ravi pointed out, a significantly slower person still has a decent chance of winning if SDs are high enough; different scrambles will be of different difficulties when some competitors are opposite or fully color neutral and some even use other methods; OLL and PLL skips are almost completely random; and even veteran cubers can have hardware issues (or gravity issues!) that can spoil a time regardless of their skill.

Essentially, I don't think it is the goal of a competition to determine the _best_ cuber (mainly because this is really hard), but to determine the cuber who _delivered the best performance_. To me, this goal is best served by giving the top prize to the person who got the best average in any round. If you want to determine who is really objectively fastest I suggest the following:
- First round, every competitor does up to 5. Get rid of anyone who did not either achieve a counting single which was at least as good as the winning average, or come within (say) 2 seconds of it. The idea here is to prune it down to the people who could plausibly be the best.
- Have everyone left do 5 more solves. Now we have 10 per person.
- From now on, we use a trimmed average of every solve done so far with 1/5 of the solves (rounded down) trimmed off from each side. Cutoff the person with the slowest average, then do one more solve. Continue with that procedure until only one person is left, and that person is likely to be the best cuber there, in terms of average.


EDIT


Ravi said:


> It sounds to me like "very negatively affected in one of the rounds" is a euphemism for "not a consistently fast speedcuber." Am I missing something?


I don't mean that someone has a high SD or messes up a lot, I mean that someone is physically or mentally not in their best condition when a certain round takes place. I was once at a competition where the organizer forgot the scorecards where we were staying and a group of people (myself included) had to drive back and fetch them. When we got there we were pretty much yelled at for holding the competition up and had to do the 3x3 first round immediately. I was not warmed up, annoyed, and nervous, and of course I did badly (enough in fact to not make the next round). Are you going to say this is my fault for not being consistently fast? Cubers are people, not sources of statistical noise.



Ravi said:


> Personally, I have a little bit of scorn for world records--especially single solve records, but averages to an extent as well. Let's say there are two hypothetical speedcubers. One has been to dozens and dozens of competitions, and usually averages in the 11-12 range, but once had, say, two PLL skips in the same round which, combined with a streak of brilliance on the other two solves, gave him/her an 8.46 average and 6.70 single. The other hasn't competed in years, but shows up at one competition and averages 8.91, 9.10, and 9.02 in the three rounds, with a best single of 8.06. Which one would you say is the better speedcuber?


The second one is better. But _records are not about judging who is better_: they are about what the best recorded performance is. This is obvious for single solve records but I guess people don't realize it applies to average records as well. The exact same situation happens in every other situation where records are kept, and it is important to realize that asking "who is best" is a much more complicated question than asking "what is the best result in the database" - and that therefore determining the answer will require more work on your part.


----------



## Ravi (Aug 10, 2010)

Pedro said:


> And since cubing, as other sports, takes into account pressure and nerves, I think it's fine to give the title to who performed best when they *had* to perform best.



I am very uncomfortable with the idea that the winner of a competition should be based on anything except who solves Rubik's Cubes fastest.



Pedro said:


> Also there is the problem that, for example, someone does really well in the 1st round but not so well in the 2nd, and doesn't qualify for the final. Suppose person A gets 10.00 and 14.00, respectively. Person B gets 11.00 and 12.00 and qualifies, but in the final he/she is nervous/tired/whatever and the average is 13.90. He/she will have a greater global average than person A, but he/she had more solves, so more chances to fail/succeed, and the comparison is not really valid in my view.
> 
> So how would we weight that? Who wins? Someone with a 12.00 average over 10 solves or someone with 12.01 average over 15 solves?



I was thinking that everyone who didn't advance to the next round would be out of the tournament. Qualification for rounds could even be based on cumulative averages/means of averages/etc. I think that would make the situation you're talking about very unlikely, albeit possible I guess. I don't know. I'm open to suggestions other than means of averages (maybe longer final rounds?)--I just think that it's a bit of a joke to declare one speedcuber better than another based on a competition whose format furnishes such enormous margins of error in comparison to the differences between its competitors.


----------



## incessantcheese (Aug 10, 2010)

i think that's the main issue here. a few of us feel that the winner of a competition is the person who stood up there in front of everyone and performed in that situation, while you feel a need to find a statistically "best" person out of the competition and give first place to him. it's not saying that one cuber is better than another - that's what the records pages are for. i would argue that for a lot of people, the records page matters a lot more than individual tournament wins and losses, and the best average is always shown on the records page.


----------



## lilkdub503 (Aug 10, 2010)

Surprisingly, I don't agree with qq on this one. I think a direct comparison to the Athletics events in Track and Field apply here.

Let's say Usain Bolt gets a world record time in the semifinals of the 100m of the London Olympics in 2012. Say, 9.51 or something (Sh*t, I can't believe that's even feasible). He doesn't win the gold medal, he doesn't get jack-sh*t if he doesn't get the fastest time in the final round. Let's say Tyson Gay runs 9.58 in the final, and Bolt runs 9.71. Gay walks away with the gold, Bolt walks away with the silver. End of story, it was about who got the best time when it mattered, when the lights, camera, and pressure were on. If we model our events and competitions similarly to athletics, than why should we change how our winners are decided?


----------



## Ravi (Aug 10, 2010)

Dene said:


> What if Rowe screwed up with a 16s average? Or a 14.5 average in the second round, and a 14 average in the third round? He was on a bad solve streak. But we all know that he is the best 3x3 cuber in the US, and he won and I think we all agree that he clearly deserved to win. If we did it your way, he wouldn't stand a chance.



Frankly, this could hardly be further from the truth. If Rowe averaged 16, or 14, or anything like that, then that would be a strong indication to me that he didn't deserve to win. What makes him the best speedcuber in the nation is not that he was able to average 10.93 once or sub-11 twice, but that he averaged well under 12 seconds in every single round. In fact, if we did it my way, Rowe would have had a much easier road to victory: since everyone else had a less-than-spectacular round or two, he could have afforded to slip once or twice himself. (In fact, any of his averages could increase by 2.88 seconds before dropping into second place. This means a 13.65 average in the final, which is slower than his worst single solve in the entire competition.) Also, remember that he only won two of the four rounds. If the cards had fallen differently, the final might not have been one of them.

To put it another way, let's compare the top three under my mean-average system:
Rowe Hessler (11.72, 11.04, 10.98, 10.93 -> 11.165)
Michal Pleskowicz (11.21, 12.85, 11.41, 12.07 -> 11.885)
John Tamanas (11.75, 10.83, 12.31, 12.85 -> 11.935)
In terms of single solves, Rowe had a mean of 11.20 and an SD of 1.34; Michal had a mean of 12.08 and an SD of 1.05; and John had a mean of 12.09 and an SD of 1.54. According to the Mathematica work I discussed in my first post, Rowe had an 86.1% chance to beat Michal and an 82.0% chance to beat John under the current system, compared to 98.5% and 96.7% respectively under my system*. So no, it is not true that my system would screw the top competitors like Rowe. Quite the contrary.

Also, note that under the best-average-whenever system proposed of Kir, riffz and qqwref, John Tamanas would be our national champion by virtue of his 10.83 average in round 2. While John does deserve better than the 7th place he got, I think most of us agree that Rowe deserves to win, and the best-average-whenever system fails to see that.



* The coefficient for "mean of 4 avgs of 5" is 0.238, from Mathematica:
StandardDeviation[Table[Mean[Table[avg[RandomReal[NormalDistribution[0, 1], 5]], {i, 4}]], {i, 1000000}]]
The probabilities come from the following codes:
N[CDF[NormalDistribution[0, Sqrt[1.34^2 + 1.05^2]*0.4765], 0.88]] -> 0.861002
N[CDF[NormalDistribution[0, Sqrt[1.34^2 + 1.54^2]*0.4765], 0.89]] -> 0.819895
N[CDF[NormalDistribution[0, Sqrt[1.34^2 + 1.05^2]*0.238], 0.88]] -> 0.98507
N[CDF[NormalDistribution[0, Sqrt[1.34^2 + 1.54^2]*0.238], 0.89]] -> 0.966513


----------



## qqwref (Aug 10, 2010)

lilkdub503 said:


> Let's say Usain Bolt gets a world record time in the semifinals of the 100m of the London Olympics in 2012. Say, 9.51 or something (Sh*t, I can't believe that's even feasible). He doesn't win the gold medal, he doesn't get jack-sh*t if he doesn't get the fastest time in the final round. Let's say Tyson Gay runs 9.58 in the final, and Bolt runs 9.71. Gay walks away with the gold, Bolt walks away with the silver. End of story, it was about who got the best time when it mattered, when the lights, camera, and pressure were on. If we model our events and competitions similarly to athletics, than why should we change how our winners are decided?



But why should he try in the early rounds? What's the point? He can do a casual run and still make it. The thing about running is that it's a race, and the winner of the finals is important because it's visually obvious. You can watch the race and see them ahead. But cubing isn't a race, and you don't have such an obvious and intuitive way of judging who's best. Times are entered in later and a ranking decides the results. Keep in mind that the first and last competitors in the finals of Nationals 2010 went about an hour apart. The way it's set up in many cases, it's not "perform the best NOW" as much as "do a few averages, but we will ignore all but the last one".

Anyway, giving the win to the best average in the competition would introduce an interesting dynamic to the finals. You'd know what bar there was to break from the previous results. Right now my experience as a finals spectator is that you're watching some fast people do fast solves, but you don't really have the memory to figure out exactly who wins, so all you can do is watch and then wait for the data entry guy to calculate who won. Not too exciting unless an amazing average pops up. But if everyone was trying to beat a certain mark, I think it would be more interesting because you'd be watching people try to break that record, and it could be exciting just like watching a WR attempt. It becomes easy to pay attention to and keep track of.



Ravi said:


> I think most of us agree that Rowe deserves to win


Rowe should have won!


----------



## Ton (Aug 10, 2010)

Ravi said:


> I have long been of the opinion that a five-solve trimmed average measures a speedcuber’s skill too unreliably to decide the winner of a competition.



You forget a lot of other factors thats are decisive

1) A final is not a second or first round, the pressure on the top cubers is only when they enter a final. Hence a cuber that can take the pressure well has a better change
2) The scramble matters on the method you use -especially on a 2x2-, how slight the difference maybe. Currently a 0.01 can be decisive! 
3) In a final we often do one solve at the time compared to other rounds, these are other circumstance. Especially is you compare them with a qualification round or a first round 
4) In a final all competitors will have the same scramble, this is not true for the other rounds
... and there are more
These factors you did not take into account

So do you not convince me


----------



## Ravi (Aug 10, 2010)

qqwref said:


> With such a small sample size (20 solves) you can't expect most of the people to get a really bad or a really good solve, and that would explain why many of the people had a moderately negative calculated kurtosis. Instead of just looking at how many people had a positive kurtosis, and saying that 9 of 16 is indistinguishable from 50%, what if we analyzed all 16 people together and took the average kurtosis? I get +1.15, supporting my conclusion that a typical person would have a higher-than-average kurtosis. My whole point was that outliers are more common than a normal distribution would imply (but still pretty rare in most cases) so you can't expect every sample of 20 solves to include one of them.



It may not entirely make sense to calculate the kurtosis of a sample including solves by different people with different average times (and I'm not sure whether this would be more likely to increase or decrease kurtosis), but I'll concede the point. Large samples are more likely to have positive kurtosis, hence the necessity for trimming.



qqwref said:


> Ravi said:
> 
> 
> > It sounds to me like "very negatively affected in one of the rounds" is a euphemism for "not a consistently fast speedcuber." Am I missing something?
> ...



Not your fault, and this can of course happen in any round (and any sport). But I'm not sure that any averaging scheme can really fix that. My mean-average proposal is intended to average out any temporary changes as much as possible, but I think discarding all but the best individual average is an overreaction: see, for example, the closing remark in my last comment.


----------



## Dene (Aug 10, 2010)

Oh I forgot another point. What about people that only get to do one average? It would be inconsistent for the winner to have 4 averages averaged, and for people in the first round only to have one average. Unless this inconsistency can be justified, this cannot happen.


----------



## qqwref (Aug 10, 2010)

Dene said:


> Oh I forgot another point. What about people that only get to do one average? It would be inconsistent for the winner to have 4 averages averaged, and for people in the first round only to have one average. Unless this inconsistency can be justified, this cannot happen.



I think the idea is that only people who make the final are worthy of being ranked at all.

But if you rank by the best average achieved, everyone who competed in the event can be included in a single overall rank. I think this would be pretty cool.


----------



## riffz (Aug 10, 2010)

qqwref said:


> Dene said:
> 
> 
> > Oh I forgot another point. What about people that only get to do one average? It would be inconsistent for the winner to have 4 averages averaged, and for people in the first round only to have one average. Unless this inconsistency can be justified, this cannot happen.
> ...



Ew. That would discourage slower competitors from showing up to comps. Not a good idea. (I realize you aren't supporting this.)


----------



## nck (Aug 10, 2010)

What about a fourth round for the top five cubers from the third round to do averages of 12? Or even for the top 3cubers if time is an issue.


----------



## gasmus (Aug 10, 2010)

I agree an average of 5 is not completely fair but i dont think there is a better option. i would love if it was the best average of the whole competition as the winning average but that would only really be fair if everyone competed in every round(or at least the same number of people in the 2nd round onwards).

A mean of averages may be more accurate and fair but in my opinion would change how a lot of people compete, too much for what its worth. in my case(and i'm sure many other cases) i always save my energy for the finals, or maybe go for fast singles in the first and second rounds. with a mean of averages i would go for more consistent averages which in my opinion is much less fun. i much prefer the idea of everything relying on the finals. if any changes were to be made i think it should be to the finals only.

The most fair change i could think of would be best of 2(or more^^) averages of 5 for the finalists, though i'm sure that has its problems too.

btw, i dont think naming names is fair in this case since almost every competition doesnt have a clear winner. Both mine and Rowe's averages were all full step completely non-lucky solves so luck didnt come into it. the finals are just when it counts.


----------



## Mitch15 (Aug 10, 2010)

ill admit i didn't read all the extremely long posts of this thread, but i agree with garron about different rounds carrying different goals and meaning. and i definitely agree with separate rounds and a final round deciding all. that's how competitions work.

would you decide wimbledon on how who had the best set win ratio of the competition?
of course not. now i know this isn't a 1v1 situation. but there is a lot to be said about a final where you have no laurels to rest on and everything rests on your next 5 solves. cubing under pressure is a big part of who is a better cuber.


----------



## qqwref (Aug 10, 2010)

Mitch15 said:


> would you decide wimbledon on how who had the best set win ratio of the competition?



Should the World Series teams be decided by who got the most points in their most recent game? Not all sports work the same, or should. Tennis is single elimination anyway, but many sports (including cubing) are not optimally run in such a directly competitive way.


----------



## macky (Aug 10, 2010)

Performing when it counts is of course important. The problem with comparisons to other sports is that speedcubing inherently involves chance. I don't know how much of a top cuber's time variation is from pure chance, but it's considerable. Even when performing the very best, a top CFOPer's times will vary > 0.5 second just by the difficulty of last layer cases--and we aren't even considering F2L. With the continuing clustering of times near the top, this variation is starting to become enough to significantly affect the ranking in a competition.

I thought about all this back in WC07 and with my prayer before WC09. Because of complications such as those mentioned in this thread, I'm fine having the winner of the last round be the winner of the competition. Or if an organizer feels strongly, he can use the fastest average or the mean of averages. In any case, there's no rule change needed.

The winner of a competition is the cuber who performed "the best" (in a predefined sense) at that competition, not necessarily the best cuber. I therefore don't see too much need to change the competition format so that the best cuber has a better chance of winning the competition.* What I've suggested instead since 2007 is an official yearly overall ranking (see this thread for an example).

* But I'm sure people will soon be needing my prayer more often!


----------



## qqwref (Aug 10, 2010)

Oh, here's another small reason to rank people in a competition by the best average they got: it would mirror the format of the WCA rankings (which are also best average, not most recent average) which as we all know are more important than any individual competition's results (except perhaps Worlds).


----------



## reThinking the Cube (Aug 11, 2010)

Ravi said:


> Then any speedcuber can be represented essentially by two numbers: his or her mean and standard deviation. The purpose of taking averages, of course, is to decrease the standard deviation while preserving the mean. The question: How much do standard deviations decrease when averages are used?



*Can you calculate comparisons using 15 solves, with the fastest (2) and slowest (3) solves discarded - leaving 10 for a competition trimmed average?* 

I like the idea of using the median average of 10 taken from 15 solves, even better than taking the best trimmed average, or a median average from 3 individual rounds of 5 solves. 



Kirjava said:


> I'd rather that the person who set the best average at the comp won, no matter what round it was in.



If the best average is going to win, then we should make it the ONLY average (all solves factoring into 1 competition average). By starting over each round, we make some results not as important as others, but the way things are formatted now, Lucas is correct:



Lucas Garron said:


> An important fact to consider: In the earlier rounds of a tough competition, cubers often try to make it to the next round rather than a fast average.
> Two DNFs are unacceptable, adding 2 seconds to your average can be. In the finals, however, it's about the maximum speed and all its risks.
> 
> 
> ...



Possible Solution: double trim on 5th solve leaving a single median value as the basis to eliminate early. From that point on, averages (and competition rank) can be computed based on the averages of the median values gotten so far, after double trimming. This will prevent early elimination of serious competitor, unless they DNF in 3 out of the 5 attempts, and in that case, they deserve to be eliminated! 


qqwref said:


> Ravi, there are two important factors that you are not considering. First: there is always a chance of a very nice time from a skip or a very bad time from a pop or similar problem. This even extends into averages as it is always possible in theory to get two DNFs (or bad times) or two skips (or very nice solves). Very bad times are more likely in a competition due to pressure, especially in a final like the one at Nationals 2010 where every competitor did their five solves in front of hundreds of people, the only one on stage in a silent and huge room. So the standard deviation approach doesn't do well. *You can tell because your numbers give an average of 5 as varying LESS than a trimmed average of 5; this isn't true in real life as many competition averages are strongly hurt or helped by the removal of the best and worst times. *So considering times to fall in a strict bell curve is an approach that may be easier but will not give a good model.





Ravi said:


> I entered into Mathematica all of the statistics for everyone who made the finals (20 solves for each person, except for the two who DNFed earlier), and calculated kurtosis values, which are listed below in the order of final rankings: ...<snip>
> 
> *Thus it is only barely true that, as you argued, most real speedcubers' distributions in competition are more leptokurtic than a normal distribution, and that effect is almost entirely negated and even reversed in the process of trimming averages.* Normal distributions aren't perfect models, but they're surprisingly accurate, especially when we're trimming the averages anyway.



*What would the kurtosis values be for those competitors if using all 15 solves, the slowest 3, and fastest 2 solves were trimmed out first?*



cmhardw said:


> So each cuber who makes it to the final, and gets to do 3 averages, has an approximately 1% chance that one of their 3 averages has at least two skips. With 16 competitors making it to the final, the probability that at least one of those competitors gets at least one average with at least two skips is:
> 
> \( 1-(1-0.0099)^{16} \approx 0.147 \)
> 
> So this competition has an approximately 15% chance that the winner wins by luck, and not necessarily by skill.



*How does double trimming (as described above) on a set of 15 solves compare with this?*


qqwref said:


> Anyway, giving the win to the best average in the competition would introduce an interesting dynamic to the finals. You'd know what bar there was to break from the previous results. Right now my experience as a finals spectator is that you're watching some fast people do fast solves, but you don't really have the memory to figure out exactly who wins, so all you can do is watch and then wait for the data entry guy to calculate who won. Not too exciting unless an amazing average pops up. But if everyone was trying to beat a certain mark, I think it would be more interesting because you'd be watching people try to break that record, and it could be exciting just like watching a WR attempt. It becomes easy to pay attention to and keep track of.



Excellent point. This would be perfect if it incorporated all the solves, instead of just using the results from the best *lucky/easy* round.


----------



## Tim Major (Aug 11, 2010)

reThinking the Cube said:


> Kirjava said:
> 
> 
> > I'd rather that the person who set the best average at the comp won, no matter what round it was in.
> ...


I agree with Kir, however, I don't think your solution is feasible due to practical reasons. Do you really think that would work in competitions? Why make a simple thing complex? What Kir is suggests is great I think, however, in Football/Baseball/Basketball/etc, even if team 1 beats team 2 sometime throughout the season, if team 2 wins in the final, then team 2 wins.
Whilst I agree with Kir, I think at the same time, it's more exciting, to have a final, that actually matters, and to have a winner determined by the winner of the final.
What are you going to do, if there is more than one group/heat in the earlier stage of an event. That would be unfair. If you have a cuber who averages 10, Vs. another cuber averaging 10, but they're in different groups/heats, maybe one of the two will be unfairly advantaged, by having an easier set of scrambles (I'm not saying this will always happen). If you're going to have whoever gets the best average throughout the whole competition wins, then there can't be separate groups/heats. 
If a long jumper trips in all their runs on the day of the Olympics, they don't win if they had a better jump at a previous official competition.


Spoiler



I've convinced my self by these arguments, that I think whoever wins in the final, wins overall  Sure sometimes "unfair" but not really. Such is life, and life isn't always fair.
Often in the final, there is way more pressure than the other rounds, and that's part of winning, you need to be able to hold up under pressure.


----------



## qqwref (Aug 11, 2010)

ZB_FTW!!! said:


> If a long jumper trips in all their runs on the day of the Olympics, they don't win if they had a better jump at a previous official competition.



You need at least ONE good result in the competition...

(You did describe how the official overall rankings work, though  And I'm a fan of those. I like the idea of trying to improve your personal best a lot more than the idea of trying to beat everyone else on these particular scrambles.)


----------



## Gavin (Aug 11, 2010)

Lucas Garron said:


> An important fact to consider: In the earlier rounds of a tough competition, cubers often try to make it to the next round rather than a fast average.
> Two DNFs are unacceptable, adding 2 seconds to your average can be.
> 
> In the finals, however, it's about the maximum speed and all its risks.



I think this pretty much hits the nail on the head. At Nationals, I remember someone (might have been Rowe) saying that the first round average didn't really matter as long as they got into the next round. Now you could either think of it this way, or you could think of it as every round I have a chance to get a new WR average. I think at Nats though, more people were focused on winning the competition.


----------



## Olivér Perge (Aug 11, 2010)

I see no problem with the current format. The final round counts, so do your best at it! If you are going for a good single, saving yourself or just practicing in the former rounds, that's up to you. 

In my opinion it has never worked really well (at least for me). Like does anyone have a story when she/he wanted to go for a good single/average and got exactly what she/he wanted, not more and not less? I think expecting times is, what really killes the official solves. Overthinking is really bad for competing, in my opinion. (Two examples: Breandan at the world chamipionship: no one expected him to win, including himself, many expected Tomasz to win. At Austrian Open I was talking with Stefan Huber and he told me he wants a 10.5-ish average. Well, it didn't work out quite well. )

I don't like the idea of making the best average in the competition the winner average.


----------



## Kirjava (Aug 11, 2010)

Anyone who thinks that someone setting three averages of 14 is better than someone setting averages of 9, 25 and 9 is extremely naive.


----------



## reThinking the Cube (Aug 12, 2010)

Kirjava said:


> Anyone who thinks that someone setting three averages of 14 is better than someone setting averages of 9, 25 and 9 is extremely naive.



Anyone who thinks that someone setting averages of 9, 25 and 25 is better than someone setting averages of 10, 10 and 10 is extremely naive.


----------



## Kirjava (Aug 12, 2010)

I disagree. The former competitor would be ranked higher than the latter.


----------



## reThinking the Cube (Aug 12, 2010)

reThinking the Cube said:


> Kirjava said:
> 
> 
> > Anyone who thinks that someone setting three averages of 14 is better than someone setting averages of 9, 25 and 9 is extremely naive.
> ...





Kirjava said:


> I disagree. The former competitor *would* be ranked higher than the latter.



I disagree. The latter competitor *should* be ranked higher than the former.


----------



## ExoCorsair (Aug 12, 2010)

Why did the numbers change from 14 to 10?


----------



## PhillipEspinoza (Aug 12, 2010)

I think the point is that you do it when it counts. There are many people who do good at home but can't really do that good in competition. For one competition, right before the final round, I was practicing with Chris Dzoan and got two 8-second times in a row. I was called up shortly after that, competed, and received a DNF average which resulted in Michael Gottlieb winning the competition with a 13.67 average. This is when I averaged around high-10, low-11 seconds.

The point is you gotta do it when it counts.

Oh and I think in a competition we're measuring speed and not so much consistency. It's not the person with the lowest SD who should win but the person with the fastest average.


----------



## Kirjava (Aug 12, 2010)

reThinking the Cube said:


> I disagree. The latter competitor *should* be ranked higher than the former.




Have you literally lost your ****ing mind?

Please stop trolling.


----------



## reThinking the Cube (Aug 12, 2010)

Kirjava said:


> reThinking the Cube said:
> 
> 
> > I disagree. The latter competitor *should* be ranked higher than the former.
> ...



That's very funny, I was thinking the same thing about you. Your use of some feeble-minded namecalling and *****ing reveals some things already quite lost.

What don't you get about a 10 sec. final round average beating a 25 sec. final round average? Which competitor will get the better rank for that event? Sorry to make you look so *******, but that 9sec. result in an earlier round with the current format, would obviously not affect the final rankings for that event.

9,25,25 > 10,10,10

Now, PLEASE stop *condoring*. I think just posting a picture of your most recent attempts at self-mutilation, would suffice.


----------



## cmhardw (Aug 12, 2010)

Please keep this discussion civil, or we will resort to infractions and/or temp-bans.

Chris


----------



## Kirjava (Aug 12, 2010)

reThinking the Cube said:


> final rankings




WCA ranking != final ranking



reThinking the Cube said:


> Now, PLEASE stop *condoring*




Could you define this for me? It doesn't appear to be a word.



reThinking the Cube said:


> I think just posting a picture of your most recent attempts at self-mutilation, would suffice.




Aside from the technique you've been using in this thread where you copy the post someone made with slight changes to it, this is one of the stranger things you do...


----------



## qqwref (Aug 12, 2010)

reThinking the Cube said:


> Kirjava said:
> 
> 
> > [...]The former competitor *would* be ranked higher than the latter.
> ...


You can't disagree. That's the way the overall rankings are set up.



PhillipEspinoza said:


> I was called up shortly after that, competed, and received a DNF average which resulted in Michael Gottlieb winning the competition with a 13.67 average.


lol, sorry

Things like this would be much much less likely if we used the 'best average in any round wins' style. I think it is more accurate to say "A is better than B because A got a faster average in this competition" than "A is better than B because B got two DNFs in five consecutive solves". (Is it a good thing to let someone who is clearly not the fastest solver there have a non-negligible chance of winning? I am not sure what I think.)



reThinking the Cube said:


> That's very funny, I was thinking the same thing about you. Your use of some feeble-minded namecalling and *****ing reveals some things already quite lost.


You are a surprisingly devoted hypocrite.



reThinking the Cube said:


> What don't you get about a 10 sec. final round average beating a 25 sec. final round average? Which competitor will get the better rank for that event? Sorry to make you look so *******, but that 9sec. result in an earlier round with the current format, would obviously not affect the final rankings for that event.
> 
> 9,25,25 > 10,10,10


This whole topic is about which result should be considered better. As you've somehow managed to figure out, a 25 sec. final round average would indeed lose under current rules. However, the question of whether someone who gets 9,25,25 is better than someone who gets 10,10,10 (and whether they should be ranked better in a given competition and overall) is nontrivial. This is especially true since it is a LOT easier in cubing for someone who is normally around 10 seconds to get a 25 second avg5 than for someone who is normally around 25 seconds to get a 10 second avg5. Now that you know this, the discussion should make more sense.


----------



## reThinking the Cube (Aug 13, 2010)

PhillipEspinoza said:


> I think the point is that you do it when it counts.



Good point. Currently, 5-solve trimmed average for the final round is arbitrarily defined as what *counts*. In 1982 only 1 solve counted. WCA can make whatever it wants count. Kirjava, and qqwref advocate making the best average in ANY round *count*. I really don't have a problem with that either, but if all the solves where treated as one round, and a 15-solve double trimmed average was used to decide the winner, then ALL the solves would *count*. 



qqwref said:


> reThinking the Cube said:
> 
> 
> > Kirjava said:
> ...



LOL. We are discussing a format change that would potentially affect the rankings - are we not? Reread it in the original post, and notice *would* and *should* are 2 different words. There is also the difference between overall WCA rankings, and the rankings for any single tournament event. Now that you know this, the discussion should make more sense.



qqwref said:


> PhillipEspinoza said:
> 
> 
> > I was called up shortly after that, competed, and received a DNF average which resulted in Michael Gottlieb winning the competition with a 13.67 average.
> ...



@qqwref/kirjava: The key thing to realize is that the serious competitors of a major comp. are now too close in ability for any one 5-solve trimmed to decide it fairly. I think this has been presented rather convincingly in some earlier posts. Whether or not tournament results should be, or need to be fair, is another debate altogether, but there is about as much aberation in results for final round now as there is for a 1-solve only for Round#1 elimination, which isn't done for that same reason. Using the best average from any round would be an improvement, but it is (IMO) not as good as using all 15 solves with double trimming. 

BTW - Phillip would have won (Bayview Hills Open 2010 3x3x3)http://www.worldcubeassociation.org/results/c.php?allResults=All+Results&competitionId=BayviewHills2010 if any of these proposed formats were used. By trimming 2 best and 3 worst times - Sikan Li would have made it to the final round, and Michael Gottlieb would have placed 5th overall in my book, instead of getting the win.



qqwref said:


> This whole topic is about which result should be considered better. As you've somehow managed to figure out, a 25 sec. final round average would indeed lose under current rules. However, the question of whether someone who gets 9,25,25 is better than someone who gets 10,10,10 (and whether they should be ranked better in a given competition and overall) is nontrivial. This is especially true since it is a LOT easier in cubing for someone who is normally around 10 seconds to get a 25 second avg5 than for someone who is normally around 25 seconds to get a 10 second avg5.



No ****Sherlock! What have I actually been suggesting as a nontrivial improvement for determining which result(s) should be considered better?


----------



## hr.mohr (Aug 13, 2010)

Guys how can a discussion about statistics and competition format result in such tone?

There is a few things that got lost in this discussion and that's the other properties in the current format that makes organizing competitions more manageable. 

Having more attempts takes more time and are imho less exciting for spectators to watch. 

Also in the arguments about performing when it counts there is a difference. There is much more pressure when you know that the next few solves determine if you go thru to the next round or win the competition. Taking the best average or the mean of averages will not induce the same pressure because the next solve does not win or loose a competition. If your first rounds was much better than your main competitor then the last round should only be on par with the other guys result. Less pressure and less excitement for spectators.

Fairness is also a tricky thing to discuss. The current format only favor one cuber over the other in the first rounds because of the different scrambles. The final are fair as all competitors get the same scramble. While a shorter average enables cubers to perform much better than larger averages, it's still fair to determine a winner because all competitors get the same terms. I think that some people see competition differently. I see it as a chance to feel pressure and do my best when it matters and some see it as a way of determine the best overall. I find that pretty boring. I like that there is chance of big upsets. I like when you can discuss results from previous rounds and have thought experiments about who could have won instead.


----------



## reThinking the Cube (Aug 13, 2010)

hr.mohr said:


> Guys how can a discussion about statistics and competition format result in such tone?



All right, you have earned my utmost respect.



hr.mohr said:


> There is a few things that got lost in this discussion and that's the other properties in the current format that makes organizing competitions more manageable.
> 
> Having more attempts takes more time and are imho less exciting for spectators to watch.



This is true, but I don't recall anyone suggesting increasing the number of attempts.



hr.mohr said:


> Also in the arguments about performing when it counts there is a difference. There is much more pressure when you know that the next few solves determine if you go thru to the next round or win the competition.



Yes, this is a very valid consideration. Trimmed running averages can also keep the pressure on, but using the best average of any round might not, since the result might have already been clinched.



hr.mohr said:


> Fairness is also a tricky thing to discuss. The current format only favor one cuber over the other in the first rounds because of the different scrambles. The final are fair as all competitors get the same scramble. While a shorter average enables cubers to perform much better than larger averages, it's still fair to determine a winner because all competitors get the same terms.



Well said. This (scramble equity) is one of the key objections to using the best average for any round format, but this shouldn't be much of a problem with a trimmed running average.



hr.mohr said:


> I think that some people see competition differently. I see it as a chance to feel pressure and do my best when it matters and some see it as a way of determine the best overall. I find that pretty boring. I like that there is chance of big upsets. I like when you can discuss results from previous rounds and have thought experiments about who could have won instead.



I can relate to this pressure/challenge/chance aspect. Using a trimmed running average should place that aspect on every solve.


----------



## Kirjava (Aug 13, 2010)

reThinking the Cube said:


> but if all the solves where treated as one round, and a 15-solve double trimmed average was used to decide the winner, then ALL the solves would *count*.




This will never happen.



reThinking the Cube said:


> qqwref said:
> 
> 
> > reThinking the Cube said:
> ...




A 9 average will currently be ranked higher than a 10 average. This is a fact you cannot disagree with.

You're still doing that childish thing where you mirror other people's posts.

Also, I'd still love to know what you think the word 'condoring' means.


----------



## Pedro (Aug 13, 2010)

reThinking the Cube said:


> hr.mohr said:
> 
> 
> > Fairness is also a tricky thing to discuss. The current format only favor one cuber over the other in the first rounds because of the different scrambles. The final are fair as all competitors get the same scramble. While a shorter average enables cubers to perform much better than larger averages, it's still fair to determine a winner because all competitors get the same terms.
> ...



Not really. Easy scrambles may affect your average by a big margin, and someone who didn't get the same scrambles but has about the same speed would be much slower than you.

This is the main problem I see with couting every solve (or 10 out of 15 or whatever) or the best average of the competition: different scrambles. I know that for the top people, the scramble doesn't matter that much, but as Macky said, bad LL or F2L cases can slow you down 0.5 or so, and in a world where sometimes competitors are 0.05 apart, that's gotta be considered.

EDIT:

Just to illustrate:

Suppose 3 rounds.
Competitor A and B compete in different groups in the first 2 rounds.

1st round:
A does 10,11,10,(11),(9) ("easy" scrambles)
B does 11,12,12,(11),(12) (not that easy)

2nd round:
A: (9),10,10,(11),10 (easy again)
B: (10),11,11,(12),10 (not so easy)

Final: (all in the same group)
A: (10),10,11,12,(12)
B: (10),11,10,11,(12)

In the current format, B wins (10.67 x 11.00).

If we took the best average, A wins (10.00 avg on 2nd round)

If we average all of them (13 out of 15), A gets 10.38 avg and B gets 11.08 avg.

Suppose they were in the same group for the 2nd round and A kept the same results, but B got
B: 10,10,(9),10,(11)

Now B's average is 10.85. 

My conclusion (not claiming it's the absolut thruth):
First round scrambles were so easy that A got much better than what he actually is. B seems to be faster to me (two 10.00 x 10.00 and 10.67 x 11.00 when they had the same scrambles).


----------



## cmhardw (Aug 13, 2010)

Guys there is one other thing to bring up here. If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds? I can tell you that at some of the smaller competitions, with fewer judges, and also at the much larger competitions, with MANY competitors, it will not be feasible to give every competitor 15 solves. Someone, and in fact many people, *must* be eliminated at some point to make the running of the competition feasible. This would mean that not every competitor gets 15 solves, and yet you must still fairly rank the ones that do not get all solves.

I know that, from a theory standpoint, everyone getting all 15 solves is best. But, try giving every competitor at a small competition 15 solves when you're already short on judges for the current format. It will not be easily done without either eliminating other events or running over time.

Chris


----------



## CubesOfTheWorld (Aug 13, 2010)

I think that the winner should be decided by whoever had the best average in any of the rounds. There should still be elimination to move onto the next round.


----------



## Daniel Wu (Aug 13, 2010)

If the winner is decided by the best average, then it would seem that it would really take away from the entire concept of the finals.


----------



## riffz (Aug 13, 2010)

rickcube said:


> If the winner is decided by the best average, then it would seem that it would really take away from the entire concept of the finals.



I disagree. By the time finals roll around, each competitor knows the mark that they need to aim for to jump into 1st place. It's comparable to watching olympic speed skating, where you know the current first place time, and I still think that's very exciting to watch.

Each competitor also knows what they averaged in previous rounds, so if one of their previous averages was already fast, they can really put it on the line and go all out, instead of holding back a bit because they want to secure a spot on the podium.

And let's not forget that even if a really fast average has been set in previous rounds that no one feels they have a chance of beating in the finals, people are still striving to beat their PBs. I think most people would agree that they care more about their WCA ranking in an event than how they placed at a specific competition.


----------



## qqwref (Aug 13, 2010)

reThinking the Cube said:


> LOL. We are discussing a format change that would potentially affect the rankings - are we not?


We are not. This would only affect the rankings in individual competitions, which will also affect who becomes a national/regional/world champion.



reThinking the Cube said:


> There is also the difference between overall WCA rankings, and the rankings for any single tournament event.


I know Kirjava well enough to be certain of what he meant in his post; this is one advantage of making friends on the forum. Either you've misunderstood his post, or you misinterpreted the nature of the discussion (see the above).



reThinking the Cube said:


> The key thing to realize is that the serious competitors of a major comp. are now too close in ability for any one 5-solve trimmed to decide it fairly.


Completely false. I watched the finals of Nationals 2010 and it was clear that only two or three competitors had the speed and calm to even be capable of matching or beating Rowe's average, and he was clearly better than they were. I have already stated that I think factors such as these exert much more of an effect than pure randomness along the lines of a normal distribution. Because the finals feature the same scrambles and very similar methods, this is even more the case than it would be in a general situation.



reThinking the Cube said:


> No ****Sherlock! What have I actually been suggesting as a nontrivial improvement for determining which result(s) should be considered better?


Do you know what I meant by "nontrivial"? The point of my 3rd sentence was that the question of which of those two competitors is better is not at all obvious (nontrivial, that is) and is more worthy of debate than you seem to think from your previous post. The alternatives (best round, final round, trimmed overall average, untrimmed overall average, average of averages) have been suggested and now we are on the stage of debating which provides better results.



hr.mohr said:


> Guys how can a discussion about statistics and competition format result in such tone?


When any participant is the type who might launch into a profanity-filled rant when his posts are criticized. Just sayin'.



reThinking the Cube said:


> This (scramble equity) is one of the key objections to using the best average for any round format, but this shouldn't be much of a problem with a trimmed running average.


This could be a good point, but - why not? People can still get different scrambles in almost all of the competition. Why are you concerned about an avg5 with all different scrambles, but not about an avg15 with 10 different scrambles? Anyway, I still don't think scrambles matter as much as human factors, unless at least two extremely easy scrambles pop up in one round.



cmhardw said:


> If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?


I imagine that you would need many different setups depending on the number of solves that have been done so far, whether it be 5, 10, 15, or something odd like 2 or 7. These trimmed averages of N would then be used for deciding who makes the next round. I think it's safe to say that 15 is just an example - not all competitions have 3 rounds, and if we're going to be doing an overall average we don't want to make up numbers or discard solves.



rickcube said:


> If the winner is decided by the best average, then it would seem that it would really take away from the entire concept of the finals.


Suppose Rowe goes up first in the finals and gets the best average so far in the competition. Now the situation is exactly the same as in the best-average-ever scenario, at least for everyone but Rowe. Would that take away from the concept of the finals?

If we used the best-average-ever condition, finals would mean the last opportunity to improve your rank in the competition. This would run parallel to the idea that the finals is the last opportunity to improve your rank in the world (which is how I treat it since I care more about the world rank than about the rank at any individual competition except Nationals or Worlds).


----------



## reThinking the Cube (Aug 14, 2010)

cmhardw said:


> Guys there is one other thing to bring up here. If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?



Here is a format that can easily and fairly determine which competitors deserve to advance to the next round, but doesn't eliminate strong competitors early, AND be implemented without placing additional burden on organizers, uses all 15 solves to better decide the winner, generates official WCA rank times for each round, and is more spectator/competitor friendly than the current format.

All(3) rounds of 5 are done exactly the same as the current format. The 5-solve rounds are trimmed and averaged the same, to produce an official WCA time for that round for that event. Round#1 is 5 solves and averaged just like it is now.

So the question now is - Who gets to advance to Round#2?

The current format is to use the Round#1 times as the basis for elimination. This leaves open the possibility of a strong cuber getting more than 1 bad result in the first round and not advancing.

Rather than using this official time as the basis for deciding who advances, I really like the idea of temporarily double trimming out the 2 highest and 2 lowest times to give just the median time for that round. For all the tournaments that I have looked at, this median value has generated a fairer selection than using the official 5-solve round time. The main reason for this being the chance that a serious competitor could get eliminated in the first round by simply getting more than 1 bad solve. If the median value is used instead, it would take 3 bad solves out of 5 to prevent that competitor from advancing. So the competitors could easily be ranked(for cutoff placement) and eliminated based on this time. This median value is only used for determining advancement into Round#2 and is not needed for anything other than that. 

Round#2 is also 5 solves and done exactly like it is now, and will produce another official WCA result for the competitors that get to do it.

So the question now is - Who gets to advance to Round#3?

The current format is to use the Round#2 times as the basis for elimination. This leaves open the possibility of a strong cuber(Sikan Li) getting more than 1 bad result in the second round and not advancing.

Rather than using this official time as the basis for deciding who advances, I really,really like the idea of temporarily trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. For all the tournaments that I have looked at, this median average value has generated a much fairer selection than using just the official 5-solve round#2 time. The main reason for this being the chance that a serious competitor could get eliminated in the second round by simply getting more than 1 bad solve in that round. If this median average value is used instead, it would take 4 or more very bad solves out of 10 to prevent that competitor from advancing to the final round. So the competitors could easily be ranked(for cutoff placement) accordingly, and eliminated based on this time. This median value is only used for determining advancement into Round#3 and is not needed after that. 

Round#3 is also 5 solves and done exactly like it is now, and will produce another official WCA result for the competitors that get to do it.

So the question now is - Who should win?

The current format is to use the Round#3 times as the sole basis for determining the winner. This leaves open the very real possibility of the strongest cubers getting more than 1 bad solve out of these last 5, and ruining their tournament placement because of it. Instead, I really,really,really like the idea of trimming out the 3 highest and 3 lowest times from all 15 solves, and then averaging the 9 median times that are left to produce a tournament time for that event. For all the tournaments that I have looked at, this value has produced by far, the fairest placement rankings. Trimming more or less than 3/3 times from the 15 is also possible, but doing it this way, keeps it consistent with the current 40% trim.


----------



## cmhardw (Aug 14, 2010)

reThinking the Cube said:


> cmhardw said:
> 
> 
> > Guys there is one other thing to bring up here. If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?
> ...



Wow... I have to admit, that is a pretty cool way to do things. Perhaps a hybrid of this and the usual trimmed average of 5 could be used as well?

Yeah, I do like this idea. I think this, or slight variations of it, could stand a real chance of being an effective way to run a competition that is also (possibly?) more fair. Anyone else have any comments?

Chris


----------



## Faz (Aug 14, 2010)

So, if this were to be implemented, this would force all competitions to have 3 rounds of 3x3 would it not? - Sometimes, due to lack of competitors, or time restrictions, this isn't possible. Out of 4 competitions I have been to, only one of them has had 3 rounds of 3x3. What about for other events? Would the winner of 4x4 be determined by the median time?


----------



## hawkmp4 (Aug 14, 2010)

reThinking the Cube said:


> Rather than using this official time as the basis for deciding who advances, I really,really like the idea of temporarily trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. For all the tournaments that I have looked at, this median average value has generated a much fairer selection than using just the official 5-solve round#2 time. The main reason for this being the chance that a serious competitor could get eliminated in the second round by simply getting more than 1 bad solve in that round. If this median average value is used instead, it would take 4 or more very bad solves out of 10 to prevent that competitor from advancing to the final round. So the competitors could easily be ranked(for cutoff placement) accordingly, and eliminated based on this time. This median value is only used for determining advancement into Round#3 and is not needed after that.


Why trim the 3 lowest and 3 highest? Why not just one? Why not take the median of all the solves? It just seems arbitrary.
Also, how would you propose to run competitions where only 2 rounds of 3x3 are possible?


----------



## qqwref (Aug 14, 2010)

reThinking the Cube said:


> Here is a format [...]
> 
> Round#1 [...] just the median time for that round. [...]
> Round#2 [...] trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. [...]
> Round#3 [...] trimming out the 3 highest and 3 lowest times from all 15 solves, and then averaging the 9 median times that are left to produce a tournament time for that event.


OK, but why do you choose to do trimming in exactly this way? Doesn't it seem a bit arbitrary? And what would we do if we have 20 solves to choose from (4 rounds)? What if the first round is a qualification and there are only 2 solves there, so instead of 10 after the second round we have 7? (This has happened.)



fazrulz said:


> What about for other events? Would the winner of 4x4 be determined by the median time?


Good point. I don't think a trimmed global average is a good way to do all events. For 4x4 or Square-1, for instance, if parity always gives you a slower time, you will then have to be lucky to get 3 non-parity solves rather than 2 (in which case your median will be a parity solve and thus slow). I'm just talking about the first round, but of course few comps will have a second.


Here is something interesting: I think that, as a cube gets bigger, the quality of a solve starts to matter a lot more than how easy it is. Essentially, there will be far more variation due to turnspeed, lookahead, lockups, tired wrists, and concentration than due to scramble difficulty or random case factors. So, for large cubes with multiple rounds in a competition, I think that taking the mean average or some kind of overall trimmed average will mostly measure how well someone holds up under pressure, whereas taking the fastest average will measure how well someone does in good circumstances.


----------



## JeffDelucia (Aug 14, 2010)

I think that winners of event should just be decided by who already has the fastest official average because averages are a good representation of speed and whoever has the fastest speed should win!




Spoiler



rofl sarcasm


----------



## RyanO (Aug 14, 2010)

The winner of a competition shouldn't necessarily be the best cuber. A competitive environment changes how you perform. The best cuber may not be the best competitor. If you don't perform well in the finals you shouldn't win the competition.


----------



## cmhardw (Aug 14, 2010)

Good points about having 2 or 4 rounds for 3x3x3, as well as how things would be decided for bigger cubes. So far it seems that the consensus is that the current method, though also arbitrarily picked by the first organizers of competitions, is at least not worse than any of the other proposed methods when all factors are taken into account. If this is the case I would strongly favor not changing the rules unless a clearly superior method is discovered, as the effects of the change-over would be quite far reaching.

Chris


----------



## reThinking the Cube (Aug 14, 2010)

fazrulz said:


> So, if this were to be implemented, this would force all competitions to have 3 rounds of 3x3 would it not? - Sometimes, due to lack of competitors, or time restrictions, this isn't possible. Out of 4 competitions I have been to, only one of them has had 3 rounds of 3x3. What about for other events? Would the winner of 4x4 be determined by the median time?



Any number of rounds is easy, and done the same way. The median times are only used to determine who advances to the next round. As it turns out, the winner of the event is also the only competitor to advance into the hypothetical final+1 round. The only difference for the last round is that the total number of trims allowed will always be 2 for each 5 round solve completed to get there.

Example, for a 2 round event, it is easy to use the median gotten from round 1 to get the final group of competitors. Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. All Official WCA 5-solve round times are always still preserved. This will produce a better Round2 selection, and also a fairer determination of the winner of the event, since 10 solves are used, and the trims can be applied uniformly to the whole set.

A 4 round event would be done the same way by extension, it makes no difference how many rounds there are. If it was 1 round, then the normal 5-solve trimmed average would decide the winner. That also answers the 4x4x4 question. These are not problems at all.


----------



## Pedro (Aug 14, 2010)

Agreed with Chris.
There's too much variation in number of rounds/solves to be able to achieve a consistent method.
And, as some others said, the best cuber is determined by the world ranking, not the result of a single competition. Who wins the competition is who performed better that day/round.

Recently, I've been to a comp (ABC Open 2010[/url) and, not that I'm bragging about it, but I was clearly the best cuber there. Turns out my cube popped once (I was using a OH cube) and I twisted a piece in the final round, getting an average of 17.xx, which was almost not enough for 3rd place.

Does that mean I wasn't the best cuber there? I don't think so. It means I performed bad in the last round, due to many factors.

I think competitions measure a different thing than the world rankings.
Within a competition, some people in some place are cubing together, and the best *performer* wins that comp. 
But everyone's times go to the world ranking, which includes each competion that happened, no matter where. The best cuber, to me, is Feliks at the moment, because he has done the best performance ever.


----------



## hawkmp4 (Aug 14, 2010)

Okay, again, why do you want to trim the averages the way you do? It's arbitrary. Why trim 4 from the averages of 10? If you're looking for consistency, taking the median time is going to be less affected by outliers.


----------



## qqwref (Aug 14, 2010)

reThinking the Cube said:


> Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. [...]
> 
> If it was 1 round, then the normal 5-solve trimmed average would decide the winner.


So you're changing your mind, then? You don't want to trim 2 on each side off a 5-solve average, and 3 on each side off a 10-solve average?


----------



## Daniel Wu (Aug 14, 2010)

All of this is so convoluted that it would seem that the current system is better just because of the simplicity.


----------



## reThinking the Cube (Aug 14, 2010)

qqwref said:


> reThinking the Cube said:
> 
> 
> > Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. [...]
> ...



No change, it is very simple. All 5-solve rounds will always get 2 trims just like normal. Now, when there is another round to advance into, then you are given 2 extra trims (you could consider them to be borrowed in advance, from the upcoming 5-solve round) to be applied to ANY of the solves that have already been done in ANY previous round. The solve times that are left after trimming are then averaged, and this is the value that is used to determine who advances into that next round and that is all. This works perfectly no matter how many rounds there are. The final round cannot borrow, so that it trims at 2 per 5 solves, which gives the same trimming ratio as the current method.

be moar nice


----------



## Faz (Aug 14, 2010)

rickcube said:


> All of this is so convoluted that it would seem that the current system is better just because of the simplicity.



This.


----------



## aronpm (Aug 14, 2010)

I don't see any problems with the current system.


----------



## reThinking the Cube (Aug 14, 2010)

aronpm said:


> I don't see any problems with the current system.



LOL. You're mostly blind anyway - right?


----------



## qqwref (Aug 14, 2010)

reThinking the Cube said:


> No change, it is very simple.


You said in a previous post that you wanted to trim out four times for a single round, and six times for two rounds, to determine who would get into the next round. Now you say that you want to trim out two times per round, which would mean removing two for a single round and four for a double round. I cannot make this any clearer - your recommendation has changed.

If you're not intelligent enough to seriously contribute to the thread, you should probably just stop posting here.


----------



## aronpm (Aug 14, 2010)

reThinking the Cube said:


> aronpm said:
> 
> 
> > I don't see any problems with the current system.
> ...



LOL. You're mostly 4x4-edge-flip anyway - right?


----------



## reThinking the Cube (Aug 14, 2010)

qqwref said:


> If you're not intelligent enough to seriously contribute to the thread, you should probably just stop posting here.



LOL. Does that statement make you intelligent enough to be considered a ............cough***hypocrite***? If you're not intelligent enough to seriously consider what I have clearly explained to you, then you should probably just stop posting insults, and go back and read it again. It is very clear that you are more interested in posting negative comments in an attempt to save face, or to convolute this issue so that others have a more difficult time endorsing it, rather than seriously contributing to the discussion. You can do much better, and I for one will not stop you from making better posts in the future.


----------



## hawkmp4 (Aug 14, 2010)

reThinking the Cube said:


> qqwref said:
> 
> 
> > If you're not intelligent enough to seriously contribute to the thread, you should probably just stop posting here.
> ...



When you actually address the questions we've been asking you, qq will probably be more inclined to take you seriously. As it stands, you're still ignoring the issue of you changing your mind and also why you chose the numbers of trims that you did.

If you read qq's last post, he actually contributed to the discussion. And then, you follow up with a post with no discussion of the issue at stake- just whining about how we're (mostly qq) calling you out on you changing your mind. You didn't actually address the issue. You ignored it.

If we're so wrong, care to point out how your previous posts _aren't_ incompatible? Because you HAVE made two different recommendations. That's clear to anyone who can proficiently read the English language, and I think we can agree that everyone involved in this discussion can.
Once you address the issues of inconsistency, can you explain how you came to that recommendation? As multiple people have said, it seems arbitrary. 
If you really are intent on contributing to this discussion, it seems that at a minimum you should address those issues, instead of continuing to plug your ears and sing.

EDIT:


> If you're not intelligent enough to seriously consider what I have clearly explained to you, then you should probably just stop posting insults, and go back and read it again.


Maybe _you_ should read what qq is saying- if you did, it'd be clear that he understands your proposed methods, that's not the issue. It's that you've proposed multiple methods, changing your mind without explanation. So point the finger at yourself, first.


----------



## reThinking the Cube (Aug 14, 2010)

hawkmp4 said:


> When you actually address the questions we've been asking you, qq will probably be more inclined to take you seriously. As it stands, you're still ignoring the issue of you changing your mind and also why you chose the numbers of trims that you did.



You both are glossing over what I have already explained, but I can see that for me to keep repeating that, is not making progress, and so I will go over this once again, so that you, and everyone else can see how simply better this really is.

From the beginning -



reThinking the Cube said:


> *Here is a format that can easily and fairly determine which competitors deserve to advance to the next round, but doesn't eliminate strong competitors early, AND be implemented without placing additional burden on organizers, uses all 15 solves to better decide the winner, generates official WCA rank times for each round, and is more spectator/competitor friendly than the current format.*
> 
> All(3) rounds of 5 are done exactly the same as the current format. The 5-solve rounds are trimmed and averaged the same, to produce an official WCA time for that round for that event. Round#1 is 5 solves and averaged just like it is now.
> 
> ...



That was the original post describing the format. Because there are additional rounds, the trimming after Round#1 and Round#2 is specifically being used to determine who gets into the next round. The trimming that is done after the final Round#3 is specifically being used to fairly determine who should win the event. 

Now, some misconceptions concerning whether or not this would work for varying number of rounds led to this post -



reThinking the Cube said:


> fazrulz said:
> 
> 
> > So, if this were to be implemented, this would force all competitions to have 3 rounds of 3x3 would it not? - Sometimes, due to lack of competitors, or time restrictions, this isn't possible. Out of 4 competitions I have been to, only one of them has had 3 rounds of 3x3. What about for other events? Would the winner of 4x4 be determined by the median time?
> ...



Note that the 6 trims on 10 solves that were applied at the end of Round#2 in the 1st example is for getting a value that will determine who will go into Round#3. The 2nd example I gave was specifically for a 2 round event, since that was the question (how would this work for events with more or less rounds?). For a 2 round event, 10 solves completes it, and for the final round the final trimming is solely for the purpose of fairly determining the winner. In this case 4 trims applied using all 10 solves, averaging the rest. These cases are not the same, and are trimmed accordingly. 

I also posted a more detailed explanation in the following post to further clarify this. 



reThinking the Cube said:


> qqwref said:
> 
> 
> > reThinking the Cube said:
> ...




Now that you have coerced me into breaking my previous record for longest and most redundant post, I hope that you feel totally convinced that I have not been ignoring anything, and that qq is in need of a good trim


----------



## qqwref (Aug 14, 2010)

I think I see what you are trying to say. You're intensely bad at communication, though; generally if someone is misunderstanding your point, it is useful to add LESS text (distill it down to the essentials, and summarize) and not to add MORE. But I can't and don't expect you to understand your own point well enough to summarize it.

Here is what I think reThinking was trying to say (and watch as I explain it in such a way that everyone will understand):
- After N five-solve rounds, there are two possible ways we could take an average, which will be used for different situations.
- To determine the winner (if this is the finals), we take averages by taking all 5N solves and trimming off N from each side.
- However, to determine who makes the next round (if this is not the finals), we take averages by taking all 5N solves and trimming off N+1 from each side.

Now, I don't really like the idea of trimming an extra solve on each side. It seems like it will just change the averages a bit, in an essentially random way, compared to the standard 20% trim. If one of the two gives better results (i.e. better distinguishes between a faster cuber and a slower one) then it should be used both to decide winners AND to decide advancements. Personally I don't see anything wrong with trimming exactly 20% and I think that, if we want to adopt a winning condition where the average of all solves matters, we should do just that. We're already used to a 20% trim from our average of 5 metric and so doing that for multiple rounds would not break tradition so much as extend it. (I'm still in favor of the best average winning, though.)


----------



## reThinking the Cube (Aug 15, 2010)

qqwref said:


> I think I see what you are trying to say. You're intensely bad at communication, though; generally if someone is misunderstanding your point, it is useful to add LESS text (distill it down to the essentials, and summarize) and not to add MORE.



Well then, I guess it would be fair for me to say - "You're intensely bad at comprehending".  You said you wanted MORE, so that you could understand it better. You got what you wanted, so you should be thankful. Instead, you are distilling down to whatever essential spitefulness you can come up with.



qqwref said:


> But I can't and don't expect you to understand your own point well enough to summarize it.



Is this YOUR example of "intensely *bad* communication"?
How can that be interpreted as anything other than a childish insult? I really want to respect your contributions here qqwref, but it has to go both ways. Do you think it is reasonable for you, to keep on sowing thistles, and then expect to be rewarded with roses? 



qqwref said:


> Here is what I think reThinking was trying to say (and watch as I explain it in such a way that everyone will understand):
> - After N five-solve rounds, there are two possible ways we could take an average, which will be used for different situations.
> - To determine the winner (if this is the finals), we take averages by taking all 5N solves and trimming off N from each side.
> - However, to determine who makes the next round (if this is not the finals), we take averages by taking all 5N solves and trimming off N+1 from each side.



This is a rather concise explanation of the procedure, and I appreciate you presenting it like that, but you must also realize that you did not have the additional burdens of explaining the rationales behind doing it this way. I don't think your condescending remark has any merit, and if there has been any trouble with people understanding what I had posted earlier, then it is primarily due to your deliberate and incessant attempts to muck up the discussion. You are whining about the smoke that got into your eyes - from a fire that you started yourself.



qqwref said:


> ... *Personally* I don't see anything wrong with trimming exactly 20%....



You are (maybe deliberately) overlooking some important advantages that are derived from adding the extra trims to help decide more fairly who advances into the next round. This has already been explained very well, in some earlier posts, and so I won't repost it again here. 

The bottom line, this new format is clearly a nice improvement over the current format, or any other format that you have proposed as an alternative, and that is that.
No need to get so personal about it.


----------



## hawkmp4 (Aug 15, 2010)

reThinking the Cube said:


> qqwref said:
> 
> 
> > ... *Personally* I don't see anything wrong with trimming exactly 20%....
> ...



If you want consistency, again, I will ask, why not take the median of all solves? If you want to trim more, why not go all the way and trim all but one solve (or two, if there is an even number of solves)?


----------



## reThinking the Cube (Aug 15, 2010)

hawkmp4 said:


> If you want consistency, again, I will ask, why not take the median of all solves? If you want to trim more, why not go all the way and trim all but one solve (or two, if there is an even number of solves)?



This valid point -



qqwref said:


> fazrulz said:
> 
> 
> > What about for other events? Would the winner of 4x4 be determined by the median time?
> ...



And, using an average eliminates the discreet variances between the solves that are not considered to be outliers.


----------



## qqwref (Aug 15, 2010)

reThinking the Cube said:


> qqwref said:
> 
> 
> > I think I see what you are trying to say. You're intensely bad at communication, though; generally if someone is misunderstanding your point, it is useful to add LESS text (distill it down to the essentials, and summarize) and not to add MORE.
> ...


When did I say I wanted more text? I'm pretty sure I just wanted you to be more clear. Anyway, when communicating ideas, the onus is generally on the *sender* to make sure they are understood. If your ideas are not understood, it is your fault for communicating them badly.



reThinking the Cube said:


> How can that be interpreted as anything other than a childish insult? I really want to respect your contributions here qqwref, but it has to go both ways. Do you think it is reasonable for you, to keep on sowing thistles, and then expect to be rewarded with roses?


From your first appearance in this thread, you have been antagonistic: responding to even mild criticisms with insults; ignoring any problems with your ideas, yet insisting that anyone who does not agree with them is wrong; being insulting rather than cooperative when asked to explain things. I'm not interested in being "rewarded with roses" from you because I know you are not capable of this. Your last sentence applies more to you than to me, anyway: your posts and attitude in general portray an extremely arrogant person who expects respect from everyone but does not know how to respect others (and I don't mean saying "I respect you"). I assume this is why you refuse to follow even the most basic etiquette and standards of the community. If people thought you were capable of adult discussion, they would almost certainly be nicer to you.



reThinking the Cube said:


> This is a rather concise explanation of the procedure, and I appreciate you presenting it like that, but you must also realize that you did not have the additional burdens of explaining the rationales behind doing it this way.


Procedure and rationale are two different sides of a coin and should not be presented together in a giant wall of text. Again this is about you being bad at communicating. The reason people talk about organization and conciseness when communicating is because it makes it easier to understand; again, if you communicate badly, it is your fault, not the fault of others. If you were able to respect others, you would realize that it is indeed possible (easy, even) to communicate badly enough that an intelligent but finitely patient reader could misinterpret what you write. 



reThinking the Cube said:


> qqwref said:
> 
> 
> > ... *Personally* I don't see anything wrong with trimming exactly 20%....
> ...


I don't agree with this. If trimming two extra times provides such an advantage in fairness, why not keep trimming until only one or two times remain? Your idea has some arbitrary numbers which you have not provided any explanation for. I proposed 20% because it is NOT arbitrary - we already trim by this amount, so if we always trim any multiple of 5 solves by 20% on each side we have a logical progression. But adding one extra trim is arbitrary because you have not established that it is (a) better than adding no extra trims, but also (b) better than adding two or more extra trims. Keep in mind that I do not consider a measure which gives a lower standard deviation (assuming everyone is a normal distribution, which isn't true) to, in every case, be fairer.

Look at Square-1 or 4x4x4: the more numbers you trim, the more there is a difference between getting some number of counting parity solves and getting one more counting parity solve. On Square-1, for instance, if a parity solve is always worse than a non-parity solve, trimming 4 solves out of 5 will mean that everyone has a 50/50 chance of having their only counting solve be one with parity. The more solves you count, the less this matters, but there's a significant luck component when you trim too much. If you only trimmed 2 solves on one round of Square-1, like we do now, there is a 1/8 chance of 0 parities, 3/8 chance of 1, 3/8 chance of 2, and 1/8 chance of 3. So the extremes of having every counting solve be parity or every counting solve be non-parity are 1/4 as likely.

I suppose at this point it's worth it to bring up the question: what exactly is our goal here? It seems like you are trying to estimate every cuber's median, but I don't consider this fair because it puts too much emphasis on exactly how many good (as in, below a certain margin) solves someone has. Instead of many solves counting, wanting a sub-X median is just a question of whether you got (N+1)/2 solves under X or not. If you're trying to estimate a mean, it's important to note that you can't reliably estimate the mean on a non-normal distribution if you trim off everything that looks like it might be on one of the tails. My goal, on the other hand, is to find the person who gave the best performance, and I think the averages in each round are the data points of that, which is why I'd prefer to rank by either the best average or, if necessary, the mean of the averages.


----------



## reThinking the Cube (Aug 15, 2010)

qqwref said:


> reThinking the Cube said:
> 
> 
> > qqwref said:
> ...



I don't understand. Sorry, but you are a very bad communicator. 



qqwref said:


> ....a very large pile of self-serving insults and abelian scented jibberish.



Did anyone ever tell you qqwref, that you can't be right ALL the time?



qqwref said:


> But adding one extra trim is arbitrary because you have not established that it is (a) better than adding no extra trims, ...



Yes I have, but you stubbornly refuse to acknowledge it. With the current format, if a serious competitor gets more than 1 bad solve in a round, then they may be eliminated because of it. With my format, a serious competitor can get at least 2 and sometimes more bad solves in a round and still advance. When there is a good reason for something, then it is not *arbitrary*.



qqwref said:


> Look at Square-1 or 4x4x4: the more numbers you trim, the more there is a difference between getting some number of counting parity solves and getting one more counting parity solve. On Square-1, for instance, if a parity solve is always worse than a non-parity solve, trimming 4 solves out of 5 will mean that everyone has a 50/50 chance of having their only counting solve be one with parity. The more solves you count, the less this matters, but there's a significant luck component when you trim too much. If you only trimmed 2 solves on one round of Square-1, like we do now, there is a 1/8 chance of 0 parities, 3/8 chance of 1, 3/8 chance of 2, and 1/8 chance of 3. So the extremes of having every counting solve be parity or every counting solve be non-parity are 1/4 as likely.



Yes, but the format I proposed doesn't trim it that way if there is only 1 round for the event. There is no difference between the formats for these typical one rounders.

2010 Nationals had a 2 round Square-1 event. (Do you have a good way to post those here so that they display correctly?) Using a median value for qualification would have put Ian Jones (26.15) into the final, since his 1st round times appeared to have had 2 bad parity cases that would have been trimmed out. I think that would have been fairer. All the other competitors that advanced would have done so anyway, with either format. So in this case (and in every case that I have looked at so far) using the median value would have been the better method. Another advantage to using the median, is that it can easily be determined by everyone, without the use of a calculator.

Now if all 10 solves (-4) were used to determine the tournament places, there would have also been a couple of improvements. David Woner(5th) and Takao Hashimoto(4th) both appeared to have gotten at least 2 bad parity cases in the final round, and were hurt somewhat by that. By trimming it my way they each would have moved up 1 place. I think that also would have been fairer. Forte Shinko looked to be coasting a little in the 1st round (another problem of the current format), and that would have affected his placement negatively, but he probably would have gotten better times if he had known that it would matter. All the other results would have been pretty much the same. I couldn't find any negatives at all for doing it my way. 
The parity concerns that you have, are actually MORE of a problem with the current format, since it only takes 2 bad parity-solves to start hurting a round. My way of trimming can usually mitigate that.


----------



## ExoCorsair (Aug 15, 2010)

tl;dr keep it civil or else bad things will happen.


----------



## Daniel Wu (Aug 15, 2010)

reThinking the Cube said:


> With my format, a serious competitor can get at least 2 and sometimes more bad solves in a round and still advance.


:confused: Why should one advance if they mess up a bunch of solves. 

This entire discussion hasn't made any sense in a while.


----------



## qqwref (Aug 15, 2010)

reThinking the Cube said:


> qqwref said:
> 
> 
> > When did I say I wanted more text? I'm pretty sure I just wanted you to be more clear. Anyway, when communicating ideas, the onus is generally on the *sender* to make sure they are understood. If your ideas are not understood, it is your fault for communicating them badly.
> ...


Really? What part do you have trouble with? (Or are you dishonest enough to be lying? Unlike you, I would help someone understand what I have written, if they need it.)



reThinking the Cube said:


> qqwref said:
> 
> 
> > ....a very large pile of self-serving insults and abelian scented jibberish.
> ...


You don't even have a rebuttal? What a joke. Do you think I'm the only person who thinks you're incapable of serious discussion?



reThinking the Cube said:


> qqwref said:
> 
> 
> > But adding one extra trim is arbitrary because you have not established that it is (a) better than adding no extra trims, ...
> ...


And I don't think that trimming four solves out of five will provide a more fair measure. You haven't provided any justification about this other than complaining that I don't immediately agree to your flawed reasoning.



reThinking the Cube said:


> 2010 Nationals had a 2 round Square-1 event. [...] Using a median value for qualification would have put Ian Jones (26.15) into the final, since his 1st round times appeared to have had 2 bad parity cases that would have been trimmed out. I think that would have been fairer.


I strongly disagree. You have clearly never tried to solve the Square-1. Parity is a matter of luck and cannot be affected by a solver in 15 seconds of inspection. Why do you think Ian deserves to be in the final, when if he had had three parity solves instead of two (and this is 100% luck based here) he would have a median of 33 or so? I had four parity solves in that round. **** happens. But how is putting even MORE reliance on luck more fair? If getting 3 or more parity solves has a 50% chance, and you just take the median, why not just do a best of one? You'll have parity half the time there too. I'm not being stubborn here, I'm showing you a legitimate problem with your approach. Somehow I doubt you will provide a reasonable answer.



reThinking the Cube said:


> Now if all 10 solves (-4) were used to determine the tournament places, there would have also been a couple of improvements. David Woner(5th) and Takao Hashimoto(4th) both appeared to have gotten at least 2 bad parity cases in the final round, and were hurt somewhat by that. By trimming it my way they each would have moved up 1 place.


And you don't think Forte deserves his 3rd place spot at all, and should be in 5th? He got the same 5 scrambles as Woner and Takao, and did better on them. How do you know he didn't get bad scrambles in the first round and do badly as a result? (You don't solve Square-1, so I'll tell you: each scramble has a specific "cubeshape", which is the same for all methods, and some are much easier than others. So all scrambles have a small objective difficulty component.) I don't see a fairer ranking, just a DIFFERENT ranking.


----------



## reThinking the Cube (Aug 15, 2010)

qqwref said:


> reThinking the Cube said:
> 
> 
> > qqwref said:
> ...


Not really?! Unlike you, I have a marvelous sense of humor, and can also comprehend the meaning of that *sarcasm* symbol in the context above. Now having distilled this down to its essentials, I think it would be best if you could just focus on the topic under discussion.



qqwref said:


> reThinking the Cube said:
> 
> 
> > qqwref said:
> ...


That joke WAS the rebuttal.  Do you think I'm the only person who thinks YOU'RE INCAPABLE OF AGREEING WITH ME ON ANYTHING? If you want a serious discussion, then you have to at least be capable of agreement. 
I suppose at this point it's worth it to bring up the question: What exactly is our goal here? 



qqwref said:


> reThinking the Cube said:
> 
> 
> > qqwref said:
> ...



First, you were attacking this format with the false charge that - "the one extra trim was *arbitrary*". That was flawed reasoning on your part, not mine, as clearly explained in the post above. There is also the justifiable advantage that the median can easily be determined without the use of a calculator.

BOTTOM LINE: If you don't think that trimming to the median would provide a better way for determining who gets to advance, then I would suggest that you justify your opinion with some specific tournament examples (starting with 2 round 3x3x3), rather than continue to complain about me not agreeing with your flawed reasoning.


----------



## macky (Aug 16, 2010)

cmhardw said:


> Good points about having 2 or 4 rounds for 3x3x3, as well as how things would be decided for bigger cubes. So far it seems that the consensus is that the current method, though also arbitrarily picked by the first organizers of competitions, is at least not worse than any of the other proposed methods when all factors are taken into account. If this is the case I would strongly favor not changing the rules unless a clearly superior method is discovered, as the effects of the change-over would be quite far reaching.
> 
> Chris





fazrulz said:


> rickcube said:
> 
> 
> > All of this is so convoluted that it would seem that the current system is better just because of the simplicity.
> ...








Pedro said:


> Agreed with Chris.
> There's too much variation in number of rounds/solves to be able to achieve a consistent method.
> And, as some others said, the best cuber is determined by the world ranking, not the result of a single competition. Who wins the competition is who performed better that day/round.
> 
> ...



I agree.

Now can I at least hear some opinions on [URL="http://www.speedsolving.com/forum/showthread.php?t=23261"]my suggestion for a "global" ranking? Would such a ranking be useful? Worth being displayed on the WCA database? Is it just interesting, or is it as important as the best average of 5? Is it important enough to warrant giving some title based on it? Would it be hard to agree on a consistent ranking method? Are there other practicality considerations?


----------



## qqwref (Aug 17, 2010)

I'm sorry, reThinking, but I facepalmed too hard at your post to bother responding to what you said. I've been trying to stay on topic but it's pretty clear you don't actually care about that, since you haven't addressed any of the points I brought up, and are instead either copying whatever criticisms I make (legitimate or not - I don't think you actually ever considered any of them) or stating that your position is obvious (and that, I guess, I must be being deliberately stubborn if I don't agree?). If you ever honestly want to discuss competition ranking formats, you know how to contact me.


----------



## Pedro (Aug 17, 2010)

That square-1 argument is not that valid, in my opinion.

As Michael said, square-1 is a different puzzle, which involves more luck (cube shapes and parity). Do you have any evidence to say that "it looks like they had some bad cases/parity/whatever"?

At my last square-1 comp, I did pretty bad, mostly because I was out of shape. But I had a stupidly easy cube shape, which was a 30+ solve because I messed up even more stupidly. Should we trim that? What about everyone who actually did well with that scramble? Should they be penalized?

Also, as I said before, I had a terrible final round in ABC Open. One pop and one corner twist (DNF and 22.xx, respectively). Suppose this happened in the 2nd round and people did really well, kicking me out of the final. With your system, I'd have a median of 13.91, which is much better than my 17.xx average, and would have surely advanced. How is that fair? Yes, I did have 2 bad solves. Why not trim those?

Because they were my fault. For using a OH cube for regular 3x3, for not being enough careful with that cube, for not being well focused.

We already trim best and worst solve to account for a pop or distraction or whatever. I don't think we should trim more than that, even if it's just to determine who advances. If you mess up twice in a round, well, too bad. Better luck next time. If you get bad cases or are disturbed by people clapping (which is kinda not your fault), should you get a "discount"? I don't think so...how do you know if people just messed up or were harmed by external factors? I don't see a good way. So you would be 'helping' people by allowing them to mess up even more and still qualify.

Suppose this scenario:
Competitor A gets (13.10), 13.30 ,14.51, 14.58, (14.90) = 14.13 avg
Competitor B gets (13.20), 14.30, 14.50, 18.56, (19.70) = 15.79 avg

Is it fair for B to advance because of the median (14.50 x 14.51)?


----------



## PhillipEspinoza (Aug 17, 2010)

riffz said:


> I think most people would agree that they care more about their WCA ranking in an event than how they placed at a specific competition.



Unless that competition is Worlds...


----------



## reThinking the Cube (Aug 17, 2010)

@qq: apology accepted, and we surely can, "concordia res parvae crescent" - work together to accomplish more.



Pedro said:


> Do you have any evidence to say that "it looks like they had some bad cases/parity/whatever"?



I looked at the *evidence*, which was the tournament results. The times for those solves were rather poor in comparison to other solves that were done by those same competitors. To me it looked like bad cases, but it could have been bad gas.  Only they would know for sure.



Pedro said:


> At my last square-1 comp, I did pretty bad, mostly because I was out of shape. But I had a stupidly easy cube shape, which was a 30+ solve because I messed up even more stupidly. Should we trim that? What about everyone who actually did well with that scramble? Should they be penalized?



How does trimming your bad solve, from your times, penalize anybody else? This is how it is done now.



Pedro said:


> Also, as I said before, I had a terrible final round in ABC Open. One pop and one corner twist (DNF and 22.xx, respectively). Suppose this happened in the 2nd round and people did really well, kicking me out of the final. With your system, I'd have a median of 13.91, which is much better than my 17.xx average, and would have surely advanced. How is that fair? Yes, I did have 2 bad solves. Why not trim those?



If you had done really well in the 1st round, and both of the 2 worst times came in the 2nd round, then YES it would be fair to trim them. That is NOT how it is done now, but the main idea behind the extra trimming is to give the Cubers with the best overall performance at that point in the tournament, the fairest chance of advancing into the final round. It should be noted that even though you may have advanced into the final with 2 poor times in the 2nd round, those poor results could eventually impact on your overall performance, since you only get a finite# of trims (applied to all 15 solves) for calculating your tournament average (my way).



Pedro said:


> We already trim best and worst solve to account for a pop or distraction or whatever. I don't think we should trim more than that, even if it's just to determine who advances. If you mess up twice in a round, well, too bad. Better luck next time. If you get bad cases or are disturbed by people clapping (which is kinda not your fault), should you get a "discount"? I don't think so...how do you know if people just messed up or were harmed by external factors? I don't see a good way. So you would be 'helping' people by allowing them to mess up even more and still qualify.



The standard trimming that is done right now isn't considered to be a *discount*. Besides, all competitors' rounds would be trimmed the same, no matter which format. Trimming to median value will just weed out more of the bad/good luck solves that would have been used (IMO unfairly) to decide who gets to advance. I respect your opinion - "If you mess up twice in a round, well, too bad.", but it doesn't HAVE to be that way. Look at Sikan Li in round 2 of the Bayview Hills Open 2010. Trimming it my way, he advances to the finals, which I think would have been fairer, based on his OVERALL performance. And in the same tournament, an even clearer example would be Phillip Espinoza's final round disaster.



Pedro said:


> Suppose this scenario:
> Competitor A gets (13.10), 13.30 ,14.51, 14.58, (14.90) = 14.13 avg
> Competitor B gets (13.20), 14.30, 14.50, 18.56, (19.70) = 15.79 avg
> 
> Is it fair for B to advance because of the median (14.50 x 14.51)?



Thank you for presenting this question. I am assuming these times are for 3x3x3 round 1. With my way there is indeed a .01 difference going to B. This will only actually make a difference, if and only if, the last spot for advancement into round 2 is taken by B. Now that scenario is not very likely, but in either case then, both A and B are the SLOWEST competitors coming from round 1, that can advance. Also note that those 2 poorer times of B will not go away, and may impact latter on (my way). It may also be the case that B is really faster than A, and just got a terrible round. 

My main concern is to prevent the strongest competitors from being eliminated early simply due to bad luck. 

Suppose this scenario: [1st Round 3x3x3]
Competitor A gets (13.10), 13.30 ,14.51, 14.58, (14.90) = 14.13 avg (same as above)
Competitor B gets (9.20), 9.30, 14.50, 19.70, (DNF) = 14.50 avg (strong cuber+bad,bad luck)

Is it fair for B to advance because of the median (14.50 x 14.51)?

For this question, both competitors would also have the 2 slowest medians (14.50, 14.51), and since B clearly shows the potential to outperform A, I would favor B advancing. Yes, this IS fair. Using the median value here, would clearly be superior to using the 5-solve average to decide which competitor gets to advance.


----------



## hawkmp4 (Aug 17, 2010)

I find it silly to be talking about what's 'fair' or not.

Everyone is subject to the same rules, regulations, and procedures. It's not unfair for a good cuber not to advance if they don't perform well. It SUCKS, I understand that, but it's not unfair to them.


----------



## Tim Major (Aug 17, 2010)

I haven't really read much of this thread, and this has prolly already been proposed... why not just have, the best averages of the comp go through to the finals.
If competitor A gets a 10 average first round, but in the second average they DNF, or have a 20 average, then they make it through. So the best averages of the previous rounds make it through to the next one. However, in the final, the person who gets the best average in the final wins.

(I personally think it should stay as it is. Sure a bit of luck is involved, but that's the same with most competition of any hobby/sport/whatever.)


----------



## Pedro (Aug 17, 2010)

reThinking the Cube said:


> *If you had done really well in the 1st round, and both of the 2 worst times came in the 2nd round, then YES it would be fair to trim them.* That is NOT how it is done now, but the main idea behind the extra trimming is to give the Cubers with the best overall performance at that point in the tournament, the fairest chance of advancing into the final round. It should be noted that even though you may have advanced into the final with 2 poor times in the 2nd round, those poor results could eventually impact on your overall performance, since you only get a finite# of trims (applied to all 15 solves) for calculating your tournament average (my way).



I don't see why it is fair. Why should my 1st round be a bias to my 2nd round? Different conditions (weather, audience, light, whatever) happen on each round, even our physical condition, cube condition is different. In my opininion, someone with a 10.xx and a 20.xx average shouldn't take the spot in the final from someone with two 12.xx averages.



> The standard trimming that is done right now isn't considered to be a *discount*. Besides, all competitors' rounds would be trimmed the same, no matter which format. Trimming to median value will just weed out more of the bad/good luck solves that would have been used (IMO unfairly) to decide who gets to advance. I respect your opinion - "If you mess up twice in a round, well, too bad.", but it doesn't HAVE to be that way. Look at Sikan Li in round 2 of the Bayview Hills Open 2010. Trimming it my way, he advances to the finals, which I think would have been fairer, based on his OVERALL performance. And in the same tournament, an even clearer example would be Phillip Espinoza's final round disaster.


Sikan Li avg of 8/10 = 16.56, avg of 6/10 = 16.41
Cameron Brown avg of 8/10 = 16.08, avg of 6/10 = 16.10
Alexander Lin avg of 8/10 = 16.21, avg of 6/10 = 16.10

So, no, Sikan didn't deserver to qualify, in my view 



> Thank you for presenting this question. I am assuming these times are for 3x3x3 round 1. With my way there is indeed a .01 difference going to B. This will only actually make a difference, if and only if, the last spot for advancement into round 2 is taken by B. Now that scenario is not very likely, but in either case then, both A and B are the SLOWEST competitors coming from round 1, that can advance. Also note that those 2 poorer times of B will not go away, and may impact latter on (my way). It may also be the case that B is really faster than A, and just got a terrible round.
> 
> My main concern is to prevent the strongest competitors from being eliminated early simply due to bad luck.
> 
> ...



Hmm...so you get to decide what IS fair...

The problem with your proposal is exactly what I presented. Yes, it may qualify a good cuber with some bad luck, but it may qualify a bad cuber which had a 0.01 luck, so this is NOT FAIR to me 

I say we keep the current system.


----------



## Lucas Garron (Aug 17, 2010)

Don't forget Arrow.

There *is no good ranking/advancement system*.

Personally, I'd rather keep the current system the same until a much better idea comes along (i.e. not an arbitrary tweak). Macky's proposal seems like a better idea.


----------



## Ravi (Aug 18, 2010)

Lucas Garron said:


> There *is no good ranking/advancement system*



That sounds like a bit of a stretch to me. Certainly, there is no *perfect* ranking/advancement system, but by invoking Arrow's theorem you're assuming all of its hypotheses: "unrestricted domain, non-dictatorship, Pareto efficiency, and independence of irrelevant alternatives." I'm honestly not sure which of those hold and which fail for the current system, but I would describe the existing system as "good"--just not the best.

But this raises a very important question: *What do we mean by a "good" ranking/advancement system?* I've listed below several qualities that I believe a "good" system would need to have. Not all are rigorously defined, of course.

1. *Fairness.* This means, for example, (1) that the overall result of the competition must be based on the individual results of the competitors and nothing else, (2) that the rules must apply equally to all competitors, and (3) that faster times can never hurt you and DNFs can never help. Furthermore, (4) the ranking system should be designed to rank competitors from fastest to slowest as accurately as possible, within the the bounds listed below; and (5) the advancement system should similarly aim to advance the fastest competitors after each round.

2. *Practicality.* This means, for example, (1) that competitors should not be expected to complete excessively many attempts, nor should organizers be expected to schedule excessive amounts of competing time; (2) that the rules should be simple enough for most speedcubers to understand; and (3) that the results should be reasonably easy to calculate.

3. *(Limited) forgiveness for outliers.* This means (1) that DNFs and occasional outliers, which are generally considered relatively unimportant, should not affect a competitor's result drastically, notwithstanding item 3 under "Fairness" above; but (2) that this forgiveness (ie. trimming) should not be taken to the extent of invalidating significant, near-average solves. A good guideline provided by the current system is (3) that roughly the middle 60% of attempts should count.

4. *Similarity to the current system.* This means, in addition to "Compatibility" below, (1) that the overall concept of a competition should not be changed unnecessarily, and (2) that excessive rule changes should not be applied lightly. Furthermore, (3) no significant change should be made without some consensus.

Additionally, I think the following qualities would be preferable, although perhaps not entirely necessary.

5. *Compatibility with previous records.* Single-solve records will presumably not be affected, but (1) the existing average-of-5 format should be preserved. Furthermore, (2) it should be preserved not only in name; if records are based on averages of 5, then the competition should demand good averages of 5 for advancement and/or ranking, to avoid forcing a competitor to decide between competing for ranking and competing for records.

6. *Symmetry and uniformity of rules.* This means (1) that, psychology aside, all solves should preferably be equal in value; that is, so to speak, that "a solve is a solve is a solve," and (2) that as far as practical, the same rules should apply throughout the competition.

Please comment on the guidelines above, and I will edit accordingly. I would really like to reach a consensus on this.


----------



## reThinking the Cube (Aug 18, 2010)

Ravi said:


> I'm honestly not sure which of those hold and which fail for the current system, but I would describe the existing system as "good"--just not the best.



I concur. 



Ravi said:


> 1. Fairness. This means, for example, (1) that the overall result of the competition must be based on the individual results of the competitors and nothing else, (2) that the rules must apply equally to all competitors, and (3) that faster times can never hurt you and DNFs can never help.



I have a primary concern that I would like to include here. Currently, the trims for 5-solve average are being applied to prevent outliers (good&bad) from unfairly affecting the results. Choosing to trim 1 from each side of 5 solves is completely arbitrary, but does work well for the majority of cases. There is a problem though when a competitor gets more than one good or bad outlier in any one round. The outlier(s) in that case end up skewing the results *unfairly*. Once that solve, (which should be considered an outlier), gets included into any 5-avg, the current system cannot keep it from unfairly biasing all the results that are based on it. Suggest edit (3) to address this concern.



Ravi said:


> 3. (Limited) forgiveness for outliers. This means (1) that DNFs and occasional outliers, which are generally considered relatively unimportant, should not affect a competitor's result drastically, notwithstanding item 2 under "Fairness" above; but (2) that this forgiveness (ie. trimming) should not be taken to the extent of invalidating significant, near-average solves. A good guideline provided by the current system is (3) that roughly the middle 60% of attempts should count.



For a system to be *perfect*, it would have to trim ALL the outliers (good&bad), and NONE of the others. A reasonable ideal then, is to fairly trim as many of the outliers as possible, from all the rest. The current system, is deriving an average using the middle 60%, but this is an arbitrary compromise that is assumed to approximate the ideal. Sometimes it does, sometimes it doesn't. *Anytime there is more than 1 good/lucky, or more than 1 bad/unlucky solve, in a round of 5, the arbitrary trimming for 5-avg. yields a result that is polluted.* This is not trivial, since all tournament results, whether based on a single round, or an average of averages, and all rankings, are negatively affected by outlier polluted 5-avg. IMO - deriving some results based on medians, would be much better, since outliers would all get (Perfected/Unlimited) forgiveness. 



Ravi said:


> Please comment on the guidelines above, and I will edit accordingly. I would really like to reach a consensus on this.



Most excellent guidelines, nice work. I can pretty much agree with all of it, except for the concern(s) mentioned above.


----------



## macky (Aug 18, 2010)

> Ravi said:
> 
> 
> > 1. Fairness. This means, for example, (1) that the overall result of the competition must be based on the individual results of the competitors and nothing else, (2) that the rules must apply equally to all competitors, and (3) that faster times can never hurt you and DNFs can never help.
> ...


 (emphasis mine)

@reThinking the Cube: I don't think you understand Ravi. He's imitating the kind of axiomatic approach used for example for Arrow, beginning with the bare minimum requirements and trying to construct a method that satisfies them. The "fairness" you speak of has nothing to do with the very basic fairness that Ravi describes (just go through items 1, 2, and 3). You're letting your own (I'd say misguided) sense of fairness shape both your suggestions and your arguments, without much basis.

First, read these comments:


Pedro said:


> Hmm...so you get to decide what IS fair...





hawkmp4 said:


> I find it silly to be talking about what's 'fair' or not.
> 
> Everyone is subject to the same rules, regulations, and procedures. It's not unfair for a good cuber not to advance if they don't perform well. It SUCKS, I understand that, but it's not unfair to them.



Then, back up. With the understanding that it's difficult to measure any fairness that goes beyond the bare minimum that Ravi suggests, what reasons can you provide to support that your method is more fair than the current system? I actually think your method is less fair because of the following comment from Pedro, and you need to address it:


Pedro said:


> The problem with your proposal is exactly what I presented. Yes, it may qualify a good cuber with some bad luck, but it may qualify a bad cuber which had a 0.01 luck, so this is NOT FAIR to me





Also, regarding what you wrote here (emphasis mine):


reThinking the Cube said:


> For this question, both competitors would also have the 2 slowest medians (14.50, 14.51), and *since B clearly shows the potential to outperform A, I would favor B advancing. Yes, this IS fair.* Using the median value here, would clearly be superior to using the 5-solve average to decide which competitor gets to advance.


"Potential to outperform" suggests that you want the best cubers to advance. I, and also Pedro from what I've read, believe that the cuber who performs "best" (in some predefined way) _at the competition_ should advance. I wonder if you're just starting with different idea of what a competition ranking should achieve. Neither goal conflicts with Ravi's fairness requirements.

In any case, I'd suggest writing up your argument in its entirety: what a competition ranking should achieve, your idea of fairness, and why your method is significantly more fair than the current system, enough to warrant a rule change.


----------



## Pedro (Aug 18, 2010)

Another hard thing to define is what is an outlier. How far should it be from the median/average value to be considered so? I don't have a good answer...

For someone with a 11 seconds average, a 15 time is just terrible, but a non-lucky 9 is probably an outlier, since he/she gets it not that often. But is not that far from the average. Should it be counted out?

Also, different speeds generate different ranges of times. I average high-12's now (global average, I can do 11.50 when I'm doing good) and I get non-lucky times from 9 to 16. I can also mess up and get 17 or 18, which doesn't give an equal range in both directions (up and down from the average). How we decide what should and what shouldn't count?

Someone who is really consistent may complain about using just the median value. For example:
Competitor A: (11), 11.2, 11.5, 11.6, (11.8)
Competitor B: (10), 11.3, 11.4, 13.7, (15.8)

With the current system, A clearly beats B. Using just the median, B would get a better ranking and maybe kick A out of the next round.


One that that *could* work is making a competition ranking. Everybody does an average, and the best X advance (just like it is now). Then, in the second round, the top Y *results* advance. Then we'd have to go by best average in the comp to determine the winner. It would be kinda weird to have a different system to determine who advances and who is the winner.

I don't like this very much because someone who gets a really good average in the 1st round may not even try on the 2nd because he/she will surely qualify. I prefer "forcing" people to perform good in every round if they want to advance/win. If someone can do good in all rounds, he's probably a good cuber, which is what we (supposedly) want to find


----------



## blade740 (Aug 18, 2010)

reThinking the Cube said:


> For a system to be *perfect*, it would have to trim ALL the outliers (good&bad), and NONE of the others. A reasonable ideal then, is to fairly trim as many of the outliers as possible, from all the rest. The current system, is deriving an average using the middle 60%, but this is an arbitrary compromise that is assumed to approximate the ideal. Sometimes it does, sometimes it doesn't. *Anytime there is more than 1 good/lucky, or more than 1 bad/unlucky solve, in a round of 5, the arbitrary trimming for 5-avg. yields a result that is polluted.* This is not trivial, since all tournament results, whether based on a single round, or an average of averages, and all rankings, are negatively affected by outlier polluted 5-avg. IMO - deriving some results based on medians, would be much better, since outliers would all get (Perfected/Unlimited) forgiveness.
> )



There are a few problems with your idea:



> For a system to be *perfect*, it would have to trim ALL the outliers (good&bad), and NONE of the others



Taking the median would certainly trim more non-outlier times than the middle 3. Occasionally it would trim an extra bad solve, but for cubers who perform fairly consistently, it would trim average results unnecessarily.



> *Anytime there is more than 1 good/lucky, or more than 1 bad/unlucky solve, in a round of 5, the arbitrary trimming for 5-avg. yields a result that is polluted.*



What about puzzles that have a 50% chance of parity, like 4x4 and square-1? A 50% chance of parity on any given solve means there's a 50% chance of 3 or more parities in an average of 5. What this means is that 50% of the time, the median solve will be a parity case (and so slower than the cuber's overall average) and half the time it won't (so it will probably faster than the cuber's overall average). This leaves the entire average up to chance, as it's basically a 5-second coin flip.

To give an example, what if two solvers' times in a 4x4 round were as follows:

A: 46, 47, 48, 53(p), 54(p)
B: 44, 45, 50(p), 50(p), 51(p)

Solver B is obviously the faster solver, but because he got 3 parity cases instead of two he is ranked lower. If solver B's third solve had not had parity, it would be closer to 45, putting him 3 seconds ahead instead of 2 behind.


----------



## reThinking the Cube (Aug 19, 2010)

macky said:


> reThinking the Cube said:
> 
> 
> > Originally Posted by Ravi
> ...



Fair enough. You are entitled to our opinion. IMO - maybe YOU are letting your own (I'd say misguided) sense of fairness shape both your suggestions and your arguments, without much basis. That shoe fits you *fairly* better than me anyway.  

The primary reason (basis) I expressed my concern in the above post, was that the definition of "*1.Fairness*" was being established, and I felt that the (I'd say misguided) way it was worded (i.e. "faster times can never hurt you", but - *what about faster outliers not helping you?*) created an exemption in the basic definition, for even considering that extra outliers being included in 5-avg, could even be an issue relating to fairness. This definition directly tied in with (3), and how outliers are to be trimmed *fairly* according to the very definition of 1.Fairness. 

*I think it was *fair* for me to comment, for the sake of trying to achieve a *fair* definition of *fairness*. 

How is this answer NOT *FAIR* TO YOU? 

IMO - using outlier polluted 5-avgs is NOT MORE FAIR than using averages/results/advancements based on some other methods with less outliers and more solves. 

Misguided? 

Can you please show me actual tournament examples where you feel outlier polluted 5-avgs yielded results that were *FAIRER* than using any other possible method?*



macky said:


> Also, regarding what you wrote here (emphasis mine):
> Originally Posted by reThinking the Cube
> For this question, both competitors would also have the 2 slowest medians (14.50, 14.51), and *since B clearly shows the potential to outperform A, I would favor B advancing. Yes, this IS fair.* Using the median value here, would clearly be superior to using the 5-solve average to decide which competitor gets to advance.
> 
> "Potential to outperform" suggests that you want the best cubers to advance. I, and also Pedro from what I've read, believe that the cuber who performs "best" (in some predefined way) _at the competition_ should advance.


Yes, I agree. What we are debating, is the possibility of changing this *predefined way* to score the performances. Either way then, the one who performs best will advance.:tu



reThinking the Cube said:


> For a system to be *perfect*, it would have to trim ALL the outliers (good&bad), and NONE of the others





blade740 said:


> There are a few problems with your idea:
> Taking the median would certainly trim more non-outlier times than the middle 3. Occasionally it would trim an extra bad solve, but for cubers who perform fairly consistently, it would trim average results unnecessarily.



If this is the only problem with my idea, then there is not really a problem at all. For a cuber that performs 100% CONSISTENTLY (every solve is the same time), the median would be EXACTLY THE SAME AS THEIR AVERAGE. Consistent cubers won't LOVE the median as much as inconsistent cubers, but they also should have no good reason to HATE it. Seems pretty *fair* to me, and to argue otherwise would appear to be unnecessarily *misguided*.



reThinking the Cube said:


> *Anytime there is more than 1 good/lucky, or more than 1 bad/unlucky solve, in a round of 5, the arbitrary trimming for 5-avg. yields a result that is polluted.*
> 
> 
> 
> ...



The theoretical distribution based on parity does not actually correlate to real tournament results. I don't know why that is yet, and will not attempt to address this now, so that it doesn't act as a distraction. Besides, the parity problem to be solved first - is simply agreeing on the appropriate# of solves (for deciding higher SD contests), and that should be dealt with seperately. 



Pedro said:


> *Another hard thing to define is what is an outlier. How far should it be from the median/average value to be considered so? I don't have a good answer...*



This is why I have been proposing to use medians for tournament advancement, because then you can confidently get a trim on ALL outliers, without having to worry about this definition. *The median doesn't have to discriminate, like a trimmed average.*


Pedro said:


> Someone who is really consistent may complain about using just the median value.
> For example:
> Competitor A: (11), 11.2, 11.5, 11.6, (11.8)
> Competitor B: (10), 11.3, 11.4, 13.7, (15.8)
> ...



What is wrong with this? No matter which format & definition of *best performance* is used - 5 solves is insufficient to fairly determine with 100% assurity, the *best performer*. To get it right as often as possible, is a reasonable goal. After 2 rounds, you have twice as many solves to work with (my way), and should be better able to determine which competitors should advance, based on their performance. 

I will pose the following (3) questions, so that you might better understand my reasoning here, as this relates to global ranking formats as well.

1) What happens to the difference between (avg/median), as the number of solves , in the average, is increased from 5 to 100 (assuming 20%, or even better yet, all outliers (if they could be identified) trimmed off each side)?

2) Do you agree that the 100-avg will more closely approximate the 100-median for that larger set of solves? 

3) Do you agree that the 100-median will become more and more accurate, as the number of solves is increased, and it is a more reliable reflection of that competitors average ability (neglecting outliers)?

Now take the same 100 solves and break them up into 20 trimmed 5-avg. Many of these 20 will be polluted by more than 1 good, or more than 1 bad, outlier. The average of 20 separate 5-avg will be skewed by some 5-avgs that have been polluted with extra outliers. 

The rather obvious reason that median is better than sets of 5-avg, is that - *the trims that are applied to any 5-avg in an earlier round cannot be redistributed later on, to better eliminate the outliers that are part of ALL the solves that were done by that competitor. Outliers don't have to cooperate with the arbitrary 5-avg format, so that at most, only 2 are present in any one round of 5. The median isn't affected by this lack of outlier cooperation, and that is a very good reason for using it.*


----------



## Pedro (Aug 19, 2010)

reThinking the Cube said:


> What is wrong with this? No matter which format & definition of *best performance* is used - *5 solves is insufficient to fairly determine with 100% assurity, the *best performer*. *To get it right as often as possible, is a reasonable goal. After 2 rounds, you have twice as many solves to work with (my way), and should be better able to determine which competitors should advance, based on their performance.



And is one solve better to determine the best performer?

I still think that [11, 11.2, 11.4, 11.6, 11.8] should surely beat [11.2, 11.3, 11.3, 16.7, 19.8].

We use averages because one single value is not reliable (spell?). I'm not saying 3 values averaged is the perfect way, but I think it's better than just one. 

If the best cuber is the one who can do good solve most of the time, taking the median doesn't show who is the best. In the example above, who has a better probability of getting a good time? I think it's the first one...


----------



## Kirjava (Aug 19, 2010)

You guys are forgetting that the system has to be simple.

I like that the fact that this has become so convoluted that none of you have a solution that is universally accepted. This is another reason why none of the solutions will ever be implemented. 

100+ posts on and you are yet to come up with something better than my original proposal that you all agree with.


----------



## macky (Aug 19, 2010)

Kirjava said:


> You guys are forgetting that the system has to be simple.
> 
> I like that the fact that this has become so convoluted that none of you have a solution that is universally accepted. This is another reason why none of the solutions will ever be implemented.
> 
> 100+ posts on and you are yet to come up with something better than my original proposal that you all agree with.



No one except reThinking the Cube has suggested a new idea. The rest of us are for keeping the current system, having also considered the best average method suggested by you and a few others.



reThinking the Cube said:


> The primary reason (basis) I expressed my concern in the above post, was that the definition of "*1.Fairness*" was being established, and I felt that the (I'd say misguided) way it was worded (i.e. "faster times can never hurt you", but - *what about faster outliers not helping you?*) created an exemption in the basic definition, for even considering that extra outliers being included in 5-avg, could even be an issue relating to fairness. This definition directly tied in with (3), and how outliers are to be trimmed *fairly* according to the very definition of 1.Fairness.



A faster outlier not helping is perfectly consistent with "faster times can never hurt you"--it doesn't help, but it doesn't hurt either. You of all people certainly don't want to replace this with "faster times can only help you." You would then have to count every outlier for computing the average, the exact opposite of your suggestion. Again, I don't think you've understood Ravi. "Faster times can never hurt you" means the following very basic fairness requirement: "Let S be a set of times, and let S' be another set formed by replacing one time in S with a faster time. Then S' is at least as good as S."



reThinking the Cube said:


> For a cuber that performs 100% CONSISTENTLY (every solve is the same time), the median would be EXACTLY THE SAME AS THEIR AVERAGE. Consistent cubers won't LOVE the median as much as inconsistent cubers, but they also should have no good reason to HATE it. Seems pretty *fair* to me, and to argue otherwise would appear to be unnecessarily *misguided*.


It seems like you understand that your system supports a very inconsistent cuber (messes up to 2/5 of solves) over a perfectly consistent cuber whose median is 0.01 worse than this cuber's. Then we can't really do anything to change your mind.


reThinking the Cube said:


> I respect your opinion - "If you mess up twice in a round, well, too bad.", but it doesn't HAVE to be that way.


Yes, there are situations where speedcuber who should have qualified is eliminated because of two unfortunate mistakes. Pedro has already addressed this. We, and I'm sure the majority of speedcubers, understand this and still don't agree with you. A similar consideration came up in 2004 when we decided stop awarding an extra solve for a POP. One argument was that a POP followed by a bad solve that gets trimmed would mean 2 bad solves out of 6, both not counting. Many members in the community thought that 33% inconsistency (less than your %40) was already too much to go unpenalized.

Your system would be great if all mistakes people ever made were timer failures. Competitors should then always perform around their true potential except for errors outside his control, so the median would be the best measure. In reality, most slow times are the competitor's fault to some degree. POPs and even corner twist DNFs are mistakes that can been avoided by better preparing the cube.


----------



## Ravi (Aug 19, 2010)

Pedro said:


> And is one solve better to determine the best performer?
> 
> I still think that [11, 11.2, 11.4, 11.6, 11.8] should surely beat [11.2, 11.3, 11.3, 16.7, 19.8].
> 
> ...



This.

My mean-average proposal is actually very similar to RTC's, except for basically two things:

1. For advancement from the first round, averages are used instead of medians. I believe that averages are more accurate here, as Pedro argued above.

2. In later rounds, the cumulative mean of averages is used instead of the multi-trimmed average. I'm guessing his method is slightly more accurate here because of the (improbable but possible) situation of two outliers/DNFs in the same round, but I'm willing to sacrifice that accuracy margin for a significantly simpler system. In my system, for example, a time that is trimmed at any point is trimmed for good, and one that counts at any point counts for good; these are not always true of RTC's proposal. Also, my system's direct use of averages of 5 makes it highly compatible (see my criteria above) with the current records.

It is also important to remember why we trim averages. If real speedcubers had perfect bell-curve distributions, then there would be no need for trimming--in fact, as my statistics in the original post showed, untrimmed means would be slightly more accurate. But there are two features of real distributions which make trimming necessary. First, real cubers sometimes get DNFs. If we used untrimmed means, a DNF would immediately eliminate the cuber from the competition; a single solve should not have that much power. The second problem is that real cubers have more outliers than there are in bell-curve distributions. I'm not talking about when a 12-second cuber gets a 14 or 15. I'm talking about, well, mostly pops. When a cuber gets a time more than three or four standard deviations away from his/her mean (which in a normal distribution would happen, respectively, once out of 741 or once out of 31547 solves), it is generally fair to dismiss it as an anomaly. Pops are considered forgivable to a point, but there is little excuse for popping more than once within five solves. There's also little excuse for popping more than, say, twice within fifteen solves, so I think if anything we should trim less rather than more.

Also, it is not practical to simply trim everything we consider to be an outlier and nothing else. Some people may have two outliers in a competition, while others (most) may have none. Trimming outliers for some people but not others would obviously favor those who get outliers. It also wouldn't make sense to trim the greatest number of outliers that anybody gets in the competition, because that would allow one person to affect the results of others. That's why we need to set some rule standard rule about trimming outliers. It will be arbitrary, but it's necessary.


----------



## Kirjava (Aug 19, 2010)

macky said:


> No one except reThinking the Cube has suggested a new idea.




I was blinded by the stupid.


----------



## Ravi (Aug 19, 2010)

macky said:


> No one except reThinking the Cube has suggested a new idea.



Um, me?



Kirjava said:


> You guys are forgetting that the system has to be simple.



Not all of us. See the following:



Ravi said:


> 2. *Practicality.* This means, for example, (1) that competitors should not be expected to complete excessively many attempts, nor should organizers be expected to schedule excessive amounts of competing time; (2) that the rules should be simple enough for most speedcubers to understand; and (3) that the results should be reasonably easy to calculate.



My system would be identical to the current format, except that competitors in every round would be ranked based on the mean (or sum; it doesn't matter) of their averages so far. That's it.

And the benefits? My system is simple, fair, significantly more accurate than the current system or any other system proposed so far in this thread (with the possible exception of RTC's), and compatible with all existing records. Compared to RTC's system, it is nearly identical, except it is simpler and more compatible and it avoids medians, which seem to be the primary criticism against that system. Compared to the existing system, it is again nearly identical, except for a single simple change narrowly designed to make results more accurate without requiring any new solves or changing the trimming rules. Simply put, it fulfills all of my criteria better than any other system I have seen.

There, I've demonstrated my system's superiority. Now, instead of just saying "I don't see why we need to change," can someone provide a convincing argument that any other system is as good as (or better than) mine?


----------



## reThinking the Cube (Aug 19, 2010)

Kirjava said:


> You guys are forgetting that the system has to be simple.



What is simpler than taking the median? You don't even need a calculator for that.



Kirjava said:


> I *like* that the fact that this has become so convoluted. that none of you have a solution that is universally accepted. This is another reason why none of the solutions will ever be implemented.



Think about it.



macky said:


> A faster outlier not helping is perfectly consistent with "faster times can never hurt you"--it doesn't help, but it doesn't hurt either.



My point was/is - that 2 or more faster outliers could also help *unfairly*. That was relevant to the definition of 1.Fairness ....and all the rest of axioms. I have always understood your point(s). Do you understand mine?



macky said:


> It seems like you understand that your system supports a very inconsistent cuber (messes up to 2/5 of solves) over a perfectly consistent cuber whose median is 0.01 worse than this cuber's.



OK, let's look at Pedro's 1st attempt to make this case. Please read it carefully.



reThinking the Cube said:


> Pedro said:
> 
> 
> > Suppose this scenario:
> ...



My answer strongly disputes Pedro's concern.

Ok, let's look at Pedro's 2nd attempt to argue the same point, and my answer to that.



reThinking the Cube said:


> Pedro said:
> 
> 
> > Someone who is really consistent may complain about using just the median value.
> ...




Once again, my answer strongly disputes that using the median should be considered any less fair than using outlier polluted arbitrary 5-avg.



Pedro said:


> Someone who is really consistent may complain about using just the median value.
> For example:
> Competitor A: (11), 11.2, 11.5, 11.6, (11.8)
> Competitor B: (10), 11.3, 11.4, 13.7, (15.8)



If the current format was different, then someone who is a really good cuber, could strongly complain about using just the arbitrary 5-avg.

For example:
Competitor A: (11), 11.2, 11.5, 11.6, (11.8) (same as above) [11.43 avg]
Competitor B: (9), 9.3, 11.4, (DNF), 15.8 (i.e. Pedro going-for-gold, and popping on 4th solve) [12.17 avg]

*Which competitor's performance should allow them to advance into the next round, A or B? *


----------



## Kirjava (Aug 19, 2010)

reThinking the Cube said:


> nonsense




om nom nom 

why bother replying when you'll end up ignoring me anyway XD


----------



## reThinking the Cube (Aug 19, 2010)

Kirjava said:


> reThinking the Cube said:
> 
> 
> > nonsense
> ...



LOL. Fair trim, with a simple & practical format. I can't argue with that anyway XD


----------



## Pedro (Aug 19, 2010)

Ravi said:


> My system would be identical to the current format, except that competitors in every round would be ranked based on the mean (or sum; it doesn't matter) of their averages so far. That's it.
> 
> (...)



One problem I can see is that someone who got a bad 2nd round but still advances would have a very small chance of winning.

For example, people get this averages: (suppose a very strong competition)

A. 10.89
B. 11.01
C. 11.20
D. 11.50
E. 11.79
F. 11.93

2nd round:
B. 11.10
C. 11.15
A. 11.30
F. 11.49
E. 11.68
D. 14.30

I'm not sure how you sum/median the averages, but let's say we sum them up and divide by 2.

A: 11.10
B: 11.06
C: 11.18
D: 12.90
E: 11.74
F: 11.71

So competitor D is over a second behid the worst one of the others. Suppose everyone performs 0.5 worse than this median in the final round. D would need a sub-8 average to win. 

Even with a less worse 2nd round (let's say 13.30), he'd still need a sub-9.

(going to read your first post to see what exactly you propose)


----------



## macky (Aug 19, 2010)

reThinking the Cube said:


> macky said:
> 
> 
> > A faster outlier not helping is perfectly consistent with "faster times can never hurt you"--it doesn't help, but it doesn't hurt either.
> ...


No, you still don't understand what I said. Ravi, can you clarify what you meant by your fairness requirements? Did you mean: "Let S be a set of times, and let S' be another set formed by replacing one time in S with a faster time. Then S' is at least as good as S." ? If so, does what RTC is saying here make any sense?



Ravi said:


> macky said:
> 
> 
> > No one except reThinking the Cube has suggested a new idea.
> ...


Sorry, I mean since Kirjava posted his idea / since RTC started on his. I think it's safe to say that most of us (except RTC) agrees on using the average of 5 for the first round. The question is what to do for the subsequent rounds.

In the first couple of pages, I wasn't convinced that your suggestion was better enough than the current system to warrant a change. There were concerns brought up about cubers who go slower but safer in the first rounds, for example; your system is compatible with the average of 5 ranking, but it would certainly change competition strategies. If you already addressed this point, could you repost it? I can't find anything in this thread anymore.


----------



## reThinking the Cube (Aug 20, 2010)

macky said:


> reThinking the Cube said:
> 
> 
> > macky said:
> ...





Ravi said:


> 1. *Fairness.* This means, for example,.....
> <snip>
> 3. *(Limited) forgiveness for outliers.* This means (1) that DNFs and occasional outliers, which are generally considered relatively unimportant, should not affect a competitor's result drastically, *notwithstanding item 2 under "Fairness" above;* but (2) that this forgiveness (ie. trimming) should not be taken to the extent of invalidating significant, near-average solves. A good guideline provided by the current system is (3) that roughly the middle 60% of attempts should count.
> <snip>
> *Please comment on the guidelines above, and I will edit accordingly. I would really like to reach a consensus on this.*



Does it make sense now?


----------



## macky (Aug 20, 2010)

Item 2 under Fairness is " (2) that the rules must apply equally to all competitors." Your point is still irrelevant to any of the Fairness items. Again, you're distorting Ravi's intention. You presented your concern as one involving the Fairness items, not "(Limited) forgiveness for outliers." If you meant to discuss "(Limited) forgiveness for outliers," I now understand why you brought up your concern. But I still disagree because of the arguments Pedro presented.


----------



## reThinking the Cube (Aug 20, 2010)

macky said:


> But I still disagree because of the arguments Pedro presented.



Pedro failed to win the arguments you are choosing to agree with, and you can change your opinion at any time. 
I will trust you to be *fair*.



reThinking the Cube said:


> For example:
> Competitor A: (11), 11.2, 11.5, 11.6, (11.8) (same as above) [11.43 avg]
> Competitor B: (9), 9.3, 11.4, (DNF), 15.8 (i.e. Pedro going-for-gold, and popping on 4th solve) [12.17 avg]
> 
> Which competitor's performance should allow them to advance into the next round, A or B?



Now, here is one from Chris Hardwick that I happen to also agree with:



cmhardw said:


> Kirjava said:
> 
> 
> > I'd rather that the person who set the best average at the comp won, no matter what round it was in.
> ...



Outlier Polluted Averages(OPA) - SUCK!


----------



## Kirjava (Aug 20, 2010)

Personally, I don't have a problem with luck influencing results.


----------



## reThinking the Cube (Aug 20, 2010)

Kirjava said:


> Personally, I don't have a problem with luck influencing results.



Do you have a problem with having just 1 solve per round?


----------



## Kirjava (Aug 20, 2010)

Yes, because that would be dumb.


----------



## hawkmp4 (Aug 20, 2010)

Kirjava said:


> Personally, I don't have a problem with luck influencing results.



Agreed. If someone is able to take advantage of two easy F2Ls or two PLL skips or whatever in an average, that's just part of cubing. Good for them.


----------



## reThinking the Cube (Aug 20, 2010)

Kirjava said:


> Yes, because that would be dumb.



...luck?


----------



## macky (Aug 20, 2010)

rTC, you ignored this part:


macky said:


> Item 2 under Fairness is " (2) that the rules must apply equally to all competitors." Your point is still irrelevant to any of the Fairness items. Again, you're distorting Ravi's intention. You presented your concern as one involving the Fairness items, not "(Limited) forgiveness for outliers." If you meant to discuss "(Limited) forgiveness for outliers," I now understand why you brought up your concern. But I still disagree because of the arguments Pedro presented.



And this:


macky said:


> reThinking the Cube said:
> 
> 
> > For a cuber that performs 100% CONSISTENTLY (every solve is the same time), the median would be EXACTLY THE SAME AS THEIR AVERAGE. Consistent cubers won't LOVE the median as much as inconsistent cubers, but they also should have no good reason to HATE it. Seems pretty *fair* to me, and to argue otherwise would appear to be unnecessarily *misguided*.
> ...


----------



## Kirjava (Aug 20, 2010)

You're implying that I shouldn't have a problem with a round with 1 solve because I don't mind luck influencing results, right?

I thought I told you to stop trolling.


----------



## reThinking the Cube (Aug 20, 2010)

macky said:


> rTC, you ignored this part:



:fp


----------



## macky (Aug 20, 2010)

RTTC DE MACKY = UR TOO GUD DR MAN TOO GUD = 73 CUAGN DE MACKY \SK E E E


----------



## Kirjava (Aug 20, 2010)

I think you broke macky.


----------



## hawkmp4 (Aug 20, 2010)

I lol'd. Poor macky.

Only fools would argue with RTC.


----------



## Pedro (Aug 20, 2010)

Dude, you still failed to argument against my example where a better cuber would be out for 0.01 if we used the median time...is it fair?

A: 10, 10, 11.5, 11.6, 12
B: 11.2, 11.3, 11.4, 15, 20

also, did someone read this?


Pedro said:


> To summarize: someone with a bad 2nd round would have minimal chances to win, in a strong competition (like needing a sub-9 average, dependind on the other people results, if we used Ravi's suggestion of taking the mean of 3 averages to determine the winner.


----------



## reThinking the Cube (Aug 20, 2010)

macky said:


> RTTC DE MACKY = UR TOO GUD DR MAN TOO GUD = 73 CUAGN DE MACKY \SK E E E



Too funny. My eyes starting watering. Reminded me of something -

http://www.youtube.com/watch?v=PbcctWbC8Q0


----------



## hawkmp4 (Aug 20, 2010)

Pedro said:


> Dude, you still failed to argument against my example where a better cuber would be out for 0.01 if we used the median time...is it fair?
> 
> A: 10, 10, 11.5, 11.6, 12
> B: 11.2, 11.3, 11.4, 15, 20
> ...



I read it, and I agree. He may have read it, but he's not interested in intelligent conversation anymore, just trolling.


----------



## Pedro (Aug 20, 2010)

hawkmp4 said:


> I read it, and I agree. He may have read it, but he's not interested in intelligent conversation anymore, just trolling.



No, I don't mean him, I mean the rest (specially Ravi).


----------



## reThinking the Cube (Aug 20, 2010)

hawkmp4 said:


> Pedro said:
> 
> 
> > Dude, you still failed to argument against my example where a better cuber would be out for 0.01 if we used the median time...is it fair?
> ...



No. I asked him my question first, and he has been avoiding answering me. Lighten up please.



reThinking the Cube said:


> For example:
> Competitor A: (11), 11.2, 11.5, 11.6, (11.8) (same as above) [11.43 avg]
> Competitor B: (9), 9.3, 11.4, (DNF), 15.8 (i.e. Pedro going-for-gold, and popping on 4th solve) [12.17 avg]
> 
> *Which competitor's performance should allow them to advance into the next round, A or B? *


----------



## Daniel Wu (Aug 20, 2010)

reThinking the Cube said:


> hawkmp4 said:
> 
> 
> > Pedro said:
> ...


*A* obviously.


----------



## reThinking the Cube (Aug 20, 2010)

rickcube said:


> reThinking the Cube said:
> 
> 
> > For example:
> ...



Why you no likeee B.Pedro?:confused:


----------



## hawkmp4 (Aug 20, 2010)

I agree...A should advance.
At the Boulder 2009 comp, I ended up letting someone borrow my 2H cube, and I used my OH cube for 2H. I popped twice. If I hadn't popped, and instead got a normal solve (even a bad solve for me, 5 seconds slower than my average), I would have advanced to the second round. But I didn't, and I shouldn't have. I made a decision, and it led directly to poor performance. The competition rules shouldn't try to take away the risk I took.

In other cases, it may not be so easy to see...but why did competitor B pop? A combination of recklessness and inaccurate turning and a cube that was loose. It wasn't a random act of God, so to speak. Competitor B did things that led to his poor performance. He took a risk, and he lost the gamble.


----------



## Ravi (Aug 20, 2010)

macky said:


> reThinking the Cube said:
> 
> 
> > macky said:
> ...



Yes, that's what I meant by part 3 of my fairness requirements. (Parts 4 and 5 were more far-reaching but also more vague. Those were meant to carry double meanings: results should of course be ranked from fastest to slowest, but ranking/advancement systems should also be designed to make the best speedcubers most likely to get the best results.) I think RTC's point is that fast outliers, like slow outliers, can potentially provide an inaccurate view of how fast a competitor is. (Fair enough, although the outlier effect comes into play more often and to a greater extent on the opposite end.) His point is somewhat relevant to #4 and #5 under "fairness" in that ranking/advancement schemes should ideally dull the effects of freak outliers in both directions, not just one. I would say it's more directly relevant to the trimming section. IMO, lucky times are a relatively small issue (in fact, I think fast outliers are actually less common than a normal distribution would suggest). I think the main reason to trim fast times is to keep the symmetry with trimmed slow times: if we only trimmed the slow ones, then our averages would be unrealistically fast (and therefore incomparable with past records).

Still, though, RTC is welcome to clarify what he meant.



macky said:


> Item 2 under Fairness is " (2) that the rules must apply equally to all competitors."


Sorry, I inserted another item 2 after typing that. The "item 2" I was referring to there is now item 3, "that faster times can never hurt you and DNFs can never help." The original post is fixed now.



macky said:


> In the first couple of pages, I wasn't convinced that your suggestion was better enough than the current system to warrant a change. There were concerns brought up about cubers who go slower but safer in the first rounds, for example; your system is compatible with the average of 5 ranking, but it would certainly change competition strategies. If you already addressed this point, could you repost it? I can't find anything in this thread anymore.



This is probably the most substantial criticism of the mean-average system I've seen here. I'll try to address some potential effects of the change, both positive and negative, in another post (let's say tomorrow).


----------



## Ravi (Aug 20, 2010)

reThinking the Cube said:


> hawkmp4 said:
> 
> 
> > Pedro said:
> ...



Wait, who may have read what, and who asked whom what question? Pronouns are confusing.

About Pedro's first question, A should of course beat B.

About the second, I think it is absolutely fair for someone who performs poorly in the second round to suffer a lower placement, just as it is if that same person had performed poorly in the final round instead of the second. This is in accordance with my "a solve is a solve is a solve" guideline listed under "symmetry" in post number 103. As I've been saying since the first post of this thread, *averaging more solves gives a greater chance for the best speedcuber to win.* If Competitor D (or whoever) really is the fastest speedcuber in the competition, then there is a very small chance of him/her choking enough to lose under my plan--and a significantly smaller chance than there is under the current system. See for example my calculations in post 23:



Ravi said:


> According to the Mathematica work I discussed in my first post, Rowe had an 86.1% chance to beat Michal and an 82.0% chance to beat John under the current system, compared to 98.5% and 96.7% respectively under my system



Also note that a 14-second average can be enough to keep someone out of the finals in some competitions these days. Under my advancement system, the 14 would average out with the first-round result (11?) and make that competitor place ahead of someone who averaged, for example, 13 and 13. That is fair. But I do not believe averages of 11, 14, and 11 should beat averages of 11, 11, and 12. In my view, the winner should be the person who performed the best in the competition--the *whole* competition.



reThinking the Cube said:


> Suppose this scenario: [1st Round 3x3x3]
> Competitor A gets (13.10), 13.30 ,14.51, 14.58, (14.90) = 14.13 avg (same as above)
> Competitor B gets (9.20), 9.30, 14.50, 19.70, (DNF) = 14.50 avg (strong cuber+bad,bad luck)
> 
> ...



What if A's third solve had been 14.49 instead of 14.51? Then A would beat B in spite of B's "potential to outperform." It seems what you mean by "potential to outperform" here is "best solve (or two)." Under the median system, B does not beat A because of the nines, but because his 14 just happened to be a little faster than A's 14. That's chance. I think we should dull the effects of chance by averaging more.


----------



## Ton (Aug 20, 2010)

Ravi said:


> I have long been of the opinion that a five-solve trimmed average measures a speedcuber’s skill too unreliably to decide the winner of a competition.



My last thought on this topic

If I look at darts, these competitions (the big one) are more like a statistical format, with many rounds and attempt (legs). If cubers become closer and closer in their times at average you might consider other formats. This will always effect the lesser cuber as they will have less attempts or can not compete. 

If you want to distinguish cubers they should compete 1 on 1 and use a format like darts. This can only be in a tournament format

Just my 5ct


----------



## macky (Aug 20, 2010)

Ravi said:


> macky said:
> 
> 
> > Item 2 under Fairness is " (2) that the rules must apply equally to all competitors."
> ...


In that case, RTC, I apologize for the confusion. But then I don't understand why the phrase "notwithstanding item 3" is necessary here at all. Perhaps I'm reading the criteria more strictly than Ravi intended.

Of course, RTC's post #104 still doesn't make sense since he only quotes items (1)-(3) and specifically suggests modifying (3):


reThinking the Cube said:


> Ravi said:
> 
> 
> > 1. Fairness. This means, for example, (1) that the overall result of the competition must be based on the individual results of the competitors and nothing else, (2) that the rules must apply equally to all competitors, and (3) that faster times can never hurt you and DNFs can never help.
> ...


But that was a fault in the (format of the) argument, not in RTC's suggestion itself, and I shouldn't have bothered to point this out. If I start going after small things, he'll defeat me, no question. That's what happened to qq.

Seriously, RTC, I hope you're not trolling us. Because that would be too good.



reThinking the Cube said:


> rickcube said:
> 
> 
> > reThinking the Cube said:
> ...


You could have inferred from our posts that we would answer A. That's why I said in post #128 that you ignored this part:



macky said:


> reThinking the Cube said:
> 
> 
> > For a cuber that performs 100% CONSISTENTLY (every solve is the same time), the median would be EXACTLY THE SAME AS THEIR AVERAGE. Consistent cubers won't LOVE the median as much as inconsistent cubers, but they also should have no good reason to HATE it. Seems pretty *fair* to me, and to argue otherwise would appear to be unnecessarily *misguided*.
> ...


Now please actually address this.


----------



## PhillipEspinoza (Aug 20, 2010)

This is RethinkingTheCube. 

Given the examples by Pedro, and having been used as an example of extreme suckiness myself, I would have to say that I'm in support of the current system.


----------



## Pedro (Aug 20, 2010)

Ravi said:


> Also note that a 14-second average can be enough to keep someone out of the finals in some competitions these days. Under my advancement system, the 14 would average out with the first-round result (11?) and make that competitor place ahead of someone who averaged, for example, 13 and 13. That is fair. But I do not believe averages of 11, 14, and 11 should beat averages of 11, 11, and 12. In my view, the winner should be the person who performed the best in the competition--the *whole* competition.



Yeah, I just made the number up. But even considering a not-that-bad average by D, 13.30, for example (suppose a pop/DNF and a bad lock-up, giving him a 14 time) and A messing up more (getting 11.9), D would still need a sub-10 average to win. If he's not capable at all of doing that, he wouldn't be really motivated in the final.

What I mean is that the mean-of-avgs system can be discouraging for people who get a 'bad' average, since it would pretty much ruin their chances of anything.

Michael/Kirjava idea of best avg in the comp seems not that bad, but I still don't like the fact that in the 1st round, where people may get different (and thus easier/harder) scrambles, a competitor may set an 'impossible' average.

If we were to use this system, I think we should do what I said somewhere here, using the best X results until the 2nd round to define who advances to the final. First round would obviously not change, but people with a good average would 'automatically' qualify.

Or we could do like athletics and have heats with 8 people, with the top-3 from each heat and the 4 other best results advancing


----------



## cmhardw (Aug 20, 2010)

Pedro said:


> Michael/Kirjava idea of best avg in the comp seems not that bad, but I still don't like the fact that in the 1st round, where people may get different (and thus easier/harder) scrambles, a competitor may set an 'impossible' average.



I personally am still a bit hesitant on this idea, because of the approximately 15% chance that at least one competitor will get a, possibly, lucky enough average to skew their abilities and possibly win the competition for that reason, even if s/he *is* the fastest cuber there and would otherwise deserve the win.

With the current system the chance that at least one of 16 Fridrich solvers gets at least two skip solves in their final round average, possibly providing them enough of an advantage to win the competition from this luck alone, would be:
\( 1-(1-0.0033)^{16} \approx 0.05 \)

For the most part I agree with the arguments that the mean of 3 averages system does seem to give the better cubers a higher chance to win the tournament, but it also gives slightly less than top class cubers more of a chance to win the tournament as well, by luck. This could make competitions more exciting by allowing for underdog wins and upsets.

However, I would feel strange about celebrating the following results:
Cuber A 1st round, 2nd round, 3rd round averages: 10.00, 12.50, 13.00 
Mean of 3 averages: 12.17

Cuber B 1st round, 2nd round, 3rd round averages: 12.40, 12.30, 11.90 
Mean of 3 averages: 12.20

Cuber A wins under the mean of averages system, even though I would consider Cuber B to be a "better" cuber under competition conditions. I think such a scenario of averages is not too far fetched. Cuber A, perhaps, had a first round average with 2 skips (both PLL, one OLL and one PLL, both OLL, etc.). Based on Cuber A's global average of, probably, around 12.75 s/he received a significant advantage from the 2 skips over Cuber B who's global average is probably around 12.25 seconds.

Although I understand that luck is a part of cubing, it would seem strange that someone who, potentially, came in last place in the final with a 13.00 (not too unreasonable) could win the entire tournament because of a lucky first round average. Not only this, but Cuber A did progressively worse from round to round, while Cuber B did progressively better. I think doing "progressively" better should be rewarded over doing "progressively" worse when all other factors are otherwise equal. To me, and this is personal opinion, this makes the competition feel more like it brings out the best in each competitor.

I still think the current system, although it might not give the "better" cubers more chances to do well than the mean of averages system, gives the slightly less than top class cubers *fewer* chances to beat the "better" cubers due to luck alone. This is why I am in favor of the current system over the mean of averages system.

Chris


----------



## Kirjava (Aug 20, 2010)

No matter how you try and remove it, luck will always influence results.



Pedro said:


> Michael/Kirjava idea of best avg in the comp seems not that bad, but I still don't like the fact that in the 1st round, where people may get different (and thus easier/harder) scrambles, a competitor may set an 'impossible' average.




People can still have different scrambles in the final, so this problem is not exclusive to the system I propose.


----------



## cmhardw (Aug 20, 2010)

Kirjava said:


> No matter how you try and remove it, luck will always influence results.



Yes, I agree with this. However, I feel that the mean of averages system allows luck to influence the results *moreso* than the current system. This is why I am not in favor of the mean of averages system.

Chris


----------



## ExoCorsair (Aug 20, 2010)

macky said:


> Now please actually address this.



He's just arguing for the sake of arguing at this point. :fp


----------



## Escher (Aug 20, 2010)

The length, breadth and complexity of this argument (these arguments?) should be a point in itself against adopting another method. KISS.


----------



## Kirjava (Aug 20, 2010)

cmhardw said:


> I feel that the mean of averages system allows luck to influence the results *moreso* than the current system.




I actually don't mind this (I don't think luck should be a major consideration at all), I dislike the system for other reasons.


----------



## reThinking the Cube (Aug 21, 2010)

I'm analyzing some tournament results relative to this thread, and will restart my commenting(reTrolling™) a.s.a.p.


----------



## Ravi (Aug 24, 2010)

cmhardw said:


> Cuber A 1st round, 2nd round, 3rd round averages: 10.00, 12.50, 13.00
> 
> [...]
> 
> Cuber A, perhaps, had a first round average with 2 skips (both PLL, one OLL and one PLL, both OLL, etc.). Based on Cuber A's global average of, probably, around 12.75 s/he received a significant advantage from the 2 skips over Cuber B who's global average is probably around 12.25 seconds.



First of all, let's be reasonable. There is no plausible way for a 12-13s performer to get a 10-second average based solely on luck. That would require something like the following:

13.00, 12.00, (5.00), (13.00), 5.00 -> 10.00

Of course, this assumes that the nonlucky solves actually averaged in the 12-13 range. Dropping this assumption, it is possible to conceive a slightly more plausible average like the following:

8.04, (7.89), 11.22, 10.74, (14.45) -> 10.00

but this still strikes me as highly unlikely, and at the very least it would require the cuber to perform much, much better than a 12-13s average would imply--not only on the 10 and 11, but also on the lucky solves, in order to make them that fast.

The question in question is this: *How much can two lucky solves affect an average, or a mean average?* I believe the effect is much less than you imagine.

Suppose, for example, that any given OLL/PLL skip cuts a given speedcuber's times by exactly 2 seconds. (This may be generous or slightly conservative, depending on the speedcuber, OLL/PLL, reaction times, etc.) Then, on the rare occasion when the cuber gets two skips within the same average in competition, we can imagine that his average drops by roughly 0.67 seconds. (In fact, it would sometimes be a smaller difference than this, since the counting lucky time might have otherwise been the slowest solve and therefore dropped--but let's assume 0.67.) Then, assuming there are three rounds in this competition, that speedcuber's mean average would drop by 0.22 seconds.

There's a trade-off here: single averages would give a small chance of a 0.67-second error, while mean averages give three times the chance of a 0.22-second error. (I think we can agree that the probabilities of three lucky solves in an average or two in each of two averages are negligible.) So which side of the trade-off is better? That is, which side gives a greater likelihood of a luck-based upset?

You assumed, quite falsely, that all finalists are fast enough to win with a single counting lucky time. It seems more natural to assume that the probability of being within 0.67 of the lead is roughly three times the probability of being within 0.22. (Actually, I would venture to say that it is somewhat more than three times, since competitors tend to thin out at the head of the pack and bunch up behind.) EDIT: A quick look down the speedcubing.com homepage gives 35 competitions, in which a total of 15 speedcubers were within 0.67 seconds and 3 within 0.22. This is a small sample, but the stats do support my assumption. So although the mean-average system would give competitors three times the chance to get lucky twice, there would most likely be at least three times as many people close enough to win with a 0.67 boost compared to a 0.22 boost. (A rough guess gives an expected value of 15/35 * 0.0033 = 0.00141 competitors passing the leader by luck under the current system, and 3/35 * 0.0099 = 0.00085 under my system.) Therefore I conclude that *winning by luck is at least as unlikely, and probably more unlikely, under the mean average system compared to the single average system*. In any case, the probabilities are on the order of a tenth of a percent, and we would be doing very well indeed if our competitions had anything near 99.9% accuracy.

Furthermore, I claim that lucky times are not only an _extremely_ minor issue in competitions, but that they actually fall well inside a typical cuber's bell curve. For example, in the last few weeks, I've recorded 268 official-rules solves at home. My overall average of 268 is 12.37 seconds, and my SD is 1.70. But of all 268 solves, the best single time is 9.07, _less than two standard deviations_ away from the mean. I don't recall whether the 9.07 was lucky, but there were several lucky times; they just weren't all that far from the mean. Theoretically, roughly 2.2% of times should be over two SDs away from the mean on each side of a bell curve. In my experience, virtually zero (in this case, exactly zero) solves are more than two SDs faster than average*. Conversely, there is much greater variance on the slower end of the distribution. What this means is that *lucky times are a far less important issue than a bell-curve approximation would indicate*.

As my statistics in posts #1 and #23 showed, mean averages are significantly more accurate than single averages assuming normal distributions. As I have argued previously, freak outliers on the slow end rarely affect that accuracy due to their rarity. And as I showed here, lucky solves do not make a difference either. Thus I conclude once again that *mean averages provide a more accurate view of the relative speed of cubers in a competition than single averages do, not only assuming a normal distribution, but even when one considers unusually fast and slow times as well*.




* Please tell me if this does not match your own experience.


----------



## Ravi (Aug 24, 2010)

Ravi said:


> macky said:
> 
> 
> > In the first couple of pages, I wasn't convinced that your suggestion was better enough than the current system to warrant a change. There were concerns brought up about cubers who go slower but safer in the first rounds, for example; your system is compatible with the average of 5 ranking, but it would certainly change competition strategies. If you already addressed this point, could you repost it? I can't find anything in this thread anymore.
> ...



Oops. Later today?


----------



## ~Adam~ (Aug 24, 2010)

I have never been to a competition but it seems that the overall length of the event needs to be kept down but at the end of it you want the best performer on the day to win.

So is there a good reason why the final round/s (idk how many are typical) couldn't/shouldn't be averages of 12?


----------



## Pedro (Aug 25, 2010)

I see your point, Ravi.

I was gonna argument that a not-that-good avg ruins your chances of winning, if you're in a strong competition (11, 11, 11 x 11, 12.01, 10 (sub-10 avg needed to win)). But that sounds reasonable.

Just so I understand...your proposal is only for single competitions? To determine the winner? World rankings would still be avg5? Would we have a separate ranking for average of 15?

That would be hard, because not all competitions have 3 rounds, and some have 4.

Also, qualification into the 3rd and 4th rounds take into account the mean of the 2 or 3 previous avgs?


----------



## Ravi (Aug 25, 2010)

Pedro said:


> Just so I understand...your proposal is only for single competitions? To determine the winner? World rankings would still be avg5? Would we have a separate ranking for average of 15?
> 
> That would be hard, because not all competitions have 3 rounds, and some have 4.



Yes, I agree: it would make much more sense to leave the average rankings as they are. (There could, of course, be informal rankings for averages of 10, 15, 20, or whatever.) Macky started a separate thread on global rankings: http://www.speedsolving.com/forum/showthread.php?t=23261



Pedro said:


> Also, qualification into the 3rd and 4th rounds take into account the mean of the 2 or 3 previous avgs?



That was my intention, unless someone can see a reason not to do so.



Ravi said:


> Ravi said:
> 
> 
> > I'll try to address some potential effects of the change, both positive and negative, in another post (let's say tomorrow).
> ...



aaaaahhhh too busy with cubing, preparing for college, and living life to fulfill my forum promises. Sorry. Let's just say I intend to do justice to this at some point. In the meantime, others are of course welcome to weigh in.


----------



## AvGalen (Sep 13, 2010)

Wow, long read, but I finished reading all of it carefully.

I hope to bring a fresh perspective into this discussion by stating a few points that will hopefully lead to discussion.

*Analysis of the need for change*
I hope we can all agree that the current system is actually working pretty well for both determining who proceeds to another round and for who wins. The most prominent example that has been brought forward and might possibly warrant a change was Breandans WC-winning average.
Back then, many people had tipped of Breandan as the outsider that did really well at home but couldn't handle the competition pressure (yet). His results during the competition were (as expected) relatively unstable, especially when compared with one of the other favorites "steady as a rock Zolnowski".
For all people there, the finals were as exciting as never before and even after everyone had done 3 and 4 solves, the winner had yet to be determined. If Zolnowski (or 1 of at least 3 others that still had a chance) had won, everyone would have thought he was the right person to become the WC. When Breandan turned out to have won everyone there thought he was the right person to win. Why? Because all of them had made it to the WC, through qualification and all other rounds and into the final. Nobody was interested in the results of those pre-final rounds anymore during the final. Breandan won because he was the best under the same circumstances as everyone else and the audience enjoyed the show.
This example alone should be a big warning sign not to change the current way of doing things. 

*Are there theoretical problems with the current system that could be improved?*
Yes. (I was really tempted to keep it that short)
Ravi has posted a very interesting idea and reThinking the Cube proposed an interesting variation. The level of the discussion about luck, mean vs average and even peoples behavior (although distorted with some ego-bumping) was extremely high and convinced me that we should discuss and evaluate a better system untill

*Reality* (kicked me in the rear)
- If you are doing 5 solves in an average and you manage to screw 2 of them up so badly that it influences your results beyond repair you made a tactical error and will have to live with the consequences
- If you are doing 5 solves in an average and you manage to have 2 "undeserved" great solves that not only influence your own results but even change your ranking: Good for you, sucks for others, all in the game
- external circumstances (different scrambles in the same round, lighting-difference, helping with organising, and too many other things to list are a far bigger influence and problem
- Showing of by doing qualification rounds with beginner-method/wooden-cubes/keychain/one-hand is generally appreciated by the audience and enhances the fun-factor of a competition for many people
- Some people are consistant, others have a style or method that allows for greater variation within averages and between averages. This has been long understood by competitors and they have often accepted this as their strength/weakness

*Conclusion*
Keep the current system. The proposed changes are not solving enough real world problems to warrant a change that would influence the way competitors behave and experience during a competition. Dropping best and worst results solves almost all the problems and has been entranched in the community.

*Other considerations*
- 3x3x3 is special, so I understand that most of the discussion is about that, but to test theories it would be better to look at the extremes like 2x2x2 (skips) and magic (dnf)
- 3/5 is the preferred format for most puzzles (now also for megaminx) and I long for the day that it will be available for even more events like 6x6x6, 7x7x7 and 3x3x3 blind (I am serious about that last one, but please don't fill up this already hard to maneuver topic with replies to this)
- If you use a method that gives you big variations in time and you don't like that (4x4x4 reduction -> parities), then improve the method to get rid of the variations (learn all OLL's + parity), switch to a another method, or make up a better method. Another example would be to learn all 1LLL-algs for OLL's that lead to N-Perms (assuming N-Perms are your slowest PLL). Of course, most people just accept the variations in their times as a result of their method
- Even the best boxers walk into a lucky punch, the fastest skaters fall, the best cyclers get a flat tire, etc etc etc. Sport isn't an exact science and (luckily for the audience) it isn't predefined that the best person (on paper) will win in reality


----------



## riffz (Sep 13, 2010)

I have to agree with AVG for the most part.



AvGalen said:


> - 3/5 is the preferred format for most puzzles (now also for megaminx) and I long for the day that it will be available for even more events like 6x6x6, 7x7x7 and 3x3x3 blind (I am serious about that last one, but please don't fill up this already hard to maneuver topic with replies to this)



I know you asked specifically that we don't reply to this, but I just had to.  Expecting competitors to get 4 successes out of 5 attempts in BLD is ridiculous IMO.


----------



## AvGalen (Sep 13, 2010)

riffz said:


> I have to agree with AVG for the most part.
> 
> 
> 
> ...



Yes, because people are totally incapable of solving 24 out of 24 cubes succesfully :fp

But let's continue that discussion here: http://www.speedsolving.com/forum/showthread.php?t=24079


----------

