# Proposal: Judges to verify scrambles



## LucidCuber (Aug 7, 2014)

Given how there have been quite a few incidents, especially high profile incidents of DNF's due to mis-scrambles I don't think it would be unreasonable to require judges to verify the scramble being correct before bringing the puzzle to the competitor. This would add a second layer of verification as the scramblers occasionally do not pick up on mis-scrambles.

It would prevent DNF's and also ensure that less mis-scrambles fall under the radar. It would only take 5 seconds before each solve for a judge to quickly look at the graphic and compare this with the cube. This would also stop incidences of competitors being presented with a solved cube.


----------



## BillyRain (Aug 7, 2014)

Here we go... -.-

This would never work. You say 5 seconds... but really? There are now a bunch of judges falling over themselves trying to look at the paper scramble/laptop screen to check it. Also the scramblers (if using a laptop) will have to keep scrolling to allow people to check. It would just be a logistical nightmare and would prolong rounds IMHO.

The issue is not that we need a second varification.. it's that scramblers need to simply check their scrambles against the image properly, which is clearly not happening.


----------



## Renslay (Aug 7, 2014)

Thinking about it, I wonder why this didn't become the standard procedure from the beginning.

On the other side, I can imagine clumsy judges with the attitutude "yeah-yeah, I'm sure it's fine..."
I don't know how much the the checking procedure would change just because a line in the Regulation saysyou have to check it as a judge too.

Edit:



BillyRain said:


> Here we go... -.-



Where?


----------



## MatejMuzatko (Aug 7, 2014)

I believe on Austrian open 2014, judge (which has judged the same scramble a lot and knew what's there), came back to scrambler and told him to check it... just saying


----------



## LucidCuber (Aug 7, 2014)

BillyRain said:


> This would never work. You say 5 seconds... but really? There are now a bunch of judges falling over themselves trying to look at the paper scramble/laptop screen to check it. Also the scramblers (if using a laptop) will have to keep scrolling to allow people to check. It would just be a logistical nightmare and would prolong rounds IMHO.


If you have the sheet laid out of the scrambling table then a judge would only have to take 10 seconds at most. Although perhaps if we have scramblers going "yeah yeah that's fine" then judges may do the same too.




> The issue is not that we need a second varification.. it's that scramblers need to simply check their scrambles against the image properly, which is clearly not happening.


Of course it would be best, but how do you make scramblers check it properly when it isn't happening.


----------



## BillyRain (Aug 7, 2014)

MatejMuzatko said:


> I believe on Austrian open 2014, judge (which has judged the same scramble a lot and knew what's there), came back to scrambler and told him to check it... just saying



I wonder if he decided to take it back after he had revealed it to the competitor... Quite unlikely that he noticed it while it was in the box..?



LucidCuber said:


> If you have the sheet laid out of the scrambling table then a judge would only have to take 10 seconds at most. Although perhaps if we have scramblers going "yeah yeah that's fine" then judges may do the same too.



Scramble sheets laid out all over the place. That's definitely not risky!



LucidCuber said:


> Of course it would be best, but how do you make scramblers check it properly when it isn't happening.



Like i've always said. Trusted or even Certified scramblers only. 

At many comps I've even seen new cubers scrambling. Even at UKC! This is where the problem lies IMO.


----------



## MatejMuzatko (Aug 7, 2014)

BillyRain said:


> I wonder if he decided to take it back after he had revealed it to the competitor... Quite unlikely that he noticed it while it was in the box..?



I don't know, I was sitting next to a scrambler it was given to and they talked German... 



BillyRain said:


> Like i've always said. Trusted or even Certified scramblers only.


^This.


----------



## LucidCuber (Aug 7, 2014)

Billy, I'm not saying you're wrong (I've changed the title now) but there clearly is a problem with unacceptable levels of mis-scrambling. If we are getting these that only get brought about by video evidence, how many more are flying under the radar? I agree that allowing first time cubers to scramble is not a good idea.


----------



## BillyRain (Aug 7, 2014)

LucidCuber said:


> Billy, I'm not saying you're wrong (I've changed the title now) but there clearly is a problem with unacceptable levels of mis-scrambling. If we are getting these that only get brought about by video evidence, how many more are flying under the radar? I agree that allowing first time cubers to scramble is not a good idea.



I agree it's an issue that needed to be raised. But I feel that we have reached the answer to the problem very quickly. It's glaringly obvious. 

It's just a shame that *in my opinion* it's unlikely the WCA will use their initiative and implement any changes. They obviously haven't recognized it as an issue that needs immediate attention since the first notable mis-scramble DNF.


----------



## Dene (Aug 7, 2014)

BillyRain said:


> It's just a shame that *in my opinion* it's unlikely the WCA will use their initiative and implement any changes. They obviously haven't recognized it as an issue that needs immediate attention since the first notable mis-scramble DNF.



I just want to point out (and I don't speak directly for the Board or the WRC) that this is something considered to be a very big issue, but with no practical solution (as you yourself realise). 

I very recently suggested amongst the delegates if it would be possible to design an app that checks a scrambled cube using the camera on a smartphone. I can only assume this sort of recognition technology is used for cube-solving-robots, so I don't see why it wouldn't be possible for us in cubing competitions. But we'd need someone to take the initiative to work on such an app (and deal with the myriad of problems that come with it).


----------



## bobthegiraffemonkey (Aug 7, 2014)

I feel like trying to emphasise more to scramblers that part of the process is checking the cube is scrambled correctly might help a little. I think added judges would make things more awkward, though I can't think of a better solution.


----------



## BillyRain (Aug 7, 2014)

Dene said:


> I just want to point out (and I don't speak directly for the Board or the WRC) that this is something considered to be a very big issue, but with no practical solution (as you yourself realise).
> 
> I very recently suggested amongst the delegates if it would be possible to design an app that checks a scrambled cube using the camera on a smartphone. I can only assume this sort of recognition technology is used for cube-solving-robots, so I don't see why it wouldn't be possible for us in cubing competitions. But we'd need someone to take the initiative to work on such an app (and deal with the myriad of problems that come with it).



I think that this is an unnecessarily flamboyant move to make at this point. Don't get me wrong, some kind of scanning app would work great in the future, but I don't see why we have to overlook the human error issue when it only comes down to competency. 

Trusted, experienced, certified scramblers only. How difficult can this possibly be to enforce? Not difficult at all in my opinion. However I'm sure somebody will be quick to argue this.


----------



## guysensei1 (Aug 7, 2014)

Ask the people who made the cubestormer robot to make more and sell them to the WCA.


----------



## BillyRain (Aug 7, 2014)

guysensei1 said:


> Ask the people who made the cubestormer robot to make more and sell them to the WCA.



Although cubestormers/robotic scramblers do seem like a perfect solution/awesome idea... think of it realistically. 

Could they scramble 444 and 555 faster than a human?

Would there be a risk of damage to peoples hardware? I personally wouldn't really want my main put into some kind of machine 10-15 times every comp.

I definitely think this will happen in the future as long as the WCA moves with the times/technology. But for now. As I keep stressing, it's just down to competency and having reliable enough people doing the job. 

For christ sake we can solve Rubik's Cubes. Surely we can check some colors against a picture if we're told to!!?!??! I could be wrong.


----------



## guysensei1 (Aug 7, 2014)

I never thought about that...

Question: do the delegates scramble cubes? (I was told in my last comp that the delegate scrambled one of my cubes)


----------



## BillyRain (Aug 7, 2014)

guysensei1 said:


> I never thought about that...
> 
> Question: do the delegates scramble cubes? (I was told in my last comp that the delegate scrambled one of my cubes)



Certainly. If they are available and willing then sure they help scramble sometimes. But definitely not all the time.


----------



## Stefan (Aug 7, 2014)

LucidCuber said:


> there have been quite a few incidents, especially high profile incidents of DNF's due to mis-scrambles



What kind of mis-scrambles? (I'm rather out of the loop at the moment.)

There have been suggestions before, I think including judges checking as well as having pictures of only the eight corner stickers on U and D sides (this should be quick and easy, detect most problems, and not give away much information if people see it who shouldn't). Might have even suggested printing these eight stickers on the back of the score sheet, so the judge could check this way, but I don't remember much of these discussions.


----------



## Dene (Aug 7, 2014)

BillyRain said:


> Trusted, experienced, certified scramblers only. How difficult can this possibly be to enforce? Not difficult at all in my opinion. However I'm sure somebody will be quick to argue this.



In all honesty, it can be difficult. Especially at smaller competitions or inexperienced locations.

Personally I always try to scramble as much as possible, and if not I try to get the best people I can (mostly also because they're fastest). But in Australia this isn't too difficult. In New Zealand on the other hand...


A lot of the issue stems around speed vs. accuracy in the scrambling department. If we could keep the accuracy factor perfect, while maintaining speed, that would be the best option. Hence why I think a quick-scan type app would be really useful.


----------



## DaveyCow (Aug 7, 2014)

MatejMuzatko said:


> I believe on Austrian open 2014, judge (which has judged the same scramble a lot and knew what's there), came back to scrambler and told him to check it... just saying



...and at a recent comp I want to, judge (who happened to also be a delegate - not for that particular comp) came back to scrambler and asked to verify the scramble and the scrambler just paused a second (as if to be rechecking) then said "yes" (obviously didn't recheck it).

I also have witnessed (as am sure we all have) younger scramblers seeming to not take it seriously and just talk while scrambling, obviously not paying much attention.



Renslay said:


> On the other side, I can imagine clumsy judges with the attitutude "yeah-yeah, I'm sure it's fine..."



We almost need an "official" judge (and scramblers - as BillyRain said) ... like someone actually qualified to judge.  This might just be as simple ("simple"? yeah right!) as the delegate checking "qualifications" somehow and not just letting anyone judge that wants to. I have know idea what all this would imply. But we obviously need a better standard.



BillyRain said:


> They obviously haven't recognized it as an issue that needs immediate attention since the first notable mis-scramble DNF.



I hadn't heard of this! linky? I think I'm totally out of the cubing-drama loop, which is sad


----------



## BillyRain (Aug 7, 2014)

Dene said:


> In all honesty, it can be difficult. Especially at smaller competitions or inexperienced locations.
> 
> Personally I always try to scramble as much as possible, and if not I try to get the best people I can (mostly also because they're fastest). But in Australia this isn't too difficult. In New Zealand on the other hand...



If there is a professional standard then it should be a requirement in order to hold WCA competitions. For example: It's an official requirement for an event to have a delegate in attendance in order for it to be official. If you can't find a delegate/don't have a delegate in your country, then you cannot hold an official event. It seems unprofessional to say "Oh this country can't find experienced enough cubers to scramble accurately enough or check scrambles against a picture before they release them... I guess we'll loosen the regs to accommodate this factor". Hmmm no.



Dene said:


> A lot of the issue stems around speed vs. accuracy in the scrambling department. If we could keep the accuracy factor perfect, while maintaining speed, that would be the best option. Hence why I think a quick-scan type app would be really useful.



Speed plays no part in this. 

The sole problem is that the scrambler is not checking the scramble that they have applied with the image. That and only that. So that is the part that needs to be addressed. Why are people not doing it?! There is no excuse.


----------



## MatejMuzatko (Aug 7, 2014)

I liked Stefan's idea to change puzzle pictures. It would make it easier for scramblers.
But then, again, non standard color schemes...


----------



## Pedro (Aug 7, 2014)

I don't think a scanning app would be faster than the scramblers checking...because the app would have to take pictures or video of all sides, in a particular order, and would either need someone to operate or would need scramblers to place cube in there and wait for check.

I see what Dene talks about speed x accuracy. If someone can scramble a 3x3 with no mistakes but takes 1 minute to do so, we probably don't want him/her scrambling at a competition.

But yeah, scramblers need to check if they did it correctly. That's why we have the picture in the first place.


----------



## plusCubed (Aug 7, 2014)

CubeStormers are, as said above, overkill, but using part of that idea, I think it might be plausible to make small, compact robots with motors capable of rotating cubes, connected to a phone to take pictures and check. Maybe that's still a bit far-fetched, just an idea.


----------



## dlo (Aug 7, 2014)

10 seconds * 500 competitors = 1 hour 23 minutes
10 seconds * 100 competitors = 17 minutes

The time required for an extra layer of checking is not insignificant.


----------



## LucidCuber (Aug 7, 2014)

It's an extra 50 seconds per round. assuming each competitor has 5 solves, then that competitor will have an extra 50 seconds per round while the scramble is being checked.


----------



## Kit Clement (Aug 7, 2014)

I don't like adding responsibilities to judges. Currently, being a judge is a simple job that you can have anyone at a competition do, including parents and friends. While checking scrambles isn't hard, it is something that would take quite a bit of extra training, especially if they don't know much about cubing. While checking for colors isn't necessarily hard, it's just another barrier to letting anyone help out and judge, and I'd rather spend my time making sure scramblers are doing their job in the first place.


----------



## kcl (Aug 7, 2014)

I rather like the idea of having pre-qualifications for scramblers. As stupid as it may sound, this could be loosely based on speed. I almost guarantee you a sub 13 solver scrambles correctly almost all of the time. If not that, you could just simply have a delegate verify the judge's scrambling ability (have them scramble a few cubes to prove that they can do it quickly and accurately). I'm in agreement of needing some kind of higher standard because I absolutely hate when I see people lose records due to the scrambler.


----------



## obelisk477 (Aug 7, 2014)

Why not have a separate designated scramble checker per round? Their job would be to act as a filter for all scramblers, and it would be really quick to do once you became familiar with the scramble after 2 or 3 inspections. It would also be much easier this way for it to be someone trusted by the judges, because its consolidated to one person


----------



## Stefan (Aug 7, 2014)

Pedro said:


> I don't think a scanning app would be faster than the scramblers checking...because the app would have to take pictures or video of all sides, in a particular order, and would either need someone to operate or would need scramblers to place cube in there and wait for check.



Two pics from opposite corners, doesn't matter which and in what order, should be enough.

One pic should be enough for almost all cases, too.

So the judge could lift the cover, snap a pic, put the cover back, and the app tells them "scramble 1", ..., "scramble 5" or "invalid", then the judge checks it's the correct number, and after the solve the judge snaps a pic of the timer and the competitor's QR code (instead of signature, and the judge doesn't have to sign, either, as they're logged in), and the result gets automatically entered into the database.


----------



## RicardoRix (Aug 7, 2014)

Image analysis, it's harder than you think.
Everyone has different cameras with different lightings. Yellows look like orange that looks like red.

I tried this to see if I could develop an 'auto reconstruction software' with my webcam. I soon gave up, the telly kept on flickering different light levels, it was enough to make me pack up my bags pretty quick.


----------



## Ranzha (Aug 7, 2014)

To whoever said that speed doesn't play any role in scrambling, you can't be more wrong.

At every competition I've been to, there has been an underlying stress of maintaining optimum efficiency. We don't ask people who don't know the new Clock notation to scramble clock, because that would be inefficient. We don't ask people to scramble 3x3 if there are faster scramblers available. The goal is to keep track of the time and at least attempt to stay with the schedule.
Many times I've been handed 2x2s, done the scramble, and placed them right back in the cover without a second thought because I trust what my hands do, and usually we're pressed for time. The only solution I see to this is allotting more time for each event to allow for proper checking, or recruit more scramblers.


----------



## Lucas Garron (Aug 7, 2014)

We do care about this problem. It's one of the most common problems, and especially bad for BLD.

If we just *say* that scramblers must check the cubes, it won't do very much.
While there are some nice ideas to encourage/require scramblers to check each scramble.

However, there are multiple things that could go wrong during a scramble:

- The scrambler does incorrect moves. (Depending on the move and where it was done, this could easily be caught by checking the picture).
- The scrambler does the incorrect scramble. (This happens frequently, and for this problem it's *not* enough just to tell the scrambler to check the scramble carefully.)
- The scrambler does a corner twist. (To guard against this for cubes, the scrambler needs to look at all 7 visible corners, *and* the hidden corner on the back.)

There are also fun details like the idiosyncrasies of different puzzles, and the sheer number of multi BLD cubes to handle.


I like obelisk477's idea of having a scramble checker (they could also verify whether puzzles are legal!), but most competitions don't have the resources for one. They'd also be very tempted to combine the scramble checker role with a scrambler role (or maybe have two scramblers and have each be the checker for the other), and we' have to be very clear about requirements.


----------



## BillyRain (Aug 7, 2014)

Ranzha V. Emodrach said:


> To whoever said that speed doesn't play any role in scrambling, you can't be more wrong.
> 
> At every competition I've been to, there has been an underlying stress of maintaining optimum efficiency. We don't ask people who don't know the new Clock notation to scramble clock, because that would be inefficient. We don't ask people to scramble 3x3 if there are faster scramblers available. The goal is to keep track of the time and at least attempt to stay with the schedule.
> Many times I've been handed 2x2s, done the scramble, and placed them right back in the cover without a second thought because I trust what my hands do, and usually we're pressed for time. The only solution I see to this is allotting more time for each event to allow for proper checking, or recruit more scramblers.



You have completely missed my point sir. This thread is called "Judges to varify scrambles" not "Scrambling is taking too long". I was dismissing things that are not effecting the issue which is being discussed.


----------



## Dene (Aug 7, 2014)

BillyRain said:


> You have completely missed my point sir. This thread is called "Judges to varify scrambles" not "Scrambling is taking too long". I was dismissing things that are not effecting the issue which is being discussed.



It's naive to look at this as a one-dimensional issue. A cubing competition is a dynamic environment, with multiple factors affecting multiple other factors all at once. If speed is no issue then my idea is definitely the best. If speed is a slight concern then Mr. Pochmann's suggested modification is actually a brilliant idea (although doesn't safeguard against everything).


Personally I like the idea of a "scramble checker". I might try implementing this at ausnats next weekend.


----------



## cubernya (Aug 7, 2014)

Stefan said:


> Two pics from opposite corners, doesn't matter which and in what order, should be enough.
> 
> One pic should be enough for almost all cases, too.
> 
> So the judge could lift the cover, snap a pic, put the cover back, and the app tells them "scramble 1", ..., "scramble 5" or "invalid", then the judge checks it's the correct number, and after the solve the judge snaps a pic of the timer and the competitor's QR code (instead of signature, and the judge doesn't have to sign, either, as they're logged in), and the result gets automatically entered into the database.



I really like this idea. It is just so logical, and *should* be relatively easy to implement.


----------



## TimMc (Aug 8, 2014)

Dene said:


> I might try implementing this at ausnats next weekend.



Sounds like a plan.

We don't really have Judges walking back and forth at competitions. Here are some random thoughts on implementing the Verifier role with Runners:

Runner:

Go to the "IN" section of the scramble table;
Take a scorecard out of a cover and put it on the table;
Take the solved puzzle out of the cover and place it on the scorecard so that it's white/green or light/dark oriented etc;
Go to the "OUT section of the scramble table;
Pick up as many covers/boxes as possible and drop them off at competitor stations;
Pick up solved puzzles/covers on the way back to the scrambling area.

Scambler:

Pick up a puzzle from a scorecard (orient the puzzle correctly);
Look at the scorecard to figure out what solve the competitor is up to and which scramble to apply;
DON'T MOVE THE SCRAMBLE SHEET (OTHER PEOPLE ARE USING IT TOO)
Perform the scramble;
Verify that the scramble is correct;
Move the scorecard and the scrambled puzzle to the "VERIFY" section

Verifier:

Watch the "VERIFY" section for scrambled puzzles;
Check the scramble with your own scramble sheet;
Solve the puzzle and move it back to the "IN" section with the scorecard if the scramble was wrong; <-- or just interrupt the scrambler and get them to solve/fix the mistake (whichever is faster)
Put the puzzle in a cover/box and move it to the "OUT" section for a Runner to collect.

Tim.

EDIT: I don't think judges should check scrambles (beyond "is this scrambled?") with our current setup here Runners are used. Thoughts on scaling for large events?


----------



## Tim Major (Aug 8, 2014)

Basically what TimMc has outlined above seems the only reasonable solution that works short and long term. One extra helper, vastly reducing error.

Nothing more frustrating than missing a PB average due to a bad scramble, when it was actually scrambled wrong.

TimMc's idea just needs decent scramble table space, and a quick briefing to runners, and it doesn't need extra technology. Whilst apps would be great, they would also introduce their own problems


----------



## Ranzha (Aug 8, 2014)

BillyRain said:


> You have completely missed my point sir.



Maybe because your point isn't worth anything.

@TimMc: Scramble checkers are an interesting idea. Given we have a fuller staff at the next competition BASC does, we can try it out. It doesn't slow thing down much because the output rate is more or less the same if accuracy is maintained. Of course, the only sacrifice is time to correct inaccuracies, but that should settle itself in time.


----------



## qqwref (Aug 8, 2014)

I definitely like the idea of a separate scramble verifier, if there is enough manpower. Of course the requirement to be a verifier should be much higher than the requirement to be a scrambler. It should be someone who has good attention to detail (perhaps a BLDer? ) and who has been cubing long enough to have seen many different color schemes and to be decently fast at re-solving the cube if they need to.


----------

