# Judging Certification?



## BillyRain (Aug 17, 2016)

Following the recent controversy at Big Apple 2016:






I have thought to myself for years that there really should be some kind of certification system to allow somebody to be a judge at a WCA competition. The above is an astonishing example of why I think this is needed. It's been made apparent that most of the judges at this competition were actually non-cubers. Some might say "Oh right, well that's ok then that kinda thing is going to happen", whilst I say how in the hell was this situation ever allowed to happen?!? How is it OK to have one non-cuber as a judge, let alone an entire team of them? Granted they were taken through a quick judging demonstration but the problem was that they lacked knowledge of the basic fundamentals of cubing.

It makes me realise that even new cubers at their first competition, are basically allowed to not bother to attend the "introduction to judging" session (if there is one, which a lot of times there isn't) and then just stroll up to a table and start judging. Nobody at all has checked if they know what they are doing and this in turn leads to the easy possibility of bad/incorrect results which could be polluting the WCA database on an almost daily basis.

I'm aware that the immediate problem with a certification system is the fact that sometimes it's hard enough to get people to judge at a competition and there would then be the problem that when calling for them, you could just get a load of people saying "I'm not certified", then end up with even less judges available than usual. This is obviously not ideal, but perhaps this is something everyone should have to go through before they even compete.

How about this:

1. Make holding "Introduction to competing/judging" sessions absolutely compulsory at all WCA competitions.

2. Make it compulsory for ALL new competitors to attend the session and tick them off against the competitor list to show that they attended (take a very quick register).

3. Aside from competitions, experienced competitors may be awarded the certification at the discretion of the respective delegate (ie, he/she will know them well and know that they are competent).

EDIT: Alternatively, how about a well put together online exam which people can take at home and at their leisure before competitions? Even if they cheat (google the answers), at least it still means they will learn as they go...?

4. An indicator is added to the competitors WCA profile to show weather or not they have attended an introduction session (WCA Certified Judge).

Obviously this would take a hefty level of administration to get it started, but would that not be better than potentially have results like this go into the database completely unnoticed? What if he never uploaded that video? Respectful results or not, they are incorrect and would have gone into the database causing further pollution.

We put so much time into plugging loopholes in our regulations, but is this not one that is being hugely overlooked???

Maybe it's a crazy idea, but i'd love to hear peoples thoughts.

Thanks.


----------



## SweetSolver (Aug 17, 2016)

Yeah something like this would definitely help. I've had some pretty questionable judges in the past and you're right, a lot of them have no idea. It wouldn't be easy to implement however it would prevent so many complications. The compulsory introduction session sounds like a pretty good idea. In Australia we always hold a short demonstration for new competitors although I can't recall judging being part of it (someone correct me if I'm wrong).


----------



## DJ4Y (Aug 17, 2016)

A fantastic idea, Billy which I think has been long overlooked. Although the concept would be somewhat hard to implement, I believe that the end result will be well worth it. I really hope this idea is taken into consideration by the WCA.


----------



## mark49152 (Aug 17, 2016)

Maybe a simpler solution would be to not allow someone to judge unless they have been to a comp before, and got an average in the event they will be judging. At least that would show they have knowledge of the puzzle, experience of competition and have had the opportunity to see how a more experienced judge worked.

I agree some additional guidance would help too though. From what I've seen, less experienced judges seem OK with the basics but sometimes don't know how to deal with incidents, like they won't know how to write down a +2 properly.


----------



## 1973486 (Aug 17, 2016)

BillyRain said:


> 2. Make it compulsory for ALL new competitors to attend the session and tick them off against the competitor list to show that they attended (take a very quick register).



What if they arrive to the venue late?


----------



## BillyRain (Aug 17, 2016)

1973486 said:


> What if they arrive to the venue late?



A fair point, and I'm sure there are many more that would need to be considered.

Perhaps it would have to mean that they are not allowed to judge at that competition and are told to be sure to get to the next one on time to attend the session. Or just that they won't judge that day and they could attend the session for the following day (if a two day comp). You would have to have a session each day to cater for new competitors only coming for one event etc.


----------



## Dene (Aug 17, 2016)

DJ4Y said:


> A fantastic idea, Billy which I think has been long overlooked. Although the concept would be somewhat hard to implement, I believe that the end result will be well worth it. I really hope this idea is taken into consideration by the WCA.



It definitely hasn't been overlooked, there are just way too many practical issues with it at the moment. The biggest issue being we completely rely on volunteers and most places in the world don't have a dedicated staff.


----------



## BillyRain (Aug 17, 2016)

Dene said:


> It definitely hasn't been overlooked, there are just way too many practical issues with it at the moment. The biggest issue being we completely rely on volunteers and most places in the world don't have a dedicated staff.



I hear ya. It's weird as different countries work in different ways.

In the UK we rely solely on competitors to judge when they are not competing.


----------



## LostGent (Aug 17, 2016)

This is a really good idea. Means a lot of work though for the organisers so I can see why it hasn't been implemented in the past.


----------



## DGCubes (Aug 17, 2016)

I do completely agree that we should not ever have non-certified judges. Although there are a couple points I'd like to address:



BillyRain said:


> 1. Make holding "Introduction to competing/judging" sessions absolutely compulsory at all WCA competitions.
> 
> 2. Make it compulsory for ALL new competitors to attend the session and tick them off against the competitor list to show that they attended (take a very quick register).



Like 1973486 said, arriving late is extremely common. At my first two comps, I only showed up for the events I did. To this day, driving often takes longer than expected for me and I miss my first event or get there in the middle of it. Not to mention that not every competition has the time to hold one of these sessions. If it's packed with events, an extra 10 minutes for this at the beginning could mess up the rest of the day.

Also, I do not feel that it should be made compulsory for every new competitor to go through this, nor do I feel that it should be compulsory for every judge to be certified. These requirements make competitions less accessible for parts of the world that are new to cubing.

I think fair regulations for this would be something like:
All judges must be WCA certified unless there are not enough WCA certified judges present to allow the competition to run smoothly/at all.
In the case that there are not enough certified judges:
1. Non-certified judge(s) will be allowed to judge under the discretion of the delegate.
2. They will be given a quick run-down of judging rules, and will have a sheet of judge-specific regulations that they must follow. // Perhaps the WCA should look into making something like this that encompasses more than +2s and DNFs (e.g. corner twists, cutoffs, etc.).
3. The delegate will monitor their judging to the best of his/her ability.
4. If they are found to judge poorly, they will be replaced as soon as possible, and a ban may be placed on their certification status. // I think it would be a good idea for non-certified judges to be able to have a ban on their judging certification alone and not their ability to attend competitions. Based on the severity, it could be any length of time, but after it is up, they would likely have to prove that they know what they're doing if they want to judge again. While they have a ban, they would not be able to judge at all as a non-certified judge.
5. If they are found to judge fairly, they may be WCA-certified.

One other thing I realized is that not every fair judge necessarily has a WCA profile. I think it'd be kinda cool to have actual judge certificates that you can bring to comps with you (maybe they would have to be renewed every couple of years to make sure people still know what they're doing).

Also, it would be hard to prevent cheating on this, but we could have an online multiple-choice test that allows people to be certified. I'm thinking along the lines of a driving-type thing, where you get a permit from this test (which I suppose could give you less judging abilities), and then you only get certified when a delegate sees you judging and ensures that it's fair. Pretty much, we'd have four types of people: banned (cannot judge at all), non-certified (can judge under certain circumstances), half-certified (took the test, but still needs to be approved before they are 100% trusted), and certified (can judge whenever they want).


----------



## BillyRain (Aug 17, 2016)

DGCubes said:


> Also, it would be hard to prevent cheating on this, but we could have an online multiple-choice test that allows people to be certified.



You've actually raised a really good idea there, regardless of if they cheat (google the questions), at least they will learn the answer in order to input it! 

An online exam could be the answer here!


----------



## AlexMaass (Aug 17, 2016)

maybe i should just make a good judge tutorial, that is required for all judges to watch, and also make a quiz so we can verify the judge knows what they are doing


----------



## Matt11111 (Aug 17, 2016)

I think this is a fantastic idea. Here's how I think it should go.

1. You may only judge if you know how to solve a Rubik's Cube.
2. You may only judge if you have been to at least one competition before. That way, you have some kind of idea of how competitions are run.
3. When that person signs up for their second competition, they are instructed to show up as early as possible in order to attend an instructional session for judging.

What if we paired a new judge with a veteran judge so they can have a good idea of how judging works without risking messing up and making unfair decisions.

Or we could have a guy solve at the end of an instructional session and they purposely violate one of the regulations to see if the new judges can spot it.


----------



## BillyRain (Aug 17, 2016)

I honestly think a well put together ~10 min online exam could solve this issue.

People can do it at their leisure before the comp and when they submit their details, assuming they pass, their details are added to a database which we can either feed into the WCA profiles or issue a certification card or something.

Seems so simple now... I may start developing something to propose.


----------



## Matt11111 (Aug 17, 2016)

BillyRain said:


> I honestly think a well put together ~10 min online exam could solve this issue.
> 
> People can do it at their leisure before the comp and when they submit their details, assuming they pass, their details are added to a database which we can either feed into the WCA profiles or issue a certification card or something.
> 
> Seems so simple now... I may start developing something to propose.


Well, you're getting a ton of ideas to work with, so a proposal shouldn't be too far away.


----------



## turtwig (Aug 17, 2016)

Like Dene said, I think the biggest problem is that sometimes not enough cubers volunteer so they need non-cubers.
First of all, they should try to get as many cubers as possible. Faster, veteran competitors should take priority over first time competitors who take priority over non-cubers. I don't know about other comps, but at one of the comps I went to, all the kids there were eager to judge, so all the judges were cubers. There are always competitors that aren't competing that can judge so if us cubers volunteered to judge more we could greatly lower the amount of non-cuber judges.
Second, they should definitely inform the newer cubers and non-cubers about basic rules, such as +2s and DNF. The judges at Big Apple 2016 and other comps could've easily been informed about the rules for corner twisting and edge flipping and this would've happened (honestly those judges were clueless, I thought even non-cubers know that corner twisting is cheating).


----------



## Goosly (Aug 17, 2016)

I think you're trying way too hard to fix a problem that isn't a problem.

Apparently something went wrong at one competition (I haven't seen the video), so the delegates/organizers over there should prevent that happens again, but one (big?) incident shouldn't require all other delegates/organizers all over the world to suddenly start "certifying" their judges or whatever you want to propose, when they've been doing everything (almost) perfectly so far.


----------



## BillyRain (Aug 17, 2016)

Goosly said:


> I think you're trying way too hard to fix a problem that isn't a problem.
> 
> Apparently something went wrong at one competition (I haven't seen the video), so the delegates/organizers over there should prevent that happens again, but one (big?) incident shouldn't require all other delegates/organizers all over the world to suddenly start "certifying" their judges or whatever you want to propose, when they've been doing everything (almost) perfectly so far.



Appreciate your opinion dude.

I am not just referring to one competition here, I used it as a good example. There have been many incidents like this over the years. Also the whole idea of this is to make it so that things are done as perfectly as possible, rather than "(almost) perfectly".


----------



## AlphaSheep (Aug 17, 2016)

Things that need to be considered:

I don't know about the rest of the world but we already have shortages of volunteers for judges in final rounds or at small comps. I continuously have to call for more volunteers. 

Non-competitors can make good judges. I can think of a couple of parents/partners that have a pretty good grasp of the regulations that have judged at competitions I've been to, and done it well.
You still get judges who know the regulations perfectly well but are too lazy to do anything when they're broken, or just don't pay attention to notice anything.
We generally have a combined competing + judging demo with a run through of common regulations at all of our competitions (I thought this was almost standard practice?) It helps a little but not everyone is there or pays attention so judges still get things wrong. 

At my last two comps, we printed and laminated some A4 sheets with some common judging errors and instructions on how to write penalties. That too helped a little.

Whats absolutely vital is for experienced judges (or even experienced competitors who aren't judging) to know that they should also be watching for mistakes by inexperienced judges, and that its OK for them to step in and correct them. Delegates do their best, but when one delegate has to watch a dozen solves at once and still keep an eye on the scramblers, and waiting area and all the rest, it helps tremendously if others step in and help.

I've seen a few cases, both in videos from overseas comps, and at comps I've attended where an inexperienced judge does something wrong in front of an experienced competitor, and the experienced competitor doesn't tell the judge he's done something wrong.


----------



## Umm Roux? (Aug 17, 2016)

There's usually enough cubers but they just do not want to judge right? If so, we can just place on the registration form"You may be asked to assist in judging during the competition" on the registration forms.


----------



## ender9994 (Aug 17, 2016)

Umm Roux? said:


> There's usually enough cubers but they just do not want to judge right? If so, we can just place on the registration form"You may be asked to assist in judging during the competition" on the registration forms.




Is that no longer commonplace? I remember all of my competitions when I first started, saying something along the lines of "all competitors are expected to be available for judging if needed" .


----------



## BillyRain (Aug 17, 2016)

Whether it's stated or not it doesn't really help. This is also a reason why I've always believed there should be a system by which all competitors are assigned times to judge. If people don't want to judge, they won't and they will be stubborn when called. Having a scheduled time/event to judge would also help with the fact that fast/famous people are generally (in my experience) never asked to judge and you can usually just find them warming up somewhere with other fast cubers. On occasion they tend to seem immune to judging duties! I'm not saying all the time though... and don't get me wrong there are a lot of fast cubers who do judge.


----------



## AlphaSheep (Aug 17, 2016)

Yeah, all competitors are required to judge but if you've got a competition with 30 people, of which 20 are little kids or first timers and 8 of the 10 experienced competitors are in the finals... Who judges?


----------



## GenTheThief (Aug 17, 2016)

At my first competition several months ago, I was horrified I would mess something up at the comp, so read the regulations at least five times. I think it would suck to have done all that work and then be required to show up early and go through a tutorial on stuff I had already memorized or not be allowed to judge because of my lack of practical experience.
I don't think a pre-competition tutorial is completely necessary, but I like the idea of certification via a quiz or a video where a solve is done and you have to write out what would be expected at a real comp, sort of like virtual judging.

What I do think should be mandatory would be some guideline at every judging station that states all the things that are supposed to happen in a speed/blind solve, and what to do if a rule is broken.
I'm pretty sure Spontaneous Summer did this to allow for parents and younger siblings to earn raffle points at the end of the day.

Also, at my most recent comp, I judged 4BLD. This was my first time judging BLD of any sort, so I asked a few questions about holding a thin opaque object in front of the competitor, and actually indirectly reminded the delegate that they hadn't been passed out yet. I ended up judging three DNFs, and wrote them down accordingly, and one success, and wrote it down accordingly.
Just because it was my first time experiencing something, doesn't mean I didn't know what I was doing.


----------



## turtwig (Aug 17, 2016)

AlphaSheep said:


> Yeah, all competitors are required to judge but if you've got a competition with 30 people, of which 20 are little kids or first timers and 8 of the 10 experienced competitors are in the finals... Who judges?



Well the cubers would still be better than non-cubers.


----------



## Umm Roux? (Aug 17, 2016)

turtwig said:


> Well the cubers would still be better than non-cubers.


Yeah, what's the worse a fellow competitor woul do, shout 8 SECOOONDS. Also, if there are only that few competitors, the delegate or someone of high authority could monitor.


----------



## Rcuber123 (Aug 17, 2016)

I was just at a comp and I had a couple of judge fails:

In Pyra finals the judge started turning the Pyra I warmed up with during inspection and popped it.

In BLD the judge wrote down my time but didn't write that I got a DNF.

In 2x2 finals in my last solve after already securing the NR average the judge shouted 2x2 NR before lifting the cover and everyone looked at me. I DNFed with everybody looking. Some ppl were really confused because they thought my judge was talking about NR single.

In 3x3 a lot of judges asked me what my main is a second before lifting the cover. This was super annoying.

So I think there needs to be something done to fix inexperienced judge problems...


----------



## turtwig (Aug 17, 2016)

Rcuber123 said:


> I was just at a comp and I had a couple of judge fails:
> 
> In Pyra finals the judge started turning the Pyra I warmed up with during inspection and popped it.
> 
> ...



I've never had these problems but I can imagine how annoying it would be. I think judges should try to talk to competitors as little as possible. When I judge, I just say what I have to and never talk when the competitor is solving except to my closest friends because you don't know what'll affect the concentration they need.


----------



## JustinTimeCuber (Aug 17, 2016)

I've had a stupid moment or two when I was judging (for instance at Music City and if you were the person I judged then you know what I'm talking about, and knocking over someone's clock twice in a row at my second comp), and I'm sure most people do, but non-cubers judging is just imo not the best idea.* We need judges who know what they're doing, and actually give people DNFs when they blatantly cheat.

*At some small comps (which I think attract fewer nublets) I think non-cubers may be able to be judges.


----------



## BillyRain (Aug 17, 2016)

I'll keep saying it, I think the online exam would be excellent because instead of keeping on saying in pre-comp emails that everyone must be familiar with/read the regs before they attend and them just not reading it, it would force them to have to take the test and therefore will be reminded of/taught the important factors of judging and competing. 

Hell, even parents could take it if they wanna judge! Doesn't even have to be confined to people with a WCA profile. If they pass it, they can judge! It doesn't have to be super hard. No harder than watching an introduction session at a comp and it would also just be multiple choice for automatic data entry/processing purposes.

It could quite easily be made mobile friendly too.. if someone arrives at the comp and can't judge because they aren't certified.. they could sit there for 10 mins and do it on their phone!! 

Sounds crazy but I honestly think it could really work.


----------



## biscuit (Aug 17, 2016)

BillyRain said:


> I'll keep saying it, I think the online exam would be excellent because instead of keeping on saying in pre-comp emails that everyone must be familiar with/read the regs before they attend and them just not reading it, it would force them to have to take the test and therefore will be reminded of/taught the important factors of judging and competing.
> 
> Hell, even parents could take it if they wanna judge! Doesn't even have to be confined to people with a WCA profile. If they pass it, they can judge! It doesn't have to be super hard. No harder than watching an introduction session at a comp and it would also just be multiple choice for automatic data entry/processing purposes.
> 
> ...



This is the best idea IMO. It wouldn't take to long to develop, the hardest part would be integrating with the WCA profiles.


----------



## BillyRain (Aug 17, 2016)

biscuit said:


> This is the best idea IMO. It wouldn't take to long to develop, the hardest part would be integrating with the WCA profiles.



Not necessarily. If we worked with the guys who look after the website and database it probably wouldn't be that hard.

If there are issues with that, we could just keep it as a separate entity and could issue some kind of certificate? I dunno.. maybe not.. but we'd think of something


----------



## biscuit (Aug 17, 2016)

BillyRain said:


> Not necessarily. If we worked with the guys who look after the website and database it probably wouldn't be that hard.
> 
> If there are issues with that, we could just keep it as a separate entity and could issue some kind of certificate? I dunno.. maybe not.. but we'd think of something



Oh no, I'm not saying it'd be that hard, but the quiz is as easy as creating a form and then submitting to the DB with a username*. In fact, I could go make it now. I'd be willing to mock something up if we come up with a set of questions

*okay, yes there's form validation and displaying the results, but that's hardly an issue

Edit: Honestly, we could probably just find something for this out there, or use google forms


----------



## turtwig (Aug 17, 2016)

There could be a list of certified judges, i. e. people who passed the test. If you want to judge, you give your name and the organizers can check if you're on the list, if not you can take a few minutes to complete the quiz and have your name added to the list. I guess that if you show that you're a bad judge you could get your name removed until you redeem yourself again.


----------



## BillyRain (Aug 17, 2016)

turtwig said:


> There could be a list of certified judges, i. e. people who passed the test. If you want to judge, you give your name and the organizers can check if you're on the list, if not you can take a few minutes to complete the quiz and have your name added to the list. I guess that if you show that you're a bad judge you could get your name removed until you redeem yourself again.



Excellent idea. Keeps it simple and is quite effective.

I think I'll start knocking something together tomorrow


----------



## biscuit (Aug 17, 2016)

BillyRain said:


> Excellent idea. Keeps it simple and is quite effective.
> 
> I think I'll start knocking something together tomorrow



Honestly, just put 20 questions in a google form, and link it to a google doc. We can share the link with delegates and organizers. I can bring it up with my delegate at/before the next comp. If we get it together quick enough, I'll even send it out to the competitors going to my comp next month.

I'll post a couple of questions I think would be good to add, and then we can discus them.


----------



## Cale S (Aug 17, 2016)

I disagree with the idea that non-cubers shouldn't be allowed to judge, there are many parents who don't cube themselves but consistently go to cube comps and know how to judge correctly. I like the idea of having a certification to judge though.


----------



## BillyRain (Aug 17, 2016)

biscuit said:


> Honestly, just put 20 questions in a google form, and link it to a google doc.
> 
> I'll post a couple of questions I think would be good to add, and then we can discus them.



We could even kill two birds with one stone and have two modules. One for competing and one for judging!

My girlfriend raised a good point which was how can we check that everyone who is judging is on the list at a comp. 

I think that if it's made common knowledge that you are not supposed to judge if your name is not on the list, people won't do it because they won't want to get called out on it.

Thoughts?



Cale S said:


> I disagree with the idea that non-cubers shouldn't be allowed to judge, there are many parents who don't cube themselves but consistently go to cube comps and know how to judge correctly. I like the idea of having a certification to judge though.



It was never about non-cubers not being allowed to judge. As long as they understand the rules of judging and are familiar with the regs, there is no reason why they can't. The ones at Big Apple clearly did not.

This system would cater for anybody who wanted to take it, cuber or not, it would ensure that they are certified to do the job.


----------



## biscuit (Aug 17, 2016)

BillyRain said:


> We could even kill two birds with one stone and have two modules. One for competing and one for judging!
> 
> My girlfriend raised a good point which was how can we check that everyone who is judging is on the list at a comp.
> 
> ...



Plus, you can actually have google forms set up as a quiz (With automatic Grading!). If you come up with questions (I'll contribute too) I'll work on the form. It'll also be pretty easy to sort the results using google docs (A.K.A what score they got). We could also have them enter a country (and then even further specify to a state/region) to make it able to be sorted by delegates and organizers.

Who should be able to edit this list? Just delegates I'd guess?

I'd also like to suggest that we split the form into a couple of sections, covering a couple articles at a time (Article A would probably be it's own, but you may have all the other Lettered ones - z in one section etc)


----------



## bobthegiraffemonkey (Aug 17, 2016)

If something practical gets worked out I'd be happy. I had a 5BLD judge once who apparently didn't know how BLD worked and thought only inspection was timed, so I went over 10 minutes but didn't know how long the attempt was. If the cube was actually solved that would have been a sticky situation (DNF by 2 centers iirc, CHJ was freaking out until I showed him the back of the cube). Some sort of quick check for judging knowledge and summary of correct procedure would have stopped this from happening.

Sure, issues might not happen often as far as I can tell, and it's impossible to completely eliminate problems, but if something sensible can be implemented which will help to reduce judge failures then it will go some way to improving competitions for everyone.


----------



## Lucas Garron (Aug 17, 2016)

BillyRain said:


> I think that if it's made common knowledge that you are not supposed to judge if your name is not on the list, people won't do it because they won't want to get called out on it.



Don't rely on common knowledge. Don't rely on volunteer, best-effort fixes.

If you want to get rid of this problem, you need to make sure that there are rules and incentives in place.

One problem right now is that there are no repercussions or explicit feedback cycles for failing to do your job as a judge/scrambler/whatever. They're volunteers, and often would probably prefer not to judge, so taking them off the job is actually the *opposite* incentive of what we want.
If the judge makes a mistake and it's too late to give an extra scramble, we usually rule in favor of the competitors (assuming the competitor didn't break the Regs themselves). The Regs literally don't have a mechanism to deal with this, and Delegates sort of just have to use their judgment to deal with incidents at hand and then remind people to do better.

I've tried to push for mandatory scramble checkers to deal with the problem of wrong scrambles (even if that means fewer events), but that didn't go far. Some competitions have used them. But even after egregious issues in some places, those places were not required to use scramble checkers in the future.

So, this is a hard problem, and you can't solve it using good intentions and hope.
(Which is sad, because the WCA spirit serves us well for most things.)


----------



## biscuit (Aug 18, 2016)

It think that, like many other suggestions, we should have this tried at competitions and see how it works. Like I said, I'm willing to try it with my competition.


----------



## DGCubes (Aug 18, 2016)

BillyRain said:


> I honestly think a well put together ~10 min online exam could solve this issue.
> 
> People can do it at their leisure before the comp and when they submit their details, assuming they pass, their details are added to a database which we can either feed into the WCA profiles or issue a certification card or something.
> 
> Seems so simple now... I may start developing something to propose.



Hey, I started working on one today on Google forms. If you want me to share it with you so you can work on it too just let me know what e-mail to use.


----------



## Ranzha (Aug 18, 2016)

At the most recent Stanford competition, Noah Arthurs instructed competitors to replace their judge at the timing station after completing their last attempt of a round. He provided every station with a sheet of paper with introductory content regarding judging.

I transcribed this sheet, making some additions, and Christian Pizzasegola and I worked to translate the sheet into Latin American Spanish.

The sheet: https://docs.google.com/document/d/1VyjqUM0vjld4S-mzoQRbwZhoLAfZCR1TIKU5nAgsUTU/edit?usp=sharing

Providing this introductory material, as well as providing live demonstrations to new judges (in the same way many competitions provide live demonstrations to new competitors) can help new judges with their jobs, and give new judges no excuses for not enforcing procedures in the material covered.


----------



## biscuit (Aug 18, 2016)

Ranzha said:


> At the most recent Stanford competition, Noah Arthurs instructed competitors to replace their judge at the timing station after completing their last attempt of a round. He provided every station with a sheet of paper with introductory content regarding judging.
> 
> I transcribed this sheet, making some additions, and Christian Pizzasegola and I worked to translate the sheet into Latin American Spanish.
> 
> ...



I've tried to write something like this up, but this is great! Will definitely use!


----------



## Matt11111 (Aug 18, 2016)

Ranzha said:


> At the most recent Stanford competition, Noah Arthurs instructed competitors to replace their judge at the timing station after completing their last attempt of a round. He provided every station with a sheet of paper with introductory content regarding judging.
> 
> I transcribed this sheet, making some additions, and Christian Pizzasegola and I worked to translate the sheet into Latin American Spanish.
> 
> ...


Very nice. I think it would be cool if I made a video with a friend of mine showing how to judge.


----------



## BillyRain (Aug 18, 2016)

Lots of good comments, but some peeps are missing the point. It's all well and good providing material (sheets of paper, videos) to people so that they can learn how to judge, but this is about ensuring that this material has been effective enough to trust them with authorising data which is to go into the WCA database forever. 

You can provide as much literature as you want but some people just won't lap it up or even try. The online certification test would ensure that it has at least gone through their head once.


----------



## mark49152 (Aug 18, 2016)

The challenge is enforcing it. When they call for judges and a bunch of people crowd round the scrambling table to take cubes off to judge, who is going to check they are all on the list? 

I still think it would be simpler to require by default that judges have past competition experience in that event. That info is on the record anyway. Others can be permitted to judge at the delegate's discretion.

If the delegate, organisers or other experienced competitors spot someone unfamiliar judging, they should check on them and make sure they know what they are doing. Basically that is what will happen anyway, since nobody is going to check up on the familiar faces. The only difference made by the proposal in this thread is whether you check on them by watching them and talking to them, or by asking them their name and checking if they are on a list. My money is on the list never getting checked even if it exists.


----------



## Goosly (Aug 18, 2016)

mark49152 said:


> I still think it would be simpler to require by default that judges have past competition experience in that event. That info is on the record anyway. Others can be permitted to judge at the delegate's discretion.



I will not ask anyone for permission to judge megaminx just because I've never competed in it before.



mark49152 said:


> If the delegate, organisers or other experienced competitors spot someone unfamiliar judging, they should check on them and make sure they know what they are doing.



Isn't this what happens already anyway? It's what I do at least.


----------



## mark49152 (Aug 18, 2016)

Goosly said:


> I will not ask anyone for permission to judge megaminx just because I've never competed in it before.


You are an experienced competitor who would no doubt be trusted by the delegate, but regardless, if the delegate didn't know you and suspected you didn't know how to solve a mega and wouldn't recognise what is and isn't a +2 it would be reasonable for them to have right of veto over you judging it. Which they do anyway.

Clock is a better example. I have judged clock when we were short of judges but I have no idea how to solve one. The delegate gave me some guidance including what the solved state looks like and was OK with me judging, but I would have been fine had they said no I'm not qualified.



Goosly said:


> Isn't this what happens already anyway? It's what I do at least.


Yes certainly it should. I'm not sure it always does. Obviously it didn't at Big Apple.


----------



## BillyRain (Aug 18, 2016)

mark49152 said:


> Clock is a better example. I have judged clock when we were short of judges but I have no idea how to solve one. The delegate gave me some guidance including what the solved state looks like and was OK with me judging, but I would have been fine had they said no I'm not qualified.



Mark, that is true for you because you are a competent person in general, but maybe not for others. The point is that we need some kind of blanket thing that covers all bases. Just because you would have been fine without a certification doesn't mean others would.


----------



## mark49152 (Aug 18, 2016)

BillyRain said:


> Mark, that is true for you because you are a competent person in general, but maybe not for others. The point is that we need some kind of blanket thing that covers all bases. Just because you would have been fine without a certification doesn't mean others would.


Thanks, but my point was that it was and always should be the delegate's call. With or without a certification.

Don't get me wrong, I agree there's a problem, agree that we should try to address it, and agree that better educational materials would help. I agree that an online test would be good and have no objections to a test pass being treated as a qualification. My only reservation is that I cannot see certifications actually getting checked during the melee of a comp. 

By the way, Kit's video is a great resource for new competitors, but I don't recall seeing something similar for new judges. Is there one out there?


----------



## BillyRain (Aug 18, 2016)

mark49152 said:


> Thanks, but my point was that it was and always should be the delegate's call. With or without a certification.
> 
> Don't get me wrong, I agree there's a problem, agree that we should try to address it, and agree that better educational materials would help. I agree that an online test would be good and have no objections to a test pass being treated as a qualification. My only reservation is that I cannot see certifications actually getting checked during the melee of a comp.



Indeed. The checking issue is something we will need to think on. It's the only hurdle at the moment from making this a no brainer. Also I agree that the delegates call is always final.. but this is also intended to remove that workload from them. They have enough to do at comps already


----------



## Dene (Aug 18, 2016)

No one has responded to Lucas' concerns yet, in any practical way. So here's what I would say:

I believe, with the massive changes going on within the WCA over the past couple of years and in the coming years, we're going to have to take much more responsibility for the quality of our data (i.e. accurate judging and scrambling). The only way this will be achieved is if we have:

Motivated volunteers
A "WCA judging certified" status for the people who have proven themselves capable of judging (and scrambling).
But as Lucas says, why should anyone be motivated to help out? A lot of people tend to be lazy, and would use the lack of certification as an excuse to not help. In my opinion, the best way to overcome this obstacle is through community culture. Here's an example to illustrate what I mean. For years we kinda had trouble in Australia with convincing people to volunteer. It was largely the same small group of people doing all the work. Then a few years ago everything changed, and now we're consistently turning people back because we don't have any work available for them! Every time it's "Can I judge?!" "Can I scramble?!" "Is there anything I can do???!!!"

I'm not going to act like I have the secret formula to bring about this change, but it starts with the officials taking responsibility at the top level. We need to remove a lot of the casual attitude to competitions (at least as far as the actual competing part is concerned), and instead build a culture of formality and procedure. To motivate people to help out... well I guess we have to work in our own ways depending on our local groups. Perhaps if people see volunteering as something "exclusive" they may be motivated. Or perhaps if they got a certificate, or something on their WCA profile to indicate their status, that might motivate people to get involved. (Of course the key to exclusivity is that some people have to be rejected).

Anyway, I think if anything were to be done, it would have to be done right. I think it should be something managed by the WCA, not the cubers on speedsolving. Though if anyone is able to come up with a solid plan I think they should approach the Board and see what comes of it.


----------



## BillyRain (Aug 18, 2016)

Dene said:


> No one has responded to Lucas' concerns yet, in any practical way. So here's what I would say:
> 
> I believe, with the massive changes going on within the WCA over the past couple of years and in the coming years, we're going to have to take much more responsibility for the quality of our data (i.e. accurate judging and scrambling). The only way this will be achieved is if we have:
> 
> ...



Really glad to receive your support Dene,

Just a quick note as I am at work, but I've always thought that the only way to make judging duties fair and to motivate people to volunteer is to have a structured/scheduled/assigned judging system.

Basically speaking, if you want to compete at a competition, you will have equal responsibilities with judging as per a schedule.

But you are also correct in that a certification would certainly motivate people, especially if it puts a little mark on their WCA profile. People like awards/qualifications in any respect.


----------



## Dene (Aug 18, 2016)

What I'm thinking is that not anyone would be able to judge in the future, if it's going to be a more formal process. So it couldn't just be "if you want to compete you have to judge" thing. I don't know exactly how we might do it though.


----------



## BillyRain (Aug 18, 2016)

Dene said:


> What I'm thinking is that not anyone would be able to judge in the future, if it's going to be a more formal process. So it couldn't just be "if you want to compete you have to judge" thing. I don't know exactly how we might do it though.



Why would not everyone be able to judge? It would just be a 10 minute online test that you take at your leisure at any time and before competitions. There is no reason why somebody couldn't take it... short of not having a phone or a computer.... but... really?


----------



## JustinTimeCuber (Aug 18, 2016)

As a competitor, when people judge me and make a mistake, I try to correct them. I think that we should definitely try to teach judges what they are doing wrong before just kicking them off.


----------



## BillyRain (Aug 18, 2016)

JustinTimeCuber said:


> As a competitor, when people judge me and make a mistake, I try to correct them. I think that we should definitely try to teach judges what they are doing wrong before just kicking them off.



I'd say only gross misconduct would get a judge removed from the certified list.. because if they passed the exam then they have had the correct procedures go through their heads at least once. If they then do something MASSIVELY wrong.. they really have no excuse. So yeah you are right.. minor things would still just get corrected.


----------



## biscuit (Aug 18, 2016)

AS far as the checking thing goes, you've got a computer/ phone, so you just go type in a name, and if it shows up as a passing grade, then they can judge. It really shouldn't take more than 30 seconds if you already have the site up. Even if we have it as a google form (Which is being worked on) then it just goes into a spreadsheet, and we can sort it in there.

As far as getting removed from the list, I think it should be the delegate's call who is delegating the competition where an incident happens.


----------



## BillyRain (Aug 18, 2016)

biscuit said:


> AS far as the checking thing goes, you've got a computer/ phone, so you just go type in a name, and if it shows up as a passing grade, then they can judge. It really shouldn't take more than 30 seconds if you already have the site up. Even if we have it as a google form (Which is being worked on) then it just goes into a spreadsheet, and we can sort it in there.



Still not really practical. If you call for judges and then get a huge bunch of kids running up saying "I'll help", how long is it gonna take to take all of their names and check them against a database. Those kinds of things really do delay events.

Much more efficient way needed. Also by the way, I don't think it will be an issue because IMO the exam should be compulsory for anyone about to attend a competition.. So hopefully this wouldn't be an issue anyway.

Maybe prior to a competition.. the organiser or delegate could check all registered competitors with the list to see who is outstanding and needs to take the test...?


----------



## mark49152 (Aug 18, 2016)

Maybe a test pass could be a condition of being given a WCA ID. Unless you know how to judge you can't compete.


----------



## BillyRain (Aug 18, 2016)

mark49152 said:


> Maybe a test pass could be a condition of being given a WCA ID. Unless you know how to judge you can't compete.



In an absolute ideal world... this would be it. But I can't see that getting approved 

Wait better still... 

A certification is required before you register for a competition!! Simple!

If you don't have one... make sure you have taken the test before reg opens or before registering.

Problem solved!


----------



## Goosly (Aug 18, 2016)

mark49152 said:


> Maybe a test pass could be a condition of being given a WCA ID. Unless you know how to judge you can't compete.



If we want to discourage new cubers from going to their first competition, sure, let's do it.
New cubers are scared of having to judge already (unjustified, of course). Just teach them in a friendly way at the start of their first comp, like we're doing currently. Don't ask them to pass a test. Even if the test has a lot of smiley faces.


----------



## mark49152 (Aug 18, 2016)

BillyRain said:


> In an absolute ideal world... this would be it. But I can't see that getting approved


Maybe that's true, but I agree with Dene, first step is to make a proposal to the board. No need to water that down. Propose the ideal and let them reject or compromise it if it's unworkable. 

I can't think of another way to inject a check into the process without making it a hassle or leaving it open to circumvention. Having qualification as a requirement to register puts the onus on the competitor to get it done. And it does make some sense that people should prove they are qualified to compete and judge before getting in the WCA database.


----------



## biscuit (Aug 18, 2016)

BillyRain said:


> In an absolute ideal world... this would be it. But I can't see that getting approved
> 
> Wait better still...
> 
> ...



In an Ideal world, only staffers judge/scramble and run.

If you make it clear that you had to pass the test to judge when calling for judges, and have them form a line or something, I don't think it'd be that bad. We need to get something up and test it.



mark49152 said:


> Maybe that's true, but I agree with Dene, first step is to make a proposal to the board. No need to water that down. Propose the ideal and let them reject or compromise it if it's unworkable.
> 
> I can't think of another way to inject a check into the process without making it a hassle or leaving it open to circumvention. Having qualification as a requirement to register puts the onus on the competitor to get it done. And it does make some sense that people should prove they are qualified to compete and judge before getting in the WCA database.



Well we can come up with an minimal viable product, test it then work from there. I think it should be tried before it's submitted to the board. A proposal would also have to include how it'd work on the site, and regulations (addition to 2a).

Technically we could require to pass the test before signing up because of 2a2 could we not?

2a2) Meet the competition requirements, which must be clearly announced before the competition.


----------



## JustinTimeCuber (Aug 18, 2016)

biscuit said:


> In an Ideal world, only staffers judge/scramble and run.
> 
> If you make it clear that you had to pass the test to judge when calling for judges, and have them form a line or something, I don't think it'd be that bad. We need to get something up and test it.
> 
> ...


I personally like judging. I find it fun, and it helps organizers. However, I don't want to be on staff all day in particular. I think an ideal scenario would be approved judges who check an "I volunteer to judge" box would be put on a list and if someone on that list wanted to judge for, say, an hour, they would find someone who is getting tired of judging and trade spots with them. Then, an hour later, they would find someone who wants to judge and trade spots with them.


----------



## biscuit (Aug 18, 2016)

JustinTimeCuber said:


> I personally like judging. I find it fun, and it helps organizers. However, I don't want to be on staff all day in particular. I think an ideal scenario would be approved judges who check an "I volunteer to judge" box would be put on a list and if someone on that list wanted to judge for, say, an hour, they would find someone who is getting tired of judging and trade spots with them. Then, an hour later, they would find someone who wants to judge and trade spots with them.



I understand, but in an ideal scenario, you have official staff, with certification. That way there's a way smaller chance of the judge messing up, you KNOW the judges know what they're doing, and there's some accountability.


----------



## Dene (Aug 18, 2016)

@BillyRain the problem is, in what I envision, the position of "judge" is something formal. Have you been watching the olympics? You see those officials who stand behind the swimmers before they jump off the blocks? Or those dudes who raise the red or white flag in the track and field events? That's more what I'm thinking. That is, something truly formal. Of course the idea would be for cubers to fulfil the role, but only those who have proven to be up to a high standard (and not just passing a simple quiz online).


----------



## Ranzha (Aug 19, 2016)

Dene said:


> @BillyRain the problem is, in what I envision, the position of "judge" is something formal. Have you been watching the olympics? You see those officials who stand behind the swimmers before they jump off the blocks? Or those dudes who raise the red or white flag in the track and field events? That's more what I'm thinking. That is, something truly formal. Of course the idea would be for cubers to fulfil the role, but only those who have proven to be up to a high standard (and not just passing a simple quiz online).


I like this idea. I have no idea how to implement this (as well as formalising scramblers/checkers/runners) and keep competitions community-driven and competition atmospheres comfortable.


----------



## Adamt32 (Aug 19, 2016)

Before the competition, for about 5 minutes go over judging to all the competitors and parents so they know how to judge. It would also be beneficial if you also teach the first time competitors about inspection time, and how to use the timer during those 5 minutes.



mark49152 said:


> Maybe a test pass could be a condition of being given a WCA ID. Unless you know how to judge you can't compete.


I think all competitors should know how to judge, but the competitors shoul still be able to compete.


----------



## JustinTimeCuber (Aug 19, 2016)

I've posted my opinion here before, but I think I'll restate it. I think that in general*, first- and second-timers and non-cubers should not judge. If you are at your third competition, you probably know what's happening. I judged people at my third competition without any problems. I also kinda like the quiz idea, but I think it should be up to organizers to ensure that judges are doing well and not an arbitrary list of questions.

*Non-cubers can be competent judges, especially when they get a "one-on-one" briefing. At OKC, my half-cousin spent the day judging (feel sorry for you Bryce :3) and his mom (and dad? I forgot) was a runner (Okay, being a runner takes no skill).


----------



## obelisk477 (Aug 19, 2016)

In terms of motivation, it might help to say that the first 5 people or whatever to sign up for judging for a competition get to register for free, but then they have to get there an hour before the comp to make sure they know what they're doing, or to pass a certification test, or however that would be handled.


----------



## JustinTimeCuber (Aug 19, 2016)

obelisk477 said:


> In terms of motivation, it might help to say that the first 5 people or whatever to sign up for judging for a competition get to register for free, but then they have to get there an hour before the comp to make sure they know what they're doing, or to pass a certification test, or however that would be handled.


might be hard for first-time organizers and really weird to implement but could be possible idk


----------



## GenTheThief (Aug 19, 2016)

I think it would be a really good idea for a delegate or staff member to get a quick-fake-handscramble scramble and then violate some regulations (touching the cube after the solve, video screen in view during the solve, putting the cube on the timer, etc) just to see if the judge would notice and penalize them. Then, depending on what they do, they get corrected or praised, and then told that it was a fake solve.
It would also be pretty funny to watch.


----------



## biscuit (Aug 19, 2016)

JustinTimeCuber said:


> I've posted my opinion here before, but I think I'll restate it. I think that in general*, first- and second-timers and non-cubers should not judge. If you are at your third competition, you probably know what's happening. I judged people at my third competition without any problems. I also kinda like the quiz idea, but I think it should be up to organizers to ensure that judges are doing well and not an arbitrary list of questions.
> 
> *Non-cubers can be competent judges, especially when they get a "one-on-one" briefing. At OKC, my half-cousin spent the day judging (feel sorry for you Bryce :3) and his mom (and dad? I forgot) was a runner (Okay, being a runner takes no skill).



But that generalizes everyone. The second comp I ever went to I organized... I'm a fine judge if I do say so my self. If you understand the regs, and are a competent human being, than judging really isn't an issue. The test just makes sure you have at least a basic understanding of the regs, and if you don't know they exist, then it forces you to know of them.



GenTheThief said:


> I think it would be a really good idea for a delegate or staff member to get a quick-fake-handscramble scramble and then violate some regulations (touching the cube after the solve, video screen in view during the solve, putting the cube on the timer, etc) just to see if the judge would notice and penalize them. Then, depending on what they do, they get corrected or praised, and then told that it was a fake solve.
> It would also be pretty funny to watch.



We're thinking about adding something like that to the test (I.E a video of a solve with mistakes, and then you determine what penalty to give)


----------



## GenTheThief (Aug 19, 2016)

biscuit said:


> We're thinking about adding something like that to the test (I.E a video of a solve with mistakes, and then you determine what penalty to give)



I've been following the thread; I know what has been discussed, mostly.

This would be different because it would be in comp, under pressure, and without warning. A video solve with mistakes means that the person knows that they have to look for violations, and that there will be some.


----------



## JustinTimeCuber (Aug 19, 2016)

Most people's second comp isn't one that they organize 
but my 11th might be *hint hint*
I don't think that first timers and especially second timers should be completely barred from judging, but be more cautiously screened because *in general* they don't know the WCA regs and rules as well.


----------



## Lucas Garron (Aug 19, 2016)

BillyRain said:


> Still not really practical. If you call for judges and then get a huge bunch of kids running up saying "I'll help", how long is it gonna take to take all of their names and check them against a database.



Put markers on their printed name tags ahead of time.


----------



## Goosly (Aug 19, 2016)

Lucas Garron said:


> Put markers on their printed name tags ahead of time.



At small competitions, nobody wears their name tag.


----------



## BillyRain (Aug 19, 2016)

Ya, smaller competitions don't even have name tags sometimes.

I still don't see an issue with making everyone take the online test before being able to register for the event... I mean they would have plenty of time to do it and then you can be sure that ALL competitors at the competition have been through it.

The test can be as hard or as easy as we want to make it. If you want to really ensure that someone has been through all of the regs and situations, then the test would be quite long. But why not? They only have to take it once and they can do it whenever they want.


----------



## AlexMaass (Aug 19, 2016)

Ranzha said:


> At the most recent Stanford competition, Noah Arthurs instructed competitors to replace their judge at the timing station after completing their last attempt of a round. He provided every station with a sheet of paper with introductory content regarding judging.
> 
> I transcribed this sheet, making some additions, and Christian Pizzasegola and I worked to translate the sheet into Latin American Spanish.
> 
> ...


using this for my comp on saturday thanks

I think I'll edit the stuff with timer starting and stopping, needs to be more specific with it

also cutoffs too


----------



## AlexMaass (Aug 19, 2016)

https://docs.google.com/document/d/1x0C2z3b6ulUJmGK0bMmbHjM2Ve35XngxCr_TOXJJStM/edit?usp=sharing seems good


----------



## Zoé (Aug 19, 2016)

AlexMaass said:


> https://docs.google.com/document/d/1x0C2z3b6ulUJmGK0bMmbHjM2Ve35XngxCr_TOXJJStM/edit?usp=sharing seems good


_"If a piece pops out of the puzzle, the competitor has to either reassemble their cube with the timer still running, or receive a DNF. Nobody except the competitor can touch any pieces of the cube."_

That isn't true though:

5b5) If at the end of the attempt some parts of the puzzle are physically detached or not fully placed, the following regulations apply:

5b5a) If one or more parts without coloured faces is affected, the puzzle is considered solved.
5b5b) If one part with one coloured face is affected, the puzzle is considered solved.
5b5c) If more than one part with one coloured face is affected, the puzzle is considered unsolved (DNF).
5b5d) If one or more parts with more than one coloured face is affected, the puzzle is considered unsolved (DNF).
5b5e) Regulations 5b5c and 5b5d supersede 5b5a and 5b5b.


----------



## JustinTimeCuber (Aug 19, 2016)

Suggested edit:

Former:

During the solve:
Do not distract the competitor.

Updated:

During the solve:
*Do not distract the competitor.*


----------



## biscuit (Aug 19, 2016)

Couple of suggested Edits from me.

Addressing Justin's "Edit", add something about not solving a cube while the competitor is solving. It doesn't really bother me, but it does bother a bunch of people. Maybe something like

"Do not distract the competitor by talking, solving a cube, moving around a bunch etc."

"Inspect the puzzle without moving it to see if it is solved. If there is a misalignment and you are not sure about the correct penalty, ask for help *from the delegate.*" (maybe even from the delegate/experienced cuber)

"All solves with penalties have to include an equals sign and final result." Give an example just to make super sure people get it.

"Make sure the competitor starts the timer with fingers flat on the timer, and palms down, and also that they stop the timer with their hands*, with the palms down*. This way, they can’t be touching the puzzle at the beginning or end of the solve. If they don’t stop and/or start the solve properly, ask for help *from the delegate *regarding the decision involving this." (Again, could put /experienced cuber)


----------



## Matt11111 (Aug 19, 2016)

JustinTimeCuber said:


> Suggested edit:
> 
> Former:
> 
> ...


Way to emphasize.


----------



## AlexMaass (Aug 19, 2016)

Zoé said:


> _"If a piece pops out of the puzzle, the competitor has to either reassemble their cube with the timer still running, or receive a DNF. Nobody except the competitor can touch any pieces of the cube."_
> 
> That isn't true though:
> 
> ...


Fixed


----------



## Ranzha (Aug 19, 2016)

Zoé said:


> _"If a piece pops out of the puzzle, the competitor has to either reassemble their cube with the timer still running, or receive a DNF. Nobody except the competitor can touch any pieces of the cube."_
> 
> That isn't true though:
> 
> ...


I'll make an adjustment.

To be honest, I never saw the point of being forced to immediately repair the puzzle. If a centre cap pops off, there's no way in hell I'm gonna fix it during the solve. 5b5 should be updated to say "end of the solve" instead of "end of the attempt".


----------



## cuberkid10 (Aug 19, 2016)

biscuit said:


> Addressing Justin's "Edit", add something about not solving a cube while the competitor is solving. It doesn't really bother me, but it does bother a bunch of people. Maybe something like
> 
> "Do not distract the competitor by talking, solving a cube, moving around a bunch etc."


Just as an interesting point to add, Nationals staff members were given explicit instructions to do nothing other than judge when judging. We were not allowed to be on phones, cube, and other things like that. It's not only possibly distracting to the competitor; it's unprofessional and can cause you to miss important things going on during the solve.


----------

