# Reading Rubik's Cube states into computer using Computer Vision magic



## badmephisto (May 4, 2010)

I thought some people might be interested: here are some results from my Computer Vision project:

http://www.youtube.com/watch?v=VaW1dmqRE0o

It dealt with a better/faster way to read in a state of a rubiks cube so that you can enter any particular configuration into solvers quickly, or just share some configuration with other people.

Implemented in Python and OpenCV library.

I haven't quite figured out what I can port the resulting configuration to yet

Also the code is NOT online, but I may post it sometime later... for now just sharing it with everyone!
EDIT: ah what the hell here is the code and my report:
http://www.cs.ubc.ca/~andrejk/525project/


----------



## koreancuber (May 4, 2010)

Wow, this is an upgrade version of the cubeexplorer? Mine didn't seem to work.


----------



## lilkdub503 (May 4, 2010)

So sick! How long did it take you to write the code? Oh, and have you started cubing again seriously (if so, what are you averaging)?


----------



## Chapuunka (May 4, 2010)

I could see this being on every computer with a webcam pretty soon. That would be pretty cool.

I also saw your videos about the evolutions of simple organisms. That was actually pretty fascinating.


----------



## Andreas737 (May 4, 2010)

I finally see your face lol!


----------



## oprah62 (May 4, 2010)

very nice! webcam picks up image?


----------



## IamWEB (May 4, 2010)

GREAT!

...btw can you please unblock from that channel? :3


----------



## Lucas Garron (May 4, 2010)

Something I've always wondered about: Why not read in 3 faces at a time?


----------



## riffz (May 4, 2010)

Lucas Garron said:


> Something I've always wondered about: Why not read in 3 faces at a time?



Might be tough with the bad lighting at an angle.

Cool stuff. I didn't know about your programming channel before.


----------



## badmephisto (May 4, 2010)

Lucas: good idea.
I suspect that it would suffer from lighting problems as riffz suggested.
It would also be harder to align it exactly toward the camera in that way and much harder to hold it in a way as to not cause occlusions.

The detection itself is also not to be trusted TOO much. It is more reliable if it is ok to get it slightly wrong, which is the case for a single face, but most likely not for 3 faces. Not enough wiggle.

And yes riffz thats the channel I usually post my programming stuff on. I just need to sometimes share results of some of my programs with other people and thats mostly what i use it for. In this case that someone was my prof for the class


----------



## bamboocha (May 8, 2010)

Sorry for me being a noob but how do I run this program? I downloaded Python 2.6 and OpenCV 2.1 with bindings and numpy library. 

When I try to run it I get an error:


```
Traceback (most recent call last):
  File "C:\folders\cubefinder.py", line 2, in <module>
    import cv
ImportError: No module named cv
```


----------



## badmephisto (May 8, 2010)

bamboocha said:


> Sorry for me being a noob but how do I run this program? I downloaded Python 2.6 and OpenCV 2.1 with bindings and numpy library.
> 
> When I try to run it I get an error:
> 
> ...



Pretty simple, OpenCV install didn't go right. Check out their install directions more closely. When I installed it they required me to copy the cv.pyd files into Python/Lib/site-packages folder but maybe they have a more automatic way in their newer install


----------



## Crosshash (May 8, 2010)

Very interesting. Are you going to elaborate on this code some more? 

I'm wondering if the question you asked in the other thread was related to making sure all the colours _matched_ up correctly so you could solve the cube based on colour input only. Would be very interesting to see it.

I wish I could read your code more easily but I've never used Python before (i study Java), was the reason for using an edge detector in order to identify the cube in the image?


----------



## MrIndianTeen (Feb 23, 2011)

when I try the link it says --

Sorry, something went wrong.

A team of highly trained monkeys has been dispatched to deal with this situation.

If you see them, show them this information:

[edit: lots of chars were here]


----------



## qqwref (Feb 23, 2011)

What the hell? You bumped a 6+ month old topic to post a huge amount of random youtube error data.


----------



## PatrickJameson (Feb 23, 2011)

I was going to delete the above two posts, infract, and move on but this gave me a nice little project for the next three days(or less, depending on how long it takes and how far I go with it) since I have have a slight break from robotics. I've been dealing with a bit of object tracking stuff for robotics already. Should be fun. I'll probably just use Java and v4l4j since I used it for the linked video already.


----------



## Herbert Kociemba (Mar 3, 2019)

This thread is dead since 2011 but since I are also do some experiments with Python and OpenCV I decided to revive it instead of creating a new thread. The link http://www.cs.ubc.ca/~andrejk/525project/ from above also is dead and I want to ask if anybody here has a copy of the code or a working link because I am interested in the code.

With my approach I put a grid on a video frame (gridsize about 25x35) and compute the average hue values and the variance of the hue values in a grid-square. Squares with low variances are candidates for a square which is situated completely inside a cube facelet. Omitting some details in the end I have I binary mask which I search for squares using the openCV findContours and approxPolyDP functions. This works best for cubes where the facelets are well separated of course. In this way I can detect all facelets within the picture.

The color detection itself I also use the hsv-values, h to decide the color of all non-white facelets, white faclets are detected by low s values and high v values. Distingushing between red and orange in advance is probably impossible, it depends on the colors of the actual cube and to some extend also on the light source. At least you have to know the hue range fo an orange sticker.
Here is an example: http://kociemba.org/videos/cuberec.mp4

I plan to integrate the computer vision part into my Python cubesolver project https://github.com/hkociemba/RubiksCube-TwophaseSolver as soon as possible.


----------



## abunickabhi (Mar 6, 2019)

Herbert Kociemba said:


> This thread is dead since 2011 but since I are also do some experiments with Python and OpenCV I decided to revive it instead of creating a new thread. The link http://www.cs.ubc.ca/~andrejk/525project/ from above also is dead and I want to ask if anybody here has a copy of the code or a working link because I am interested in the code.
> 
> With my approach I put a grid on a video frame (gridsize about 25x35) and compute the average hue values and the variance of the hue values in a grid-square. Squares with low variances are candidates for a square which is situated completely inside a cube facelet. Omitting some details in the end I have I binary mask which I search for squares using the openCV findContours and approxPolyDP functions. This works best for cubes where the facelets are well separated of course. In this way I can detect all facelets within the picture.
> 
> ...



Good initiative and all the best!


----------



## Herbert Kociemba (Mar 7, 2019)

Faster than I thought I was able to integrate the computervision part. The files are uploaded to Github now. Here is an example how the parts work together:

http://kociemba.org/videos/visiongui.avi

Yet some critical explanations are missing how to handle the sliders for the filters which is quite essential to achieve a good recognition rate...


----------



## Herbert Kociemba (Mar 16, 2019)

I more or less finished the project and the computer vision now also works with the Raspberry Pi and it's camera module. Some more information how to use the program can be found here: http://kociemba.org/computervision.html


----------



## abunickabhi (Mar 17, 2019)

Great work!

Are you aiming for detecting 3-4 faces simultaneously in the future?
Using an orthographic projection to map out the faces on the side, and detect stickers even there is occlusion like our fingers on some sides.
Like in one webcam shot, about 9 stickers upfront + 6-7 stickers on the side may be detected, so that the cube needs just a y2 and the webcam will register the entire cube state.


----------



## Herbert Kociemba (Mar 17, 2019)

abunickabhi said:


> Are you aiming for detecting 3-4 faces simultaneously in the future?


I do not think I can do this successfully. Think alone at the different lightning conditions on the different faces which makes this difficult. So I will stay away from this.


----------



## Claudeen (Apr 13, 2019)

Is there a way to analyze a photo previously taken instead of using the webcam?


----------



## Herbert Kociemba (Apr 17, 2019)

Claudeen said:


> Is there a way to analyze a photo previously taken instead of using the webcam?


Yes. In the function grab_colors in vision2.py you can substitute the code given between "def grab_colors():" and "bgrcap = cv2.blur(bgrcap, (5, 5))" in the original code by the lines given below if the image has the name test.jpg and is in the same directory as vision2.py.


```
def grab_colors():
    global cent, width, height, hsv, color_mask, white_mask

    bgrcap = cv2.imread('test.jpg')
    height, width = bgrcap.shape[:2]
    while 1:
        bgrcap = cv2.imread('test.jpg')
        bgrcap = cv2.blur(bgrcap, (5, 5))
```
[


----------



## Claudeen (Apr 17, 2019)

Herbert Kociemba said:


> Yes. In the function grab_colors in vision2.py you can substitute the code given between "def grab_colors():" and "bgrcap = cv2.blur(bgrcap, (5, 5))" in the original code by the lines given below if the image has the name test.jpg and is in the same directory as vision2.py.
> 
> 
> ```
> ...



Thank you so much for your work, you helped me a lot for my project


----------

