I use the one at my local makerspace.univurshul wrote:Where are you guys getting your laser cutters?
Microsoft Kinect: infrared depth maps for dewarping?
Moderator: peterZ
- rob
- Posts: 773
- Joined: 03 Jun 2009, 13:50
- E-book readers owned: iRex iLiad, Kindle 2
- Number of books owned: 4000
- Country: United States
- Location: Maryland, United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
The Singularity is Near. ~ http://halfbakedmaker.org ~ Follow me as I build the world's first all-mechanical steam-powered computer.
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
Well, this turned out to be an interesting story. The guy that made the high quality hacked driver for the PS3 cam wrote code to control it right away, but didn't release it. Instead, he made a fund asking for $10,000 (including the Adafruit money), and when that was hit, he would release code.
http://www.engadget.com/2010/11/07/kine ... ame-video/
Unfortunately for AlexP, the open source approach steamrolled him. Within a few hours of the european launch of Kinect, hacker Hector Martin released working code for Linux.
http://www.engadget.com/2010/11/10/open ... -download/
I hope Adafruit offers Hector the prize. I just ordered a Kinect for myself. Looking forward to making this my first serious coding project.
http://www.engadget.com/2010/11/07/kine ... ame-video/
Unfortunately for AlexP, the open source approach steamrolled him. Within a few hours of the european launch of Kinect, hacker Hector Martin released working code for Linux.
http://www.engadget.com/2010/11/10/open ... -download/
I hope Adafruit offers Hector the prize. I just ordered a Kinect for myself. Looking forward to making this my first serious coding project.
-
- Posts: 138
- Joined: 30 Oct 2010, 23:56
- Number of books owned: 0
- Location: Austin, Texas, USA
- Contact:
Structured light for dewarping
Both the Kinect and the page-flipping demo use the same techniques for figuring out depth and alignment: structured light.
http://en.wikipedia.org/wiki/Structured_light
The links in the Wikipedia article include other references, such as using it to automatically align projectors (where I first heard about it), and algorithms and software. The Makerbot 3D scanner has its own software that performs the same functions.
The IEEE Spectrum article about the book flipping scanner says that it uses a custom camera that does 1280x1024 at 500fps, but I'm not sure you really need a) that high a framerate or b) a custom camera. Elphel makes ~$1k video cameras that can do at least 640x480 at 100fps, but maybe 1024x768 at 44fps is enough, or something in between. For lower-resolution video, you could theoretically use superresolution techniques to interpolate multiple video frames together into a higher-resolution or more detailed image.
"Just" adding macro lenses to the Kinect hardware to refocus the IR light on a page or across the entire platen might give you enough resolution to dewarp a book, even if you're not using the RGB camera to capture the images, and instead just turning pages and photographing one or both pages with a still camera. Reconstructing two pages from a single photograph of a "spread" alleviates a lot of the issues with dual cameras if you don't mind the resolution loss and it doesn't affect OCR.
Given that there are at least two pieces of software for reference out there for doing this, I'd guess it's probably doable by a software developer with a few weeks of free time, regardless of whether you're using Kinect or making it yourself with a projector like the Makerbot 3D scanner.
http://en.wikipedia.org/wiki/Structured_light
The links in the Wikipedia article include other references, such as using it to automatically align projectors (where I first heard about it), and algorithms and software. The Makerbot 3D scanner has its own software that performs the same functions.
The IEEE Spectrum article about the book flipping scanner says that it uses a custom camera that does 1280x1024 at 500fps, but I'm not sure you really need a) that high a framerate or b) a custom camera. Elphel makes ~$1k video cameras that can do at least 640x480 at 100fps, but maybe 1024x768 at 44fps is enough, or something in between. For lower-resolution video, you could theoretically use superresolution techniques to interpolate multiple video frames together into a higher-resolution or more detailed image.
"Just" adding macro lenses to the Kinect hardware to refocus the IR light on a page or across the entire platen might give you enough resolution to dewarp a book, even if you're not using the RGB camera to capture the images, and instead just turning pages and photographing one or both pages with a still camera. Reconstructing two pages from a single photograph of a "spread" alleviates a lot of the issues with dual cameras if you don't mind the resolution loss and it doesn't affect OCR.
Given that there are at least two pieces of software for reference out there for doing this, I'd guess it's probably doable by a software developer with a few weeks of free time, regardless of whether you're using Kinect or making it yourself with a projector like the Makerbot 3D scanner.
- rob
- Posts: 773
- Joined: 03 Jun 2009, 13:50
- E-book readers owned: iRex iLiad, Kindle 2
- Number of books owned: 4000
- Country: United States
- Location: Maryland, United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
Ha ha HA! Hector indeed got the $3k bounty. I doubt AlexP even had a working driver.daniel_reetz wrote:...Within a few hours of the european launch of Kinect, hacker Hector Martin released working code for Linux.
However, the depth output seems rather coarse...
The Singularity is Near. ~ http://halfbakedmaker.org ~ Follow me as I build the world's first all-mechanical steam-powered computer.
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
I know... so great.
Staring at my Kinect now... I'd start hacking on it, but I'm working on another book scanner design.
Staring at my Kinect now... I'd start hacking on it, but I'm working on another book scanner design.
-
- Posts: 496
- Joined: 04 Mar 2014, 00:53
Re: Microsoft Kinect: infrared depth maps for dewarping?
I'm buying my girlfriend a Kinect for Christmas. hehehe...
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
Let me guess... she doesn't have an XBOX? Perfect!
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
I just had a meetup with Rob and Spamsickle, and Spamsickle said there was a video of the Kinect infrared lighting... I hadn't seen it, but I went looking and found this: http://www.youtube.com/watch?v=nvvQJxgykcU
Pretty interesting stuff...
Pretty interesting stuff...
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
Alright, so I am now working to get the Kinect up and running. Warning, this is going to be super-verbose and mostly notes to myself.
First things first: Need to get the code onto my machine. I'm using Ubuntu Linux for my starting point.
The code is stored on a Git server. Git is a version control system for code. I have never used it before. Using this tutorial as a starting point, I entered the following magic incantation:
Which got me the following error message/solution:
I installed Git and tried the command again.
Now it's time to build this code. But damn, I don't have the right tools. Just trying make or gcc fails badly.Lookslike people are building this code with cmake. So, to install cmake:
Then I made a directory called "build"
Ugh, so all that and there's apparently no cmake file, blah. I am going to clone the OpenKinect git repository, instead, and start with that -- it looks to be where the action is happening.
hmm, more trouble in paradise, blarg:
Huh, I guess that means that GCC is not installed. Well, that's not so bad. More installing. This time the necessary package is called "build-essential"
Installed fine. Now to try that command again...
WHEW, that was RIDICULOUS. Now I have a makefile?? So I need to run MAKE:
Alright, well, something bad happened, it had something to do with GLUT. I'm going to try installing that and then I am going to quit for the night, I need to be well rested tomorrow.
BAGH. No. Stupid linux.
I went and installed everything related to FreeGLUT and GLUTG3. Result:
I think that means it built. Gonna try to run it now...
First things first: Need to get the code onto my machine. I'm using Ubuntu Linux for my starting point.
The code is stored on a Git server. Git is a version control system for code. I have never used it before. Using this tutorial as a starting point, I entered the following magic incantation:
Code: Select all
fake@polar:~$ git clone git://git.marcansoft.com/libfreenect.git
Code: Select all
The program 'git' is currently not installed. You can install it by typing:
sudo apt-get install git-core
fake@polar:~$ sudo apt-get install git-core
Code: Select all
fake@polar:~$ git clone git://git.marcansoft.com/libfreenect.git
Initialized empty Git repository in /home/fake/libfreenect/.git/
remote: Counting objects: 74, done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 74 (delta 37), reused 0 (delta 0)
Receiving objects: 100% (74/74), 17.44 KiB, done.
Resolving deltas: 100% (37/37), done.
fake@polar:~$
Code: Select all
sudo apt-get install cmake
Code: Select all
fake@polar:~$ cd libfreenect
fake@polar:~/libfreenect$ mkdir build
fake@polar:~/libfreenect$ ls
build CMakeLists.txt examples include lib Modules README.txt
fake@polar:~/libfreenect$ cd build
fake@polar:~/libfreenect/build$ cmake
Code: Select all
fake@polar:~$ git clone https://github.com/qdot/OpenKinect.git
Initialized empty Git repository in /home/fake/OpenKinect/.git/
remote: Counting objects: 182, done.
remote: Compressing objects: 100% (155/155), done.
remote: Total 182 (delta 77), reused 54 (delta 15)
Receiving objects: 100% (182/182), 352.37 KiB | 180 KiB/s, done.
Resolving deltas: 100% (77/77), done.
fake@polar:~$ ls
fake@polar:~$ cd OpenKinect
fake@polar:~/OpenKinect$
Code: Select all
fake@polar:~/OpenKinect/c$ cd build
fake@polar:~/OpenKinect/c/build$ cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is unknown
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
CMake Error: your CXX compiler: "CMAKE_CXX_COMPILER-NOTFOUND" was not found. Please set CMAKE_CXX_COMPILER to a valid compiler path or name.
-- Looking for include files CMAKE_HAVE_PTHREAD_H
-- Looking for include files CMAKE_HAVE_PTHREAD_H - found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- checking for module 'libusb-1.0>=1.0.3'
-- found libusb-1.0, version 1.0.6
-- Found USB
-- Found USB
-- Configuring incomplete, errors occurred!
fake@polar:~/OpenKinect/c/build$ ls
CMakeCache.txt CMakeFiles examples lib
fake@polar:~/OpenKinect/c/build$
Code: Select all
fake@polar:~/OpenKinect/c/build$ sudo apt-get install build-essential
Code: Select all
fake@polar:~/OpenKinect/c/build$ cmake ..
-- The CXX compiler identification is GNU
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found USB
-- Found USB
-- Configuring done
-- Generating done
-- Build files have been written to: /home/fake/OpenKinect/c/build
fake@polar:~/OpenKinect/c/build$
Code: Select all
fake@polar:~/OpenKinect/c/build$ make
[ 25%] Generating inits.c
Scanning dependencies of target freenect
[ 50%] Building C object lib/CMakeFiles/freenect.dir/cameras.c.o
[ 75%] Building C object lib/CMakeFiles/freenect.dir/inits.c.o
Linking C static library libfreenect.a
[ 75%] Built target freenect
Scanning dependencies of target glview
[100%] Building C object examples/CMakeFiles/glview.dir/glview.c.o
/home/fake/OpenKinect/c/examples/glview.c:24:21: error: GL/glut.h: No such file or directory
/home/fake/OpenKinect/c/examples/glview.c:25:19: error: GL/gl.h: No such file or directory
/home/fake/OpenKinect/c/examples/glview.c:26:20: error: GL/glu.h: No such file or directory
/home/fake/OpenKinect/c/examples/glview.c:47: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘gl_depth_tex’
/home/fake/OpenKinect/c/examples/glview.c:48: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘gl_rgb_tex’
/home/fake/OpenKinect/c/examples/glview.c: In function ‘DrawGLScene’:
/home/fake/OpenKinect/c/examples/glview.c:68: warning: implicit declaration of function ‘glClear’
/home/fake/OpenKinect/c/examples/glview.c:68: error: ‘GL_COLOR_BUFFER_BIT’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:68: error: (Each undeclared identifier is reported only once
/home/fake/OpenKinect/c/examples/glview.c:68: error: for each function it appears in.)
/home/fake/OpenKinect/c/examples/glview.c:68: error: ‘GL_DEPTH_BUFFER_BIT’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:69: warning: implicit declaration of function ‘glLoadIdentity’
/home/fake/OpenKinect/c/examples/glview.c:71: warning: implicit declaration of function ‘glEnable’
/home/fake/OpenKinect/c/examples/glview.c:71: error: ‘GL_TEXTURE_2D’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:73: warning: implicit declaration of function ‘glBindTexture’
/home/fake/OpenKinect/c/examples/glview.c:73: error: ‘gl_depth_tex’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:74: warning: implicit declaration of function ‘glTexImage2D’
/home/fake/OpenKinect/c/examples/glview.c:74: error: ‘GL_RGB’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:74: error: ‘GL_UNSIGNED_BYTE’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:76: warning: implicit declaration of function ‘glBegin’
/home/fake/OpenKinect/c/examples/glview.c:76: error: ‘GL_TRIANGLE_FAN’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:77: warning: implicit declaration of function ‘glColor4f’
/home/fake/OpenKinect/c/examples/glview.c:78: warning: implicit declaration of function ‘glTexCoord2f’
/home/fake/OpenKinect/c/examples/glview.c:78: warning: implicit declaration of function ‘glVertex3f’
/home/fake/OpenKinect/c/examples/glview.c:82: warning: implicit declaration of function ‘glEnd’
/home/fake/OpenKinect/c/examples/glview.c:84: error: ‘gl_rgb_tex’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:95: warning: implicit declaration of function ‘glutSwapBuffers’
/home/fake/OpenKinect/c/examples/glview.c:56: warning: unused variable ‘fcnt’
/home/fake/OpenKinect/c/examples/glview.c: In function ‘keyPressed’:
/home/fake/OpenKinect/c/examples/glview.c:102: warning: implicit declaration of function ‘glutDestroyWindow’
/home/fake/OpenKinect/c/examples/glview.c: In function ‘ReSizeGLScene’:
/home/fake/OpenKinect/c/examples/glview.c:109: warning: implicit declaration of function ‘glViewport’
/home/fake/OpenKinect/c/examples/glview.c:110: warning: implicit declaration of function ‘glMatrixMode’
/home/fake/OpenKinect/c/examples/glview.c:110: error: ‘GL_PROJECTION’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:112: warning: implicit declaration of function ‘glOrtho’
/home/fake/OpenKinect/c/examples/glview.c:113: error: ‘GL_MODELVIEW’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c: In function ‘InitGL’:
/home/fake/OpenKinect/c/examples/glview.c:118: warning: implicit declaration of function ‘glClearColor’
/home/fake/OpenKinect/c/examples/glview.c:119: warning: implicit declaration of function ‘glClearDepth’
/home/fake/OpenKinect/c/examples/glview.c:120: warning: implicit declaration of function ‘glDepthFunc’
/home/fake/OpenKinect/c/examples/glview.c:120: error: ‘GL_LESS’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:121: warning: implicit declaration of function ‘glDisable’
/home/fake/OpenKinect/c/examples/glview.c:121: error: ‘GL_DEPTH_TEST’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:122: error: ‘GL_BLEND’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:123: warning: implicit declaration of function ‘glBlendFunc’
/home/fake/OpenKinect/c/examples/glview.c:123: error: ‘GL_SRC_ALPHA’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:123: error: ‘GL_ONE_MINUS_SRC_ALPHA’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:124: warning: implicit declaration of function ‘glShadeModel’
/home/fake/OpenKinect/c/examples/glview.c:124: error: ‘GL_SMOOTH’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:125: warning: implicit declaration of function ‘glGenTextures’
/home/fake/OpenKinect/c/examples/glview.c:125: error: ‘gl_depth_tex’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:126: error: ‘GL_TEXTURE_2D’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:127: warning: implicit declaration of function ‘glTexParameteri’
/home/fake/OpenKinect/c/examples/glview.c:127: error: ‘GL_TEXTURE_MIN_FILTER’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:127: error: ‘GL_LINEAR’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:128: error: ‘GL_TEXTURE_MAG_FILTER’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:129: error: ‘gl_rgb_tex’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c: In function ‘gl_threadfunc’:
/home/fake/OpenKinect/c/examples/glview.c:140: warning: implicit declaration of function ‘glutInit’
/home/fake/OpenKinect/c/examples/glview.c:142: warning: implicit declaration of function ‘glutInitDisplayMode’
/home/fake/OpenKinect/c/examples/glview.c:142: error: ‘GLUT_RGBA’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:142: error: ‘GLUT_DOUBLE’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:142: error: ‘GLUT_ALPHA’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:142: error: ‘GLUT_DEPTH’ undeclared (first use in this function)
/home/fake/OpenKinect/c/examples/glview.c:143: warning: implicit declaration of function ‘glutInitWindowSize’
/home/fake/OpenKinect/c/examples/glview.c:144: warning: implicit declaration of function ‘glutInitWindowPosition’
/home/fake/OpenKinect/c/examples/glview.c:146: warning: implicit declaration of function ‘glutCreateWindow’
/home/fake/OpenKinect/c/examples/glview.c:148: warning: implicit declaration of function ‘glutDisplayFunc’
/home/fake/OpenKinect/c/examples/glview.c:149: warning: implicit declaration of function ‘glutIdleFunc’
/home/fake/OpenKinect/c/examples/glview.c:150: warning: implicit declaration of function ‘glutReshapeFunc’
/home/fake/OpenKinect/c/examples/glview.c:151: warning: implicit declaration of function ‘glutKeyboardFunc’
/home/fake/OpenKinect/c/examples/glview.c:155: warning: implicit declaration of function ‘glutMainLoop’
/home/fake/OpenKinect/c/examples/glview.c: In function ‘rgbimg’:
/home/fake/OpenKinect/c/examples/glview.c:221: warning: unused variable ‘i’
make[2]: *** [examples/CMakeFiles/glview.dir/glview.c.o] Error 1
make[1]: *** [examples/CMakeFiles/glview.dir/all] Error 2
make: *** [all] Error 2
fake@polar:~/OpenKinect/c/build$
Code: Select all
/home/fake/OpenKinect/c/examples/glview.c:24:21: error: GL/glut.h: No such file or directory
I went and installed everything related to FreeGLUT and GLUTG3. Result:
Code: Select all
fake@polar:~/OpenKinect/c/build$ make
[ 75%] Built target freenect
[100%] Building C object examples/CMakeFiles/glview.dir/glview.c.o
/home/fake/OpenKinect/c/examples/glview.c: In function ‘DrawGLScene’:
/home/fake/OpenKinect/c/examples/glview.c:56: warning: unused variable ‘fcnt’
/home/fake/OpenKinect/c/examples/glview.c: In function ‘rgbimg’:
/home/fake/OpenKinect/c/examples/glview.c:221: warning: unused variable ‘i’
Linking C executable glview
[100%] Built target glview
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Microsoft Kinect: infrared depth maps for dewarping?
Moar learning.
So, the make process built a file called "glview". That was the sum of all that effort. But you can't just stupidly type "glview" and expect it to work:
Nope, to run an executable you have to type "./" before the command. So...
AGGHHH. It needs to run as root.
Success!
Some bad news, I tried a few normal lenses on the IR projector and camera, and in both cases it immediately stopped working. Now I need to look into IR lenses...
So, the make process built a file called "glview". That was the sum of all that effort. But you can't just stupidly type "glview" and expect it to work:
Code: Select all
fake@polar:~/OpenKinect/c/build/examples$ glview
No command 'glview' found, did you mean:
Command 'vlview' from package 'lipsia' (universe)
Command 'mlview' from package 'mlview' (universe)
Command 'mglview' from package 'mathgl' (universe)
Command 'gview' from package 'vim-gnome' (main)
Command 'gview' from package 'vim' (main)
Command 'gview' from package 'vim-nox' (universe)
Command 'gview' from package 'vim-gtk' (universe)
Command 'gview' from package 'vim-tiny' (main)
Command 'gqview' from package 'gqview' (universe)
glview: command not found
fake@polar:~/OpenKinect/c/build/examples$ ^C
fake@polar:~/OpenKinect/c/build/examples$
Code: Select all
fake@polar:~/OpenKinect/c/build/examples$ ./glview
Kinect camera test
libusb couldn't open USB device /dev/bus/usb/002/019: Permission denied.
libusb requires write access to USB device nodes.
Could not open device
fake@polar:~/OpenKinect/c/build/examples$
Code: Select all
fake@polar:~/OpenKinect/c/build/examples$ sudo ./glview