Camera + Double Periscope -> 3D Vision -> Portable Scanner?

[andrewgreendf]

Hi everyone,

I've been bitten by a book-scanner-design-idea bug. I really don't have time to realize the design, at least not now, but I'd like to throw it out there.

In a nutshell: take one camera, attach it to a double periscope to get two images of the book. Use existing free software libraries to create a 3D model from the images. Use the 3D model to accurately de-warp the pages. No platen. Lay the book flat if you can, or use a cradel if the book's binding demands it. Throw the bugger in your backpack, and run to the library.

I opened a blog to describe the idea more fully: http://rebooker.wordpress.com/.

After looking quickly at some of the posts in these forums, I got the impresssion that this idea speaks to at least two recurring concerns: 3D data for de-warping and portability. I was a bit surprised not to see much talk of using regular stereo images to create 3D data--everyone seems much more interested in lasers, Kinect, defocusing and IR projectors--but maybe there are some obvious obstacles to this that I'm missing.

Comments? Suggestions?

Many thanks in advance, and thanks to everyone for a great site!

[Shaknum]

You can get stereo/3d lenses for many DSLR's and that will give you two images side by side, each with a slightly different perspective. Decapod is a project that is supposed to take this sort of technology (they use two cameras not a stereo image) to flatten out a book. If you use only one camera, you will probably want something 18+ MP, a Leaf Aptus II-12 at 80 megapixels would work great. Seriously though, Canon makes some 18MP cameras that would probably be acceptable, Fujifilm has some 19MP cameras that may work as well. Remember that you will be getting half resolution, so 8-10 MP should be nice for smaller paperbacks, but if you are scanning larger books, 14 MP (per page) is probably a better bet.

I'd love to see the software side of things, keep us updated with what you find out.

[andrewgreendf]

Thanks a lot for the info, Shaknum...

Maybe resolution is the main obstacle here. I was hoping this would make for something portable and cheap, but in fact the resolution here would not be 1/2, but 1/4, of the resolution of standard two-camera DIY designs. First, half the resolution because you're making two images in one shot, then half it again because you're taking two pages instead of one. Or even less, considering that to get the whole book to be visible in the images, you'd probably be shooting some of what's beyond the edges of the book, either on the sides, or on the top and the bottom.

(In fact, I've assumed it would be desirable to include some of the area around the book in the images; I imagine this could help to get accurate information on page dimensions and the location of content on the pages, which in turn could help with automatic layout recognition---identification of section headings, page numbers, etc.)

So a single 18MP camera would give you a resolution that's similar to what you could get with two 3MP cameras in current designs. Maybe you'd really need a Leaf Aptus to do this properly with one camera.

I didn't know about the 3D stereo lenses. I had a look at some Web sites that sell them, and they look really nice. Obviously only for professional cameras, but I'm guessing they'd work better for both focusing and 3D image processing than the DIY post-lens periscope I suggested.

So maybe the least expensive way to do this is in fact with two good consumer cameras. Decapod is using two 14.7MP Canon G10 cameras, and suggests as a minimum resolution 12MP. http://wiki.fluidproject.org/display/fluid/Hardware+Design Looks like a great project! (Regarding their software---OK, now, this should really go in a Software forum thread---I wonder what the main differences are with ScanTailor---I mean, other than programming language and UI differences, what are the major differences in the algorithms and user processes? Maybe some synergy between the two codebases could be achieved? Maybe ScanTailor could have as an option the inclusion of stereo/3D data in algorithms, so that it could be taken advantage of when appropriate?)

[daniel_reetz]

Hey Andrew,

Yeah, resolution is one issue (varies according to book aspect ratio and size), another is stereo point correspondence, a third is keeping your mirrors clean (you'll need to use first-surface mirrors, which are not cheap or easy to keep clean), and a fourth is lighting. However, I wouldn't discourage you from pursuing this at all, certainly those are all things that can be overcome and we can all use better stereo algorithms. There are also commercial scanners using a flipping mirror to accomplish this - here's one http://diybookscanner.org/forum/viewtopic.php?f=1&t=726 , I know I've seen others.

I'm personally investing quite a bit of time into some other more optical/computational approaches:

viewtopic.php?f=1&t=788

I hope you keep us up-to-date here on your progress - it's exciting work and I for one would like to see more of it around here. Glad to see you are taking an open source approach, too.

[andrewgreendf]

OK, thanks tons for your input, Dan. I'll look into all the issues you mentioned. I wonder...does anyone ever expect consumer cameras to get up to the 40+ MP range...? Is there a Moore's law for camera resolution?

I'm personally investing quite a bit of time into some other more optical/computational approaches

The thread you started on that topic is great. I'll post general comments/ideas on 3D imaging there, then, and post to this thread only stuff specifically related to the double periscope idea. (I assume that's the best way to proceed...)

Glad to see you are taking an open source approach, too.

Wouldn't have it any other way!

[Anonymous1]

I'm a deleted post. You can't see me.

[daniel_reetz]

Some literature on the topic

http://www1.cs.columbia.edu/CAVE/public ... PAMI02.pdf

[jonx]

Yeah, resolution is one issue (varies according to book aspect ratio and size), another is stereo point correspondence, a third is keeping your mirrors clean (you'll need to use first-surface mirrors, which are not cheap or easy to keep clean), and a fourth is lighting. However, I wouldn't discourage you from pursuing this at all, certainly those are all things that can be overcome and we can all use better stereo algorithms. There are also commercial scanners using a flipping mirror to accomplish this - here's one viewtopic.php?f=1&t=726 , I know I've seen others.

Hi everyone,

I too have been bitten by the scanner bug, however, I have a DSLR which I would very much like to use and therefore I have been looking for a "Single Camera Design". I know there are other single camera designs on this forum but I am looking for something more. Assuming a platen similar to those outlined in this forum is used and assuming the camera is directly above it then we have to find a way to compensate for the angle of the book on the platen which could be 90 degrees or more.

The way I see it there are three main ways to achieve this compensation:

1. Digitally (using software)
2. Mechanically (by moving the camera or the book)
3. Optically

I believe this represents one option, at optical compensation. I will be presenting my ideas for "mechanical compensation", in a separate thread in the near future. I believe enough work is being done on digital solutions.

Optical Compensation

I see the main benefit of this approach is that it allows one to place the book on a standard 90 degree (DIY Book Scanner) platen and photograph both pages with a single camera and optically compensate for the angle of the two pages so they appear flat to the camera! I have built a rough prototype based upon some of the ideas in this thread which I will post shortly. My initial prototype uses conventional mirrors, however, as Daniel points out above "first surface mirrors" will be needed for the final design.

To address some of your concerns Daniel:

Resolution:
12 MP is probably the bare minimum here giving you 250 dpi for an 8.5" x 11" book (17" x 11" image), 17 MP would be better at 300 dpi (assuming 5100 x 3300 pixels).

Keeping the Mirrors Clean:
I do not have the complete solution for this yet but I feel that as long as the enclosure is dust free at assembly (easier said than done, I accept) and if it is sealed with either acrylic or glass we should be good.

First Surface Mirrors:
Yes they are more expensive than conventional mirrors, however, in glass or acrylic they are not too expensive, starting at $28 for a 12" x 12" sheet of 1/8" acrylic, more than enough for this build. http://www.firstsurfacemirrors.com/acrylicfirstsurfacemirrors.htm

Anyone?

Jonathan.

[spamsickle]

I'm very interested in seeing your mirror design, especially if it provides same-sized rectangular images of both pages in a single photo.

If it doesn't, I'd at least consider doubling your scan time and shooting one side of the book at a time. I'm doing that now for some of my larger books, and I think it's a good compromise. I could probably save the time by shooting both sides of the book at once and "squaring up" the keystoned pages in software, as you mentioned, but I want the extra resolution in this case.

Looking forward to seeing what you've come up with.

[jonx]

I'm very interested in seeing your mirror design, especially if it provides same-sized rectangular images of both pages in a single photo.

Here is my proof of concept prototype:


In its current configuration both pages will be displayed side by side except the right page will be on the left and the left on the right.

So yes "same-sized rectangular images of both pages in a single photo" in the two center mirrors.

As it is not clear in the image, the two center mirrors are at 45 degrees (off vertical) and the two outer mirrors at 22.5 degrees (off vertical). In this prototype the angles of all the mirrors can be changed. The mirrors are 6" x 5" and the unit is 21" x 5" x 7" (wxhxd). I built it using 1/4" MDF, just what I had on hand.

I would like to reduce the size in the next version although I am not sure how much would be feasible. Any ideas anyone?