We have fixed some issues with the board email system. If you have not received an email for password reset or verification, please try again.

Methods To Sense The 3D Surface/Structure Of A Book

DIY Book Scanner Skunk Works. Share your crazy ideas and novel approaches. Home of the "3D structure of a book" thread.
User avatar
daniel_reetz
Posts: 2798
Joined: 03 Jun 2009, 13:56
E-book readers owned: Used to have a PRS-500
Number of books owned: 600
Country: United States
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by daniel_reetz »

Yeah, I'm definitely interested in very quickly trying out a few things to see what difficulties we come across. I definitely don't want to throw out anything without trying it.

A few thoughts.
1. Point and shoot cameras continue to get better and better. We have 16mp compacts out there (with the usual quality caveats). In a year or two, single-camera solutions will be the norm.
2. Lasers are really cheap. I ordered a set of line-generating lasers for $3 each from DealExtreme. I'll order more if they turn out to be any good.
3. The green channel of a Bayer-patterned sensor has the highest resolution.
4. It's easier to deal with sequential images from one camera than many images from many cameras, potentially with sequential errors.
5. The basic image geometry for the laser beams is just trigonometry, once you find the line.
6. ST already dewarps using two lines (maybe we can re-use some of ST)

However, I do like the directness of your idea - want to sense the thickness of a book? Take a picture of it!

Honestly, I think the challenge will be the same in both instances, but my hand-waving intuition is that sensing a bright green line may be easier than sensing the edge of a book against some background.

Do you have any time in the coming weekend? I'd bet we can get a lot of sample images done if we were working together at my place.
spamsickle
Posts: 596
Joined: 06 Jun 2009, 23:57

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by spamsickle »

I should be available all day Saturday, and Sunday until about 2:30, unless you end up having to work anyway, in which case I was busy too, maybe some other time.
kasslloyd
Posts: 41
Joined: 19 Dec 2010, 21:25

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by kasslloyd »

Yea scattering will be a problem, you'll need good line generators. Also green lasers I think are too bright, so they'll create pretty "fuzzy" lines. You want fairly low power dim clean red line generators. IMO

What advantage this would have is that it's essentially doing what google is doing, except removing the IR. This could be done with IR, just take a cheap digital camera and take it apart, remove the IR blocking filter infront of the sensor and blamo, you have a digital camera capable of seeing IR lines, then you'd need a IR laser grid generator... still thats probably more expensive.

$8 line generators, at most 10 of them for a grid, or maybe there exists cheaper solutions to generate a checkerboard type grid with lasers, then photograph it in visible light.

The key here rests with software, is there any open source algorithms to dewarp with this projected grid pattern?
User avatar
daniel_reetz
Posts: 2798
Joined: 03 Jun 2009, 13:56
E-book readers owned: Used to have a PRS-500
Number of books owned: 600
Country: United States
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by daniel_reetz »

Why do we need a grid? Right now I am thinking two lines are enough.

Re: IR cameras, yep, I'm familiar. :) But I don't want to be using IR because we'll likely be infringing on Google's patent, and that's not going to help us stay in business.

I'm not yet worried about the line quality at all. We can improve it somewhat with a baffle and/or pinhole. But I'll order a couple of red line generators anyway. Even if it is fuzzy, we can threshold it or use that information to produce a better line (it will get wider as the angle gets more oblique).
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 »

How is this done so quickly? http://www.youtube.com/watch?v=tCOXC5PTJj8

I'm not sure about the hardware, but just from watching, you can tell that it's really high-speed line tracing. I think that the whole process can be simplified and done completely without lasers and overkill hardware. How would a human dewarp mentally? I would determine the height of a line of text, it's relative offset from the imaginary line that determines the edge of a page, and the spacing between neighboring lines. Sure it's hard to code, but it's not impossible. Tulon's coupled snakes approach does most of this quite well (it just slips up when choosing representative lines ;) )

From all this data, you could tell what should be done to correct a page, so the reverse could generate a 3D model. Just a thought...

The easiest approach for me would be to just setup a few cameras looking at the book perpendicular to the text, so that you could have two images of the curvature of the whole book's content. Then, you just need the dimensions, the lens distortion for all the cameras, precise distances for all the components, and some good extrapolation methods. From there, it's all good.

I'll play with this method (if any at all), as I don't have the money to buy sensors and IR cameras...
kasslloyd
Posts: 41
Joined: 19 Dec 2010, 21:25

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by kasslloyd »

Re: the video, they're using a ultra-high speed HD camera + the IR 3d sensing method to in real time map the 3d structure of every fame of the high-speed camera, then using software to figure out which frame is best for each page after dewarp. I think. ;-)

This seems fairly impractical since you can't really flip accurately and not miss pages, and to write software to know where a page miss was could be difficult. It's just a "proof-of-concept", probably eventually be coupled with a robotic page turning apparatus that can turn at very high speeds... I've seen it a while ago but didn't seem very practical.
kasslloyd
Posts: 41
Joined: 19 Dec 2010, 21:25

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by kasslloyd »

daniel_reetz wrote:Why do we need a grid? Right now I am thinking two lines are enough.

Re: IR cameras, yep, I'm familiar. :) But I don't want to be using IR because we'll likely be infringing on Google's patent, and that's not going to help us stay in business.

I'm not yet worried about the line quality at all. We can improve it somewhat with a baffle and/or pinhole. But I'll order a couple of red line generators anyway. Even if it is fuzzy, we can threshold it or use that information to produce a better line (it will get wider as the angle gets more oblique).
Google's patents probably mostly pertain to the software/math side, do you know what the patent #'s are? Even still staying away from IR will reduce costs, imo. So thats another benefit. IR just allows you to go faster since you can shoot one picture.

I wonder if you can get a glass filter that blocks everything but the nm range of the laser light, and a filter that blocks the laser light only... then you could basically do what google does without IR, project the laser light continuously and take both pictures at once. That would dramatically increase costs though for those filters.. heh.

I'm personally for doing it with minimal amount of pre-processing of the images as possible... but writing the software to accept everything from poor quality lines that have lots of bleed light and not positioned 100% accurately to perfect grids would be good.

as for the grids... I donno much about the algorithms involved so I can't say.. I just know that sensing depth usualy involves grids, like the Kinect sensor and google's dewarping tech both use grids... the work being done on the software for the scannable checkerboard to dewarp the page...

My thoughts is if you project a checkerboard pattern, a grid, the same algoritms that is used in that software might work?

It really boils down to the software, do we have an idea how to create it?
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 »

kasslloyd wrote:This seems fairly impractical since you can't really flip accurately and not miss pages, and to write software to know where a page miss was could be difficult. It's just a "proof-of-concept", probably eventually be coupled with a robotic page turning apparatus that can turn at very high speeds... I've seen it a while ago but didn't seem very practical.
You can probably just dump the final page output and flip through the book a few times, and then just use OpenCV to quickly look for duplicates. But I do agree, this method is useless for older books.
kasslloyd
Posts: 41
Joined: 19 Dec 2010, 21:25

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by kasslloyd »

The cost of the high speed camera though could buy a bunch of conventional high quality book scanners though.. heh.

So unless you was going for (a) limitless budet for digitizing and (b) completely automated system with minimal staff.. I also couldn't see justification in purchasing such a camera...
User avatar
Gerard
Posts: 154
Joined: 17 Oct 2010, 07:15
Number of books owned: 0
Location: Berlin (Germany)

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Gerard »

the software seams easy,

but i am also thinking that a line is not enough, if the camera is 100% perpendicular to the book and the line, then the camera sees just an straight line, so to see the wave we need to look from the side, but in which angel? and this angel is creating an new distortion (as long you book scaner and cameras are on fixed places it can be corrected witch one measurement)
we need dots like this one http://cgi.ebay.de/6-1-Laser-Pointer-Ze ... 4394wt_907

even a random pattern could be help, then we need just train the software with a none warped page what the pattern looks like

pattern matching can work with very low signal strange, so it will be maybe possible to dim the laser and later remove from the image, this would allow to take one image per page

software:
http://biocomp.cnb.uam.es/~iarganda/bUnwarpJ/ if its posible to gain the laser signal (by software) and reduce the page content, if you have a random pattern and a image which is none wrapped and a page which is wrapped bunwrapj will find the deformation map

but it can say, this will need cpu time
Post Reply