Methods To Sense The 3D Surface/Structure Of A Book

[Anonymous1]

This is basically where I am with the dewarping. That radial warping and camera offset does make a pretty big difference, so I'm still trying to iron the bugs out of my code. Just for reference, Steve, how long does it take Java to completely dewarp an image? I'm just wondering if I'm actually getting a performance boost by using OpenCV...

In case you're interested, here's what I was using to correct for radial distortion:

Just one page...

output.png (179.33 KiB) Viewed 955 times

I was getting black rings (PIL doesn't like float coordinates), so I just used the GIMP to correct this beforehand. How were you correcting for the radial distortion originally with JAI?

[steve1066d]

Just for reference, Steve, how long does it take Java to completely dewarp an image? I'm just wondering if I'm actually getting a performance boost by using OpenCV...

Its definitely slower than your OpenCV stuff.. 11 seconds for the full size image (that include reading and writing the image). Its on a dual core machine but its only using one cpu right now.
My approach is doing more processing, but I'd be very surprised if the JAI was faster than OpenCV.

How were you correcting for the radial distortion originally with JAI?

I created custom warp method. Here's a snippet of code:

Code: Select all

   
        double y1 = y - focusY;
        double r = Math.sqrt(x1 * x1 + y1 * y1);
        double s = 1;
        if (amplitude3 != 0) {
            s += amplitude3 * r * r * r;
        }
        if (amplitude2 != 0) {
            s += amplitude2 * r * r;
        }
        if (amplitude1 != 0) {
            s += amplitude1 * r;
        }
        destRect[index++] = (float) (s * x1 + focusX);
        destRect[index++] = (float) (s * y1 + focusY);


I was using something like amplitude3 = 1E-7, with the other amplitudes 0.

I think the issue with the black rings is that you are running into issues with the forward warp you are using. basically there's no guarantee that there will be a corresponding destination pixel for every source pixel you plot.

[Anonymous1]

That seems to be the exact same thing as I posted, only split up a little. Amazingly, we get the exact same time for a full-sized image; I also get ~ 11 seconds when doing a full-sized image.

I'm not worried about speed right now, though. When I was playing with different warping techniques, I somehow made a script which takes over 20 minutes to dewarp a single scaled-down image. It was pretty cool, though, to watch it mangle images

Do your dewarped images get compressed near the crease area? My naïve approach just scales columns of pixels, which makes the text in the crease area more condensed than the text in the outer edge. I'm working on fixing that right now...

[andigit]

I've implemented dewarping in my code (not committed yet) and I have to say that it isn't fast. Also to add to the slowness you also need to implement some type of interpolation algorithm for the missing pixel. I tried bilinear and bicubic and the result looked almost the same.

The fastest I've found is pre calculate all the x,y coordinate and create a table... Lens distortion shouldn't change from image to image so calculate once apply to all.

My code is gui based so I'm creating a section in my program to manually specify the calibration and I create a table based on the number. When I go to actually dewarp, I'm not doing any calculation. This gives me a significant performance boost.

[andigit]

Algorithm looks exactly the same as mine as well

These are the 2 reference I used...
http://stereopsis.com/old/barrel.html
http://www.roborealm.com/help/Radial.php

[Anonymous1]

As I don't have access to a machine with compiling capabilities (I'm on a Windows rig), could you upload a binary or some input/output images?

We have 3 dewarping methods already, so I'd love to see the guts of your's too. I wonder how many more we can manage to implement before we exhaust all of the possible methods

As for the lens distortion, as you said, the values are constant. OpenCV uses a 3x3 matrix for the camera calibration and a 5x1 matrix for the distortion coefficients, so after you calibrate the camera well, you can just dump them to a config file and recycle them.

I was thinking about the camera calibration, and I think that using a checkerboard isn't a good idea (or at least not the best). I'm going to try to make a GUI application (or if one exists, I'll just use that) and manually dewarp a few images to find the distortion coefficients and the camera matrix.

[andigit]

I'm at work so I don't have access to my code right now but I use the following algorithm which I found on the webpage I mentioned above.

Code: Select all

r = sqrt(x^2+y^2);
z = P0 + PR*r + PR2*r*r + PR3*r*r*r + PR4*r*r*r*r;
s = -S/z;

X = x*s;
Y = y*s;

x = original x
y = original y
S = scale
P0 ~ PR4 = coefficient
X = destination X
Y = destination Y

I like this algorithm because of the scale. It won't squish your result if you put the right number in. This algorithm is almost exactly what Anonymous is using.

[Anonymous1]

That's the only radial distortion algorithm AFAICT It follows directly from the mathematical model of a lens.

Did you say that you wrote your whole application with Qt? I think that I would be able to compile it at home, as ST uses Qt and C++, and I compile that weekly.

[andigit]

Yeah, mine is written in qt and at the moment I'm not using any extra libs. So, if you grab my code and "qmake; make" you should get a binary.

I'm on mac but it should compile on all platform qt is supported on.

I should be able to finish up the calibration ui and commit my changes some time tomorrow. Works been keeping me busy and haven't had too much time.

[steve1066d]

Do your dewarped images get compressed near the crease area? My naïve approach just scales columns of pixels, which makes the text in the crease area more condensed than the text in the outer edge. I'm working on fixing that right now...

I do handle that. I break the book up in to small colums, calculate the change in height over the range (doing a separate calcuation for the top and bottom of the page). This breaks it up into a lot of skinny sections which I then render using a JAI's GridWarp. I do that transform after the the perspective transform.

Here's the guts of the code I use for calculating the change per sample:

Code: Select all

double delta = height - lastHeight;
cellDistance = Math.sqrt(delta * delta + inchesPerSample * inchesPerSample) ;


The rest of the code is all checked into subversion, if you want to check it out.