Dewraping with focus stacking
Moderator: peterZ
Dewraping with focus stacking
Hi, i just wont to ask, how accurate (minimal step) it is possible to change the focus, from a series of images with different focus, it is maybe possible calculate a 3d map
and then dewrap the page(s)
if we have a stack (maybe a video with continues changing focus) it is possible to calculate the local sharpens (e.g. variance filter) and then searching for every pixel the focus image with the highest sharpens
if some one is interested in this i could try it, the software part (i don't have a chdk enabled camera)
http://www.google.com/url?sa=t&source=w ... EeUFVSzqpg
http://en.wikipedia.org/wiki/Focus_stacking
and then dewrap the page(s)
if we have a stack (maybe a video with continues changing focus) it is possible to calculate the local sharpens (e.g. variance filter) and then searching for every pixel the focus image with the highest sharpens
if some one is interested in this i could try it, the software part (i don't have a chdk enabled camera)
http://www.google.com/url?sa=t&source=w ... EeUFVSzqpg
http://en.wikipedia.org/wiki/Focus_stacking
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Dewraping with focus stacking
Two problems with this approach for DIY scanning:
Compact cameras have a large depth of focus (in focal region) so getting good defocus can be problematic.
Taking many pictures of one page can take as long as scanning on a flatbed, so it becomes less attractive.
That said, I can provide you with some example bracketed focus shots next week. Are you a programmer?
Compact cameras have a large depth of focus (in focal region) so getting good defocus can be problematic.
Taking many pictures of one page can take as long as scanning on a flatbed, so it becomes less attractive.
That said, I can provide you with some example bracketed focus shots next week. Are you a programmer?
- daniel_reetz
- Posts: 2812
- Joined: 03 Jun 2009, 13:56
- E-book readers owned: Used to have a PRS-500
- Number of books owned: 600
- Country: United States
- Contact:
Re: Dewraping with focus stacking
Another issue is that for non-textured areas (white space) focus and defocus are underdetermined/ill specified.
I don't say this to discourage you in any way, though, I am a computational photography hobbyist and have read literally hundreds of papers in computer vision, image processing, and computational photography looking for solutions like this. I'd be happy to see you try it.
I don't say this to discourage you in any way, though, I am a computational photography hobbyist and have read literally hundreds of papers in computer vision, image processing, and computational photography looking for solutions like this. I'd be happy to see you try it.
Re: Dewraping with focus stacking
i think i would be enough to take this bracketed focus shots at the beginning or in an interval, calculation the deformation grid and using this on all imagesdaniel_reetz wrote:Two problems with this approach for DIY scanning:
Compact cameras have a large depth of focus (in focal region) so getting good defocus can be problematic.
Taking many pictures of one page can take as long as scanning on a flatbed, so it becomes less attractive.
maybe i miss the point, do we need for every page the 3d map?
or we could also use a reference picture with a grid,
would be nicedaniel_reetz wrote: That said, I can provide you with some example bracketed focus shots next week.?
yes i hope i can contribute something, but i should invest the time more in the wood workdaniel_reetz wrote:Are you a programmer
yes i had this problem also with my scanner, blank page with small amount of text was blured (auto focus problem)daniel_reetz wrote: Another issue is that for non-textured areas (white space) focus and defocus are underdetermined/ill specified.
but in this case maybe it is ok to use the data from images before
i am wondering why no one build a 3d scanner with this, maybe it will fail, or not many people had contact to chdk, with manual focus it is not accurate and auto focus does not give you the singe defocus imagesdaniel_reetz wrote: I don't say this to discourage you in any way, though, I am a computational photography hobbyist and have read literally hundreds of papers in computer vision, image processing, and computational photography looking for solutions like this. I'd be happy to see you try it.
-
- Posts: 596
- Joined: 06 Jun 2009, 23:57
Re: Dewraping with focus stacking
CHDK permits one to set the focus down to the millimeter, though in practice I think the limitations of the mechanics will make it less accurate than that.Gerard wrote:Hi, i just wont to ask, how accurate (minimal step) it is possible to change the focus, from a series of images with different focus, it is maybe possible calculate a 3d map and then dewrap the page(s)
It took me a couple of reads through this thread to understand what you were proposing. While I question how well this would work, I'm more than happy to provide test images. As far as I know, cameras don't capture focus distance in their EXIF data*, so I'm curious to begin with how you would build your 3D map. I suppose if we agree that each image in a stack is focused 1mm from the last image, and you really can determine which image is "most focused" at a given pixel (the second thing I'm curious about), you could build a relative depth map. Relating one pixel to its neighbors and resolving inconsistencies seems like the third hurdle.Gerard wrote:if some one is interested in this i could try it, the software part (i don't have a chdk enabled camera)
*EDIT: I just tried an online EXIF viewer, which reported the focus for a sample book scan as "86 cm to infinity", which seems about right. I don't know what the precision is, but apparently this IS part of the EXIF data in some form. The Times regrets the error...
It also seems to me that, if the camera is centered over the book, the "corners" of the book are further away than the "center" of the book, by substantially more than the typical "curl" from gutter to edge. If one truly determines distance by using focus, is this going to be something that the dewarping step can handle?
Doubts notwithstanding, I have a public domain "Harvard Classic" which, when opened to the middle of the book, bows up a couple of centimeters at most. Would you prefer a 400-image stack at 1mm nominal difference, or a smaller stack with a larger "step" between images?
Re: Dewraping with focus stacking
i hope that it will be possible to interpolate the data by fitting functions in the data, only the step size should be fixedspamsickle wrote:CHDK permits one to set the focus down to the millimeter, though in practice I think the limitations of the mechanics will make it less accurate than that.Gerard wrote:Hi, i just wont to ask, how accurate (minimal step) it is possible to change the focus, from a series of images with different focus, it is maybe possible calculate a 3d map and then dewrap the page(s)
i will not relay on the exif data, i think it is not imported to know the real distance, a relative distance of a pixel to an other pixel is maybe enoughspamsickle wrote:It took me a couple of reads through this thread to understand what you were proposing. While I question how well this would work, I'm more than happy to provide test images. As far as I know, cameras don't capture focus distance in their EXIF data*, so I'm curious to begin with how you would build your 3D map. I suppose if we agree that each image in a stack is focused 1mm from the last image, and you really can determine which image is "most focused" at a given pixel (the second thing I'm curious about), you could build a relative depth map. Relating one pixel to its neighbors and resolving inconsistencies seems like the third hurdle.Gerard wrote:if some one is interested in this i could try it, the software part (i don't have a chdk enabled camera)
my first scan was bad aligned, the page was not parallel to the camera, i had a warped image and the part of the page that was more far away was blurred, so i came up with the idea to use the blurring a way to measure the distancespamsickle wrote: It also seems to me that, if the camera is centered over the book, the "corners" of the book are further away than the "center" of the book, by substantially more than the typical "curl" from gutter to edge. If one truly determines distance by using focus, is this going to be something that the dewarping step can handle?
i prefer the stack with the small distance (400-image)spamsickle wrote: Doubts notwithstanding, I have a public domain "Harvard Classic" which, when opened to the middle of the book, bows up a couple of centimeters at most. Would you prefer a 400-image stack at 1mm nominal difference, or a smaller stack with a larger "step" between images?
-
- Posts: 596
- Joined: 06 Jun 2009, 23:57
Re: Dewraping with focus stacking
I haven't forgotten you. I took a couple of sequences yesterday -- let the autofocus calculate the distance (680 mm), then bracketed it between 480mm and 880mm. I couldn't see anything in the extreme images which was out of focus. Then I tried 380 to 1080 with a step of 5 mm between them, and the results were the same, visually. I think the "minimum focus" when not in macro mode is something like 510 mm, so maybe attempting to set it lower prevented the rest of the sequence from working. I'll shoot another sequence today, with 510 mm as the lower bound all the way to 10,000 mm or so, just to make sure the set_focus command is working as documented. I should have something for you in a day or two -- either a sequence of test images, or a lame excuse.
Re: Dewraping with focus stacking
thanks
i looking forward to the result
i looking forward to the result
-
- Posts: 596
- Joined: 06 Jun 2009, 23:57
Re: Dewraping with focus stacking
Okay, Dan's comment about depth of field was spot on. Taking the pictures varying the focus over a small range yielded pictures which I couldn't distinguish from one another at all. What I ended up doing is generating 300 pictures, while varying the focus with CHDK from 510 mm to 2010 mm. Even then, the 2010 mm focus was only a little blurry, still easy to read. Makes me doubly annoyed at those autofocus pictures that come out unreadable when there isn't enough text in the center where it's focusing -- I think the camera must just be focusing at infinity when that happens.
So anyway, the pictures have been split into 3 zip files, because the site where I'm hosting them has a 200 MB limit on each file. Here they are:
http://www.4shared.com/file/NgNwnTmn/FocuStk1.html
http://www.4shared.com/file/znY-0Z22/FocuStk2.html
http://www.4shared.com/file/NEiRr1bo/FocuStk3.html
Good luck, and let me know what might work better if these turn out to be not what you need.
So anyway, the pictures have been split into 3 zip files, because the site where I'm hosting them has a 200 MB limit on each file. Here they are:
http://www.4shared.com/file/NgNwnTmn/FocuStk1.html
http://www.4shared.com/file/znY-0Z22/FocuStk2.html
http://www.4shared.com/file/NEiRr1bo/FocuStk3.html
Good luck, and let me know what might work better if these turn out to be not what you need.
Re: Dewraping with focus stacking
ok my first test between 2 images,
the images have to be aligned, i've excepted this
the other problem seams to be: i see a lot of rectangles in the difference images, jpeg compression rectangles, i will try to eliminate this by averaging some neighbor pictures to one
i will try more pictures
the images have to be aligned, i've excepted this
the other problem seams to be: i see a lot of rectangles in the difference images, jpeg compression rectangles, i will try to eliminate this by averaging some neighbor pictures to one
i will try more pictures