Tutorial: 'Clocking' Cameras for Sequential Page Order

[univurshul]

'Clocking' cameras is a very basic approach that addresses sequential page order to books and media during post processing

Why should I 'clock' my book-scanner cameras?
Simply put, it makes sequencing book pages very easy. When you import images from your book-scanner cameras into your image editor, you will be able to sort and process photos according to their 'capture time'.

What software application(s) are necessary to make 'clocking' possible?
Your image importer/editor. (Not every image editor app will able to sort photos based on capture time 'seconds'. You will need to check if this is possible on your software.)

What cameras have been tested successful for 'clocking'?
As of 10/15/2010, I have tested the following to cameras support "clocking':
--Canon A590IS (the most widely used camera for book scanning)
--Sony Nex-5
--iPhone 4

Is camera 'clocking' necessary to achieve sequential book-page ordering?
No. There are other methods and apps on both Mac and PC to do this. For example, here is an alternate method for OSX users: http://www.diybookscanner.org/forum/vie ... ?f=3&t=527 'Clocking', on the other hand, uses the metadata inherit in each photo captured by your cameras to order pages intuitively. It works with both dual-camera and single-camera scanners. When setup correctly, 'clocking' eliminates the extra time normally expelled for sorting & ordering pages.

How do I clock my cameras?
This is very simple: the right camera should be clocked to the 'real' time on your computer. The left camera will be clocked 2-3 seconds later than the right camera. That's it. Done.

(When I refer to 'right' and 'left', it is facing the book scanner as you would normally operate it. i.e., right camera shoots page 1, and the left camera shoots page 2. 'Clock' accordingly.)

Most digital cameras have decent internal clocks, but they don't show "seconds". That's OK, it shouldn't matter; most cameras capture to the "seconds" even when their firmware doesn't illustrate the "seconds" in detail. For example: here is a screenshot of a Canon that doesn't have a clock capable of showing "seconds", but it in fact records 'capture times' all the way down to "seconds":

1.tiff (230.05 KiB) Viewed 10178 times

2.tiff (2.62 KiB) Viewed 10178 times

Practice manually setting 'clocking' by timing this 2-3 second delay on your cameras. Adjust the "minute" time on the corresponding time-change that you established with the right camera by changing the minute 2-3 seconds around a known minute changeover. Exit the firmware to run tests if your clocking procedure was effective by importing all the test images from both cameras into your image editor and sort the images by "capture time". If it worked, you'll see all your images staggered like sequentially ordered book pages. Pretty cool. Organization made easy. Just physically label your cameras appropriately to avoid potential confusion should you dismount the cameras from your scanner arms.

KEY DATA TO CONSIDER:
How often you're required to re-clock your cameras depends on the camera. You'd be surprised how maintenance-free it is granted your cameras' internal firmware batteries are fresh.

If you scan at very fast rates, you may need to consider clocking your cameras at 1-2 seconds apart as the need to prevent time-overlap is evident (This is rare, because the limit of a DIY scanner is usually less than 1,300 pages/hr). Attempt a dry-run on simple text-body books that normally scan fast. Use a stopwatch positioned within sight and notice how much time has elapsed from trigger press-to-trigger press from page turning and platen raising/lowering. If the time to get from capture to capture exceeds your camera clocking delay, you are in an ideal sequential ordering zone, or sweet-spot.

A more accurate snapshot of your scanning speed is to wire only one camera for scanning; simulate a very fast scan and capture as you normally do for about 2 minutes--get a nice, fast rhythm down; import these images into your photo editor; view the metadata capture time and briefly glance over images shot within 1 minute intervals, 30 second intervals, and finally between each camera fire; here, you can visually see and calculate with simple math how far apart the images were captured in time; clock the cameras accordingly to assure you will have a low probability of image-time overlap.

Skilled users who employ STM/CHDK know very well that there are methods to delay/sync-fire their cameras. This obviously works too, but delay-firing of cameras means that the sum-time of the book scan will be lengthier than that of simultaneously-appointed captures. Comparatively, clocking the cameras' firmware reduces the amount of required time the platen sits over the book and avoids the need to delay any firing to achieve sequential book-page ordering once imported and intermixed on image processing software.

We also have to understand there are a growing number of cameras that simultaneously fire from a single infrared remote controller. These cameras may not have hackable firmwares, and therefore, 'clocking' is an even stronger argument to liberate cameras otherwise not compliant with STM/CHDK.

TROUBLESHOOTING:
It didn't Work. What did I do wrong?
Make sure the year/month/day are identical.
Many camera firmware clocks are setup in military-time. Make sure that you have the exact same times with exception to the delay of the seconds from the right to the left camera. Physically position the cameras next to each other, wait and watch for the time change. If the cameras 'clock' successfully, the error is likely occurring in the software. Make sure you are sorting the view in the image editor based on ascending capture time.
Make sure it works before you scan an entire book. Run a test of 10-20 images. It's also important to physically identify what is your right and left camera. Confusing the two will result in errors.

Not all cameras and not all image editing software supports 'clocking'. Report your experience or any extra suggestions here.

[spamsickle]

Personally, I'd be reluctant to depend on clock time to sort my images, for a number of reasons. If an image is out of focus, for instance, and I go back later and re-shoot it, I'd need an EXIF editor to put it in the sequence correctly.

Since virtually all cameras assign ascending names to images, I've been using the name to sort them, and keeping my left and right originals as well as my processed end product. This thread discussed what I did with a Perl script, and prompted jakegaisser to post what I considered to be a better way, which didn't require Perl. I modified the improvement slightly, to satisfy my preference for keeping the originals and for using Scan Tailor rather than Image Magick to rotate the images, but anyone who prefers can use the original, or make their own changes. Here's a working script for Windows:

Code: Select all

echo off
set count=0
FOR /r %%A IN (*.jpg) DO CALL :NUMBER %%A
goto :EOF
:NUMBER
IF "%count%"=="0" (
   set count=1
   set odd=1
) ELSE (
   IF NOT "%~p1"=="%PREVDIR%" (
      IF "%odd%"=="1" (
         set count=2
         set odd=0
      ) ELSE (
         set count=1
         set odd=1
      )   
   )
)
set NUM=000%count%
set NUM=%NUM:~-4%
copy "%1" "%NUM%.JPG"
)
set /a count+=2
set PREVDIR=%~p1
goto :EOF

To use it, make sure the script (mine is called "DIYmerge.cmd") is in your path somewhere, and create a new directory for each book you process. Within the book directory, create subdirectories L and R, to store the left-page images and right-page images. From the root book directory, invoke the script. The left and right images will be copied to the root directory, with a new 4-digit name. The only real "gotcha" I'm aware of is that the script assumes the first image in the "left" directory contains content. In other words, don't include in the "L" directory the blank image taken by the left-page-shooting camera while the right-page-shooting camera was photographing the cover of the book.

[univurshul]

Personally, I'd be reluctant to depend on clock time to sort my images, for a number of reasons. If an image is out of focus, for instance, and I go back later and re-shoot it, I'd need an EXIF editor to put it in the sequence correctly.

That is a non-issue. Here's why: If your image editor has the capabilities to sort images according to 'capture time', changing metadata (a.k.a. EXIF editor for noobs) is highly incorporated into that environment of the software already. If you discover out-of-focus pages, it is very easy to re-insert newly focused shots--by a number of options presently inherit to the software with this exact purpose of flexibility in mind. Even if you delete the entire book from the image editor, you could theoretically save every preset, such as white balance, lens distortion, newly order page numbers, etc.; go back into the image editor; apply the preset and send it off to Scan Tailor again. And typically, I find most of my out-of-focused pages within Scan Tailor anyway. So, having a palette of saved presets for the above reason is a great idea now that you mention it.

Can you think of any other reasons why 'clocking' would give you caution?

Since virtually all cameras assign ascending names to images, I've been using the name to sort them

Nearly all cameras assign ascending capture times to images as well. I am simply taking advantage of the same metadata you are, and trimming unnecessary post-processing procedures.

I've scanned 25 books without a single issue, nor single extra script or procedure to do the job. I don't even have to think about correct page order after I scan, because this method created intuitive metadata as I scanned--not after.

Let's take a step back and look at the big picture for a minute: good image editing software at your fingertips to employ this method is essential. This doesn't mean you need expensive software nor does it bash free & intelligent scripts; it is simply another option. But if you scan with the intent to have quality reproductions that involve other steps like white balance, lens distortion, keystone correction, then I have exciting news for you: there are great tools on the horizon that utilize interesting bits of metadata to sort your photos. You won't need to run scripts because sorting--auto or manual--is no longer a requirement; the image editing software does it for you. All you need to do is tell the cameras what time it is.

If you have a workflow that is successful at producing the results you require, then 'clocking' may not be something you'd want to consider. I implied this line of thinking in the initial post in FAQ intro.

There is a difference between 'file date' and 'capture time'.

Although this approach might not be as well-known as Spamsickle's, it could easily be as popular if users are aware of the idea's basic premise. Although I can only guess others have employed this method in various photography applications, it appears 'clocking' has surfaced on DIY with a very simple yet specific purpose. It is here, spelled-out for everyone to explore .

I believe it works just as well if not easier, and just as fast, if not faster than a script. I wouldn't have bothered using it nor posting on it as a direct technique in book scanning with a dual-camera rig if I tested otherwise. (I haven't tested Spam's sort method, so I cannot compare with detailed results nor could I begin to speculate on what is better for others.)

And there are many DIY members who don't understand a lick of code for various reasons; those who don't use Windows, those who don't use Linux. 'Clocking' might be a more code-resiliant, time-saving, easier approach that looks toward a more seamless environment. It's very accurate and so sweeet to import a ton of images from 2 cameras and not have to do a thing.

Hope this makes more sense...

[spamsickle]

It's not clear to me how you go from "all the images are sorted in your image editing software" to "all the images are sorted in Scan Tailor." As far as I know, Scan Tailor is only sorting by name, not by capture time.

It sounds like you're using the image editing software to do more than simply re-order the images, but I'm doing my white balance in the camera, aligning the camera when I shoot to avoid "keystoning", and don't know what lens distortion correction would buy me. The only post-shoot pre-Scan Tailor processing I do is merging the left and right pages, and for that I'm happy with a script. I think you're the connoisseur who can actually taste the difference between the lens-distorted bottle and the corrected bottle, while I'm the guy who can't hear the difference between MP3 and vinyl.

It's good that people have a choice. My choices usually involve keeping things as simple as possible. Often, I guess, that means I cut corners that make those with higher standards and T-squares cringe. I think there's a place in the DIY pantheon for both approaches, and I look forward to hearing more details about what you're doing with the image processing software. Maybe there's something there that I need and don't even know it yet.

[daniel_reetz]

I think there's a place in the DIY pantheon for both approaches

Yep, that's my feeling and it's been borne out here over and over again (in a good way!). For myself personally, if I'm going to scan, I like to do it really well, but I also have a "low standard" way, where I just point a camera in the general direction of the book when it's not dark. As for software, ST is the greatest, but I often use PS or just look at the photograph directly.

There's value in optimizing and trying new approaches and there's value in doing things fast, cheap, and dirty. Pick your point on the spectrum and run! (or, uhh, walk!).

[univurshul]

It's not clear to me how you go from "all the images are sorted in your image editing software" to "all the images are sorted in Scan Tailor." As far as I know, Scan Tailor is only sorting by name, not by capture time.

The image editor (or Lightroom in this case) treats every image in the book. Sometimes I do nothing more than sort the huge load of RAWs I just imported, according to 'capture time', then export the books appropriately with a new name sequence as TIFF files (because Scan Tailor doesn't except RAWs). So, there's a quick and dirty approach within an image editor and then there's the full treatment. But even a full treatment on a 1,000 page book only takes a few minutes to apply presets, maybe sharpen, slide a graduated filter over ultra glossy edges, exposure correction, flatten the images with lens distortion correction, etc.

The 'Clocking' discussion is designed for a decent image editor; nothing egregious though. I built this thread with it's intent to be a compliment to the Lightroom discussion, which is being written as we speak.

We know we can dump a bunch of images into scan Tailor and the results are really great. But if we test the buffering of applications that directly address undesired variables in photographs--which is what we're doing thousands of times with scanners--we open up a new realm of digitizing media. We have now have access to quality without begging nor paying outrageous prices for it. When I first started scanning, If I saw it was possible to achieve professional results at base prices with my own device, I would have paid/donated for this research and testing. This stuff is priceless.

I recently had this premonition what it would be like out there without DIY. Then I woke up from that nightmare.

I'm the guy who can't hear the difference between MP3 and vinyl.

Someone say music? This is what got me into scanning. The challenge of perfecting transcription; I don't want to hand down half-hearted reproductions to the next generations. I've digitized over 10,000 LPs and counting. I can't tell you how many records I'd like to re-process if I had the time to do so. I thought for a long time there was one "good-enough" method to get them to CD. I don't even touch CDs anymore let alone noise-crushing compression rates...but this does bring up an interesting philosophy Dan pointed out: we love quality, but we also like fitting a ton of music onto our iPods...Books are on this divergence too! Nice! I like it!

[daniel_reetz]

These different approaches are great, but In my mind/world/opinion, the best thing that can happen for bulk processing is that someone would produce a command-line version of Scan Tailor. Then we could create a little front-end piece of software for each kind of scanning (for example, one for clockers and one for people using filename incrementing, or people using one camera VS two). This front-end software would handle the flow of images in and out.

There's an analogy there to the Hugin project, which is basically a bunch of open source image processing tools strung together very carefully and with much thought. The result is that if you don't like different parts of the program, or only use part of the workflow, there's room for that. Also, any one part can be improved without necessarily rebuilding the whole. A flexible, sensible toolchain. Works on all platforms.

I'd like to cut closed-source software out of the loop as much as possible, because so much of what we are doing is so basic and can be done with existing open source software. The complete solution isn't here just yet, but over time, we'll surely make it happen. Many members here are working on different parts of this, like Strider's utilities, or Dtic's stuff, Spam's scripts, etc. Thinking about where we were just a year ago just blows my mind.

Well, that's sufficiently OT to the point where I should probably can it.

[univurshul]

I totally agree, Dan.

If anything, our exploratory missions into commercial or closed software are intended to educate us how to make open source improved and evolved.

Think about it: companies like Abobe and their architects have been working with photos and cameras longer than many of us have been alive. Despite how galvanized their software is for us, we have a lot to learn from them.

...Albeit my involvement with software tests and postulates ideas like a kid who refuses to stop picking at a scab.

[clemd973]

OK. I've tried this method of "clocking," and it worked GREAT!!! A definite time saver if you ask me. I suppose, however, you have to have software that will sort images by capture time. I recently began using Light Room 3, and I am thoroughly satisfied with its performance. I'm using a Mac and began post-processing using iPhoto > Scan Tailor > Tiff to PDF converter. The workflow was pretty smooth, but now I've bitten the bullet and have gone the Adobe route. I'm all for open source programs, but I happened across Acrobat and LR3 for a good price and jumped in feet first. For those of you who can/desire that route, it's worth the investment. (simply the opinion of one scanner among many). If anyone IS interested in this route, I'll be posting a LR3 tutorial going through the basics of the workflow. If you happen to be using iPhoto for "pre-processing" of your images, here is a great tutorial or putting your images in sequential page order. But if you're using LR3, this clocking tutorial is worth the read.

There are many methods of pre- and post-processing your images; you just have to find the method that fits you. This was a good fit for me considering the hardware and software I have.