Lego Mindstorms scanner - A novel way of scanning magazines

Built a scanner? Started to build a scanner? Record your progress here. Doesn't need to be a whole scanner - triggers and other parts are fine. Commercial scanners are fine too.

Moderator: peterZ

JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

daniel_reetz wrote: 17 Nov 2021, 23:56 Another way to solve this problem is to build 2 copies ;)
Hmm, convince my wife to get me another Mindstorms set for Christmas...I like your thinking. Of course, the counter argument is that I am a 44-year-old man playing with Lego (and stealing bits I need from the kids' sets)...

With the Gen 1 I linked to above, I digitised about 30 magazines successfully before the speed of it started to get irritating - but you're right, a fully-automated unit doesn't need to be fast (just reliable).

My Gen 2 design prototype is working really well and is much simpler - I hope to get it finished and programmed over the weekend and then I'll post a video. If the design inspires other people to make something that is useful and more robust, then I'll be happy to have contributed. I have also been toying with an idea which would let it load magazines automatically from a stack, but my gut feeling is that it would hit the reliability too badly.
JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

Gen 2 design concept up and running - still some tweaking to do, but it's pretty reliable. I think I can get the speed up by about 20% on what it is currently, once I have it all tuned in.

The platter holding the lifting arm moves on a rack-and-pinon arrangement, pulled by a pair of GT2 timing belts on 3D printed Lego-compatible drive wheels. The Czur is triggered each cycle using a microswitch on the built-in foot pedal connector.

I've had variable results using the Czur software - sometimes excellent, but often with weird skew and curls - I'm going to play with capturing the image flat as an A3 double page, then post processing it - but in general I think I have the bones of a good working concept here.

https://youtu.be/mNaHhE-wwI0
cday
Posts: 447
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by cday »

You are testing using a book with thicker paper than the magazines to be archived?

The pages are lifted by suction?

And maybe a possibility of flattening the page to be imaged using downward airflow?

The image distortion issue should certainly be reduced if the pages can be flattened.
JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

That's one of the magazines to be archived - there's a real variability in the paper thickness between the '90s and the 2000s - this design actually struggles a bit with post-2000s magazines because the paper is more flimsy and occasionally folds as it's being flipped. I could change the angle it's raised to by the suction fan to combat that, but I don't think I will for now.

Honestly, I don't think downward airflow would flatten it much - what works better is using that small wheel to put a little crease in the vertical centre of the page - the Czur does a good job of flattening pages when I do it manually - I <think> what's happening is that it's picking up artifacts from the Lego frame which the algorithm thinks is part of the page so it's doing some weird transformations to cope - I sprayed it matt black which has helped a little, but it's variable. I'm going to try scanning just a defined area as a flat image, then working out some post processing in Linux.
cday
Posts: 447
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by cday »

JasonStonier wrote: 22 Nov 2021, 05:28 I don't think downward airflow would flatten it much - what works better is using that small wheel to put a little crease in the vertical centre of the page
Yes, that looked to be working well, but wasn't shown in the video of your second design.

...the Czur does a good job of flattening pages when I do it manually - I <think> what's happening is that it's picking up artifacts from the Lego frame which the algorithm thinks is part of the page so it's doing some weird transformations to cope - I sprayed it matt black which has helped a little, but it's variable. I'm going to try scanning just a defined area as a flat image, then working out some post processing in Linux.
I switched to Linux, currently MInt 20.2 Cinnamon, four years ago for most of my needs, mainly to avoid Windows 10, although I still have Windows 7 available for a small number of applications I can't run in in Linux. Some years back I unexpectedly got into using Windows batch files for image processing using XnView.com's NConvert, but I haven't gotten into scripting in Linux yet, interesting in principle but little need and so not a priority.
cday
Posts: 447
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by cday »

JasonStonier wrote: 22 Nov 2021, 05:28.. the Czur does a good job of flattening pages when I do it manually - I <think> what's happening is that it's picking up artifacts from the Lego frame which the algorithm thinks is part of the page so it's doing some weird transformations to cope - I sprayed it matt black which has helped a little, but it's variable. I'm going to try scanning just a defined area as a flat image, then working out some post processing in Linux.
Two more ideas at the risk of suggesting options that are already obvious to you:

Presumably the parts of the mechanism that may be upsetting the Czur image processing are entirely outside the magazine page area, might it therefore be possible to place some sort of mask with a cutout between the camera and the page? Maybe close to the Czur and supported by it, or maybe more practically a little above the mechanism provided it doesn't interfere with it? Maybe utilising the matt black thin card available from stationers?

Or, maybe crop the page images before applying corrections to them, if the Czur doesn't support output of 'uncorrected' images reverting to using a camera? Images could probably be batch cropped, or if necessary I think one software I have supports fast cropping of images while allowing adjustment of the crop area. There are well-respected utilities that have been used in book scanning that automatically correct for certain types of image distortion, although as far as possible flattening the page before it is imaged would be desirable.
JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

cday wrote: 22 Nov 2021, 11:30 Or, maybe crop the page images before applying corrections to them...
I think you're right, and I also arrived at that conclusion while not sleeping last night - I'm now capturing the images completely raw without any processing or cropping from the Czur, then using Nconvert to batch crop and split into A4 pages using a .bat script in Windows. I'll have a look at the Windows version of Imagemajik for assembling the PDF, and also I'll trawl the knowledge base here for other processing tools.

I'm just stress testing the flipping mechanism at the moment - discovered that every so often it folds a page, and I need to see what I can do to avoid that.
cday
Posts: 447
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by cday »

JasonStonier wrote: 23 Nov 2021, 11:39 I'm now capturing the images completely raw without any processing or cropping from the Czur, then using NConvert to batch crop and split into A4 pages using a .bat script in Windows.

I'll have a look at the Windows version of Imagemajik for assembling the PDF, and also I'll trawl the knowledge base here for other processing tools.
NConvert also supports creating multipage PDF files with alternative compression options... :D

Note that NConvert requires Ghostscript to be installed to open (rasterise) PDF files at the specified DPI, but it isn't required to create PDF files, it is also cross-platform so you should be able to use in in Linux if you would prefer that.

And while NConvert supports most standard image processing options, ImageMagick does support many more-advanced processing options if you need them, although I only ever had a very brief look at it.
JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

cday wrote: 23 Nov 2021, 13:51 NConvert also supports creating multipage PDF files with alternative compression options... :D
Does it indeed. I must have missed that - I suspect, at the age of 44, my GoogleFu is beginning to fail. Should have asked my teenager.
JasonStonier
Posts: 19
Joined: 16 Nov 2021, 06:29
E-book readers owned: Kindle Paperwhite
Number of books owned: 2000
Country: UK

Re: Lego Mindstorms scanner - A novel way of scanning magazines

Post by JasonStonier »

Well, here we go - I think I cracked it. This design is close to 100% reliable - in 20 or so magazine scans it's never missed a page, and rarely scans a page twice.

https://www.youtube.com/watch?v=UfI6VLClOUA

I had to add so many mechanisms to make it reliable that it's SLOW - about 20 minutes per magazine, but it works brilliantly.

Could easily be adapted to scan newspapers - or anything which will naturally lie fairly flat and has thin paper.
Post Reply