# Image Processing **URL:** \ **Category:** Factual Questions\ **Created:** [June 11, 2004, 3:47pm UTC](https://boards.straightdope.com/t/image-processing/249826 "2004-06-11T15:47:36Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![\_zz](https://avatars.discourse-cdn.com/v4/letter/_/47e85d/32.png) [@\_zz](https://boards.straightdope.com/u/_zz)\ **Post date:** [June 11, 2004, 3:47pm UTC](https://boards.straightdope.com/t/image-processing/249826/1 "2004-06-11T15:47:36Z")
Currently, I have been assigned to work with some very large image files (5k pixels by 200k+ pixels at 200dpi) usually stored in .tif format. Large scrolls of instrument readouts. Now, that being said, Photoshop can’t handle anything wider than 30k pixels. I have found a few programs out there with similar functionality that do. Problem I am having is that we would like to segment the images up into fragments approximately 25k to 30k pixels wide, and I’d prefer an automatic process to do it (although, I would certainly be happy to be able to open the image and be able to just type “30000” for a width crop.) Anyone know of any software ou there that can help me divide up these images, that doesnt have a limit on the size of the image it can work with? ---
**Author:** ![rjk](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@rjk](https://boards.straightdope.com/u/rjk)\ **Post date:** [June 11, 2004, 4:22pm UTC](https://boards.straightdope.com/t/image-processing/249826/2 "2004-06-11T16:22:43Z")
That could be a tough one! You’ve got about a billion pixels there, and most image software likes to work entirely in memory. Even if the images are monochrome at one bit per pixel, it comes to 128Mb uncompressed. The usual recommendations are [Irfanview](http://www.irfanview.com/) and [the Gimp](http://www.gimp.org/) (both free), but I don’t know if either will handle that size. Both do cropping, but I don’t know how much automation you can do. If not, you could look at Photoshop (NOT free!). If nothing else works, you might have to get a custom program to split up the files. ---
**Author:** ![\_zz](https://avatars.discourse-cdn.com/v4/letter/_/47e85d/32.png) [@\_zz](https://boards.straightdope.com/u/_zz)\ **Post date:** [June 11, 2004, 5:02pm UTC](https://boards.straightdope.com/t/image-processing/249826/3 "2004-06-11T17:02:25Z")
> [@rjk](#): > > That could be a tough one! You’ve got about a billion pixels there, and most image software likes to work entirely in memory. Even if the images are monochrome at one bit per pixel, it comes to 128Mb uncompressed. > > The usual recommendations are [Irfanview](http://www.irfanview.com/) and [the Gimp](http://www.gimp.org/) (both free), but I don’t know if either will handle that size. Both do cropping, but I don’t know how much automation you can do. If not, you could look at Photoshop (NOT free!). > > If nothing else works, you might have to get a custom program to split up the files. We have photshop here and it cant handle images that large. They are greyscale images, 8 bit. I will look into Irfanview as soon as I get some CDs to burn with. (Unfortunately the Windows box with the Wide-Format scanner is not on the network, so I am forced to download everything to mac and burn to transport.) As far as GIMP goes, I try not to do too much on UNIX if I can help it … simply because our UNIX admin doesnt much like installing things. So I’m left with Windows and Mac … OS 9.something. Unfortunatley, my OSX box has the functionality of a lima bean. As far as custom programs go, I am currently working on one for the same project, but as an intern my knowledge is limited in scope, and there seems to be little help forthcoming around here. My program currently runs on these divided images saved as PGMs … which are very very large files with little in the way of compression, but very easy to traverse in a program. But as some sort of scope … a PGM thats 25k pixels wide … at about … 600~800dpi … is around 150 MB. Figure a small scan being about 8 segments … and you’ve got more than your typical CD-R can handle. So for automatically segmenting the original image … if saved as a PGM I’m up to gigs of data that I need to move. Which just isn’t fun. Thanks for your help thus far! ~zz~ ---
**Author:** ![Earthling](https://avatars.discourse-cdn.com/v4/letter/e/90db22/32.png) [@Earthling](https://boards.straightdope.com/u/Earthling)\ **Post date:** [June 11, 2004, 5:10pm UTC](https://boards.straightdope.com/t/image-processing/249826/4 "2004-06-11T17:10:48Z")
It looks like custom sofware is needed to handle images that large. Your problems sound similar to what [this guy](http://www.tawbaware.com/maxlyons/gigapixel.htm) faced, and he had to write custom code. Maybe if you get in touch with him, he’ll be able to supply you with more details. ---
**Author:** ![crozell](https://avatars.discourse-cdn.com/v4/letter/c/3da27b/32.png) [@crozell](https://boards.straightdope.com/u/crozell)\ **Post date:** [June 11, 2004, 5:12pm UTC](https://boards.straightdope.com/t/image-processing/249826/5 "2004-06-11T17:12:30Z")
I worked at a lab once where we had to deal with very large data sets related to imaging (spectral sensing). That lab used a program called [IDL](http://www.rsinc.com/idl/index.asp) to do all of their image processing. I didn’t care for the program itself too much (mostly because I just wasn’t used to it probably). But, if I remember correctly, one of the main advantages of it is that it can handle very large pieces of data because it specifically doesn’t load the whole image into memory at once. I think it is not free (but may be cheap for students). I know NOTHING else about this program and it’s functionality (no warranty expressed or implied, your milage may vary, etc.), but I think it can probably at least load a large image and save a subimage (crop). Someone please correct me if I am wrong. ---
**Author:** ![Shoeless](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/shoeless/32/7099_2.png) [@Shoeless](https://boards.straightdope.com/u/Shoeless)\ **Post date:** [June 11, 2004, 5:19pm UTC](https://boards.straightdope.com/t/image-processing/249826/6 "2004-06-11T17:19:28Z")
Wow… if I’m doing my math right, you’ve got a TIFF image of something that’s about 2 ft wide by 85 ft long. :eek: (I’m curious what your scanner looks like!) Sorry I can’t help much here… I’ve got plenty of experience with TIFFs but generally in the 8.5x11 inch range. Where I work we use both the LeadTools and Pegasus toolkits to do our image manipulation, but (a) they’re both kinda pricey, and (b) I have no idea if they could handle something that big. Like **rjk** says, most software packages like to be able to work in memory. ---
**Author:** ![emarkp](https://avatars.discourse-cdn.com/v4/letter/e/3be4f8/32.png) [@emarkp](https://boards.straightdope.com/u/emarkp)\ **Post date:** [June 12, 2004, 12:11am UTC](https://boards.straightdope.com/t/image-processing/249826/7 "2004-06-12T00:11:35Z")
> [@~zz~](#): > > Currently, I have been assigned to work with some very large image files (5k pixels by 200k+ pixels at 200dpi) usually stored in .tif format. Large scrolls of instrument readouts. What do you mean “work with”? Do you mean interactive editing? Or do you just need to do some automated processing? ---
**Author:** ![\_zz](https://avatars.discourse-cdn.com/v4/letter/_/47e85d/32.png) [@\_zz](https://boards.straightdope.com/u/_zz)\ **Post date:** [June 15, 2004, 2:34pm UTC](https://boards.straightdope.com/t/image-processing/249826/8 "2004-06-15T14:34:02Z")
> [@emarkp](#): > > What do you mean “work with”? Do you mean interactive editing? Or do you just need to do some automated processing? We need to divide the images, because they are at least 85 feet long, so we can more easily process them. Picture these as very long graphs, but intermittently along them there are lines of text with important inormation about time, latitudes, longitudes, etc. I am currently designing a program to scan these images and automatically create a file with just these lines in it, so we can run OCR software over that. As you can imagine, or maybe not (but I’m not going to imagine for anyone else), there are a number of other things we need to do with these images, to extract data and then store them in a database. > [@Shoeless](#): > > Wow… if I’m doing my math right, you’ve got a TIFF image of something that’s about 2 ft wide by 85 ft long. (I’m curious what your scanner looks like!) Made by [these guys](http://contex.com/). Thanks all for your suggestions. I am reading into them right now. ~~zz~~ ---
**Author:** ![rjk](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@rjk](https://boards.straightdope.com/u/rjk)\ **Post date:** [June 15, 2004, 3:05pm UTC](https://boards.straightdope.com/t/image-processing/249826/9 "2004-06-15T15:05:40Z")
There is a Windows version of the Gimp (scroll down on the page I linked above), but the notes on the download page seem to imply that it wants to work in memory. It looks like you’ll either have to get the software **crozell** mentions, or write your own. If you go custom, a first look at the TIFF specification is available [here](http://www.ee.cooper.edu/courses/course_pages/past_courses/EE458/TIFF/). You may need to look at the full spec. Good luck! ---
**Author:** ![bughunter](https://avatars.discourse-cdn.com/v4/letter/b/aeb1de/32.png) [@bughunter](https://boards.straightdope.com/u/bughunter)\ **Post date:** [June 15, 2004, 6:26pm UTC](https://boards.straightdope.com/t/image-processing/249826/10 "2004-06-15T18:26:34Z")
As a remote sensing instrument developer, we frequently work with multi-frame TIFF images that are tens to hundreds of Gigabytes. (Yes, GIGs). There is only one application that we have found that will handle data in chunks that big, and **crozell** has already mentioned it. IDL, or interactive data language, is a high-level interpreted language that can be run in interactive mode, or in batch mode using scripts. It is ideal for the kinds of problems that **~zz~** describes, and there is a fairly broad user base. Unfortunately, an annual license costs a couple thousand dollars. It may be possible to get GIMP or even ImageJ to work with TIFFs as big as the OP’s, but you may need to trick out the settings and run on a machine with GBs of RAM. ImageJ is a rather nice program for manipulating and analyzing scientific data in image formats like TIFF, so it may be up to the task, and it’s FREE!! Linky-linky-thingees: - [IDL](http://www.rsinc.com/idl/) - [GIMP](http://www.gimp.org/) - [ImageJ](http://rsb.info.nih.gov/ij/) ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [June 15, 2004, 7:31pm UTC](https://boards.straightdope.com/t/image-processing/249826/11 "2004-06-15T19:31:39Z")
> [@](#): > > As far as custom programs go, I am currently working on one for the same project, but as an intern my knowledge is limited in scope, and there seems to be little help forthcoming around here. My program currently runs on these divided images saved as PGMs … which are very very large files with little in the way of compression, but very easy to traverse in a program. But as some sort of scope … a PGM thats 25k pixels wide … at about … 600~800dpi … is around 150 MB. Figure a small scan being about 8 segments … and you’ve got more than your typical CD-R can handle. So for automatically segmenting the original image … if saved as a PGM I’m up to gigs of data that I need to move. Which just isn’t fun. Let me see if I’m following this… You can work easily with the PGM files, but they’re big, so you can’t move them around easily. So you convert them to the much more compressed TIFF format. But now they’re harder to work with. Would it be possible to break them up into manageable chunks first, in the PGM format, before converting them? I’ve also worked with IDL in the past, and it’s certainly powerful enough for anything you might care to do. But it can have quite the learning curve, too: It’s really more a programming language, than an application itself. ---
**Author:** ![Danalan](https://avatars.discourse-cdn.com/v4/letter/d/76d3ee/32.png) [@Danalan](https://boards.straightdope.com/u/Danalan)\ **Post date:** [June 15, 2004, 7:52pm UTC](https://boards.straightdope.com/t/image-processing/249826/12 "2004-06-15T19:52:53Z")
[This](http://www.neuralog.com/products/neuraview/) software might present a good solution for you – it’s under $200, and supposedly handles large TIFF’s. Looks like it might work very well for your application. ---
**Author:** ![Ale](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@Ale](https://boards.straightdope.com/u/Ale)\ **Post date:** [June 15, 2004, 10:42pm UTC](https://boards.straightdope.com/t/image-processing/249826/13 "2004-06-15T22:42:16Z")
Corel PhotoPaint can crop images before opening them… my 2 cents. \*N.B: today at work I was bitching about a Photoshop file that was 1,2 x 3,5 meters, about 600 Mb all in all… \* :dubious: ---
**Author:** ![Popup](https://avatars.discourse-cdn.com/v4/letter/p/94ad74/32.png) [@Popup](https://boards.straightdope.com/u/Popup)\ **Post date:** [June 16, 2004, 7:31am UTC](https://boards.straightdope.com/t/image-processing/249826/14 "2004-06-16T07:31:02Z")
Heve you tried [ImageMagick](http://www.imagemagick.org)? It’s an open source image converter[sup]\*[/sup], which handles pretty much all known image formats, and can also do fairly advanced image manipulations. (It can be likened to Gimp, but without the graphical interface… (Although it does come with an optional graphical interface - but it’s not very good)) It can for example be used to crop images from the command line, as in your example. It uses various clever ways if the available memory isn’t enough. (It can use its own swap file for example.) It was developed for unix, but runs on most platforms. [sup]\*[/sup]Although, calling it an ‘image converter’ is like calling a Lamborghini Countach ‘a car’. ---
**Author:** ![\_zz](https://avatars.discourse-cdn.com/v4/letter/_/47e85d/32.png) [@\_zz](https://boards.straightdope.com/u/_zz)\ **Post date:** [June 16, 2004, 2:11pm UTC](https://boards.straightdope.com/t/image-processing/249826/15 "2004-06-16T14:11:56Z")
> [@Chronos](#): > > Let me see if I’m following this… You can work easily with the PGM files, but they’re big, so you can’t move them around easily. So you convert them to the much more compressed TIFF format. But now they’re harder to work with. Would it be possible to break them up into manageable chunks first, in the PGM format, before converting them? Thats exactly what I’m trying to do, but with most programs I can’t open the images to break them. TIFFs take up much less hard drive space, but its the physical amount of pixels that crash most image programs. Thanks everyone for the links. I’ll start checking these out. ~~zz~~