Blog: CEO’s Corner

agsdix-fas fa-home

Blog: Home

agsdix-fas fa-pen-fancy

Blog: CEO's Corner

agsdix-fas fa-code

Blog: Tech Talk

Blog: Product Releases

agsdix-fas fa-chalkboard-teacher

Blog: Viewing

Blog: Conversion

arrow left circle icon Blog: CEO’s Corner

Color Document Viewing and Processing- It’s all about the Memory

by | Aug 8, 2011

With so many companies processing color documents today in a client/server configuration, the burden of conversion or rendering documents has moved from the client to the server.  In many ways, this is a good thing.  It allows for greater security, the easier ability to connect to repositories, greater reliability because there are fewer concerns if a client station fails (automatic backup on the server with greater redundancy) and the ability to spread the workload world-wide through high speed networks.

Server memory versus client memory becomes a concern however, as most client stations nowadays have at least 2 GB – and Windows 7 for the client likes at least 4 GB.  Server memory requirement awareness doesn’t seem to have progressed as quickly.  This is partially because many servers don’t really do much processing; they’re more often occupied in routing data and files down to the client or to another server.  Sometimes one server will direct an auxiliary server to handle some heavy processing burden to distribute the load.  That allows the server handling client requests to be relatively unburdened and respond quite quickly.

However if the server is used for something more involved, like document or image rendering, the server load can increase quite dramatically.   For example, in an insurance claim or loan processing application, typically the document package can be 25-50 scanned pages.  If the pages are black and white, when rendered for viewing or conversion, each page requires between 60 KB (if your document processing system is memory efficient) and 1 MB. (Another consideration is DPI, which comes into play here. For example, an 8.5×11 B/W page at 300 dpi is just over 1MB; at 200 DPI it is less than half a MB).

If you have 20 agents working on that server, the load shouldn’t be too bad – between say 60KB x 20 = 1.2 MB to maybe 1 MB x 20 = 20 MB. However if your system is set up to keep 25 pages per agent in memory simultaneously, then your memory requirement goes up by 25x so it could be as high as 500 MB. On a server that only has 2 GB that starts to impact its free memory.

But there’s more. What if your client sent in a color picture of their damaged house or car and you need to keep the detail to assess the claim? Well, black and white images are 1 bit per pixel; color JPEG images are 24 bits per pixel. That’s a pretty simple multiplication of 24x the memory needed for black and white.  So now your server might need 12 GB of memory before it goes virtual.  Note that this isn’t the fault of the software – this is how things are.  Color kills performance because it eats up 24x the memory of black and white. It also eats up bandwidth meaning your documents will transmit much more slowly. Getting a faster server or faster software will only help a little – you’re still dealing with the fundamental issues of color documents needing lots and lots of memory.

Surprisingly, a lot of our customers have servers that run with 2-4 GB.  That’s fine for normal file handling but impossible for the workload described above.  One of Snowbound’s development environment servers currently runs 24 GB and it’s servicing only a few people.

What should you do?  Simple solution – tell IT you need more RAM.  Don’t investigate; don’t question it, just buy the RAM.  The most important thing you can do is support your users – you never want to see them idle.

Another important solution is to work with color documents sparingly and only scan true color documents in color. If you have a 30-page stack of documents and only three of the pages are color, it is vitally important to use a scanning method that distinguishes color from black and white.  The difference in memory usage and performance could be huge – reducing the load from 720 MB (24 MB x 30) to 100 MB (24 MB x3 + 27 x 1 MB) when it comes to the server having to decompress and manipulate each page.  Almost every modern scanning station has a means of recognizing color images vs. black and white.

 

From Snowbound Software’s RasterMaster Manual:

Table 2-2: Memory Requirements Based on Image Size

Image Size Required Memory
24-bit per pixel, 640 x 480 image 640 * 480 * (24/8) = 921,600 bytes
1-bit per pixel, 8.5″ x 11″ image, at 300 dpi (2550 pixels by 3300 pixels) 2550 * 3300 * (1/8) = 1,051,875 bytes
24-bit per pixel, 8.5″ x 11″ image, at 300 dpi (2550 pixels by 3300 pixels) 2550 * 3300 * (24/8) = 25,245,000 bytes (25 megabytes)