Resizing images for the web has always been a challenge. On one hand, most layouts need a fixed image size in order to look good. On the other hand, most images don’t fit that correct ratio. Designers often need to crop and manipulate photos in order to get them to assimilate to the target area.
With the popularity of tables and mobile devices, responsive design has thrown yet another monkey wrench into the problem. Websites now need to dynamically scale their content to the browser width and height, and change their layouts based on the device resolution and aspect ratio.
Scientists at Mitsubishi have created a novel solution to this age-old problem. Instead of simply stretching or cropping the images, the computer can shrink repetitive and redundant content, and keep only what our eyes and brains find interesting. This technique has been published in their ground-breaking paper, “Seam Carving for Content-Aware Image Resizing”.
Below is an image, resized using both standard scaling and seam carving. Notice how the seam carving algorithm retains most of the important information, while the scaling algorithm distorts the original image significantly.
The basic premise of the seam-carving algorithm is each pixel, or dot, that makes up an image has an energy level depending on its relationship to the surrounding pixels. Pixels that change significantly in color from their neighbors have high energy, while those that don’t change much are low energy. The program then runs through the image and removes as much low-energy content as necessary to achieve the target image size.
In order to make clean removals, the seam-carving algorithm doesn’t just remove one pixel at a time, it removes an entire line of pixels. Below is an example of the “Important Data” that the seam-finding algorithm decides not to remove, as well as a red-line showing an example seam that is removed from the image.
The most exciting applications of seam carving are in the world of human-computer interfaces. Now that computers can understand what our eyes find important, devices are much more capable of presenting us with relevant information. This is especially applicable to mobile phones, where the screen real-estate is quite small, and websites often need to be resized on-the-fly for mobile compatibility.
With all its potential, there is a weakness to seam carving: the algorithm is processor-intensive, and as a result has not yet been standardized into a major browser. Despite that, commercial implementations do exist in both closed-source programs such as Photoshop, as well as in open-source software such as GIMP and ImageMagick. As computers and mobile phones become more powerful, it is only a matter of time before seam-carving becomes mainstream and changes the way we browse the web.
Written by Andrew Palczewski
About the Author
Andrew Palczewski is CEO of apHarmony, a Chicago software development company. He holds a Master's degree in Computer Engineering from the University of Illinois at Urbana-Champaign and has over ten years' experience in managing development of software projects.
Google+
