A new algorithm evaluates the composition—the arrangement of visual elements—of digital photographs.
It offers feedback about the perceived composition of the photograph and provides examples of similarly composed pictures of high aesthetic value, says James Wang, professor of information sciences and technology at Penn State.
“If you think about aesthetics, everything is about composition,” says Wang. “You can look into colors, or textures, or shapes, but, if you boil it down, you eventually have to consider all of these elements as part of composition.”
Training a machine to become an art critic is not easy, according to the researchers. A machine must be trained with examples of highly regarded photographs in order to recognize good compositional elements, says Jia Li, professor of statistics, who worked with Wang.
Art school for algorithms
Li adds that the software the team developed relies on psychological theories of human vision as well as on the opinions of people to help classify images. For example, the researchers considered work done in Gestalt psychology, which focuses on how people perceive shape and form, in the development of the software.
A group of graduate students helped develop the algorithm by manually labeling hundreds of images from a collection of pictures on the website photo.net. The composition of each picture is categorized into horizontal, vertical, centered, diagonal, and textured types. These labels then serve as the training dataset that teaches the algorithm how to classify photographs.
“Aesthetics is, of course, subjective,” says Li. “What one person might find pleasing may not be pleasing to another, but there are elements that many good photos have in common.”
Pixel by pixel
The software performs a pixel-by-pixel analysis to extract features from a photograph and then uses statistical analysis to classify and compare it to highly aesthetic pictures.
Often, photos may have a blend of compositional elements. The software can detect and indicate cases when there may be several compositional options for the photographer.
“For example, a group of people could look at a photograph of, let’s say, a canyon, and some would see it as a vertical composition, some people could see it as a diagonal composition, and some might even say it’s a horizontal picture,” Wang says. “So, sometimes, composition is not well defined.”
Wang says he doubts the application will replace photographers any time soon. He hopes, instead, that it creates better photographers.
“Our goal from the beginning was to help photographers,” says Wang. “If you are an amateur photographer then, potentially, a computer can analyze your photograph’s composition and help you improve it.”
[related]
The software can be installed on a server that can be accessed by a mobile phone. The researchers expect that, as smart phones become more powerful, the application will be able to rest on the phone. Eventually, as smart phones gain more memory and power, the software would not need to be stored on a server, but could be added to the phone itself.
“It’s not even necessary to store very high resolution images for the system to work, so putting the software on a mobile phone is possible,” says Li. “The system was developed so that it doesn’t consume a lot of memory.”
The National Science Foundation partially supported this work. Wang and colleagues recently received a patent for the system.
Source: Penn State
