AI Smart Crop

The Problem

Its not uncommon for our sites to generate thumbnail images as part of a list of news stories or a group of staff members. This is easy enough to achieve as Duocms has some built in image manipulation code which can serve up an image at any resolution you need.

News lists do this by grabbing the first image on each page, resizing and cropping it to the right size and displaying it next to the story. This works most of the time, but when it doesn’t it can look pretty bad. Cutting the head off the CEO is never a good look.

To get around this, for a long time we’ve had the ability to override the above behaviour by uploading a specific image into the pages settings. This can be deliberately prepared to ensure all the critical elements of the image are right in the centre, so cropping on the whole looks pretty good. However this process adds another step to page editing and takes a little more time. This felt like a job AI could do well.

Face Detection to the Rescue?

To address this problem, we initially tried incorporating face detection into the cropping process. By identifying the face in the image and adjusting the crop accordingly, we aimed to keep the face as centrally positioned as possible. Although this approach appeared to be a straightforward solution, it often proved ineffective. This is because face detection relies on clear and visible faces, which is not always the case, especially when individuals are facing away from the camera.

The image opposite was a good example of this. The face couldn’t be seen by the AI, so they’d respond with ‘no face here’ and we’d be forced to fall back to the default of cropping to the middle.

Pose Detection

Upon exploring different AI models, we stumbled upon pose detection—an AI technique reminiscent of the XBOX camera's capabilities. It analyses an individual's image to precisely determine the position of their limbs. As well as arms, legs etc it also has eye, nose, ear, chin and shoulder positions.

Taking these key points and extrapolating the top of their head based on the distance between eyes and chin, we were able to devise a box that roughly encompassed their head and shoulders.

Detail Detection

Pose detection seemed to solve the issue of cropping off the subjects head. But what about other subject matter? To help subtly improve cropping in general we’ve combined the pose detection with edge detection. So in the absence of people in an image we can still get a better crop by focusing more on details in an image. The car in the header image is a good example. The smart crop lifts the car up away from the edge of the image, giving a more pleasing crop.

Availability

Smart cropping is undergoing testing in DuoCMS8, and once fully tested it will be incorporated into dynamic lists. This feature is expected to assist in reducing the workload required to maintain a polished and modern appearance for your website.