Computer graphics research team to present new tool for sketching faces
IMAGE: Faceshop consists of a web-based user interface (left) that enables a user to specify a region to be edited. Moreover, the user can draw strokes and color scribbles to guide... view more
Credit: Tiziano Protenier
CHICAGO--Across popular social media platforms, users are posting countless images every day. On Instagram alone, there are more than 40 billion uploaded photos--a figure that's skyrocketing by 95 million daily. This presents a clear need for intuitive yet robust photo-editing tools that allow the average user to perform advanced editing functions.
And while there is a definite need for interactive image editing with respect to social media, improved editing tools and systems also remain an important aspect of computer graphics and computer vision. However, there is a lack of tools that feature more complex editing functions for inexperienced users, such as changing the facial expression in a photo.
A research team, led by computer scientists from the University of Bern-Switzerland and University of Maryland-College Park, have devised a sketch-based editing framework that enables a user to edit their photos by "sketching" a few strokes on top of them. Their system, called FaceShop, also offers a copy-paste function, which allows users to edit any part of a photo by copying-and-pasting the portion to be edited from another (better) photo, eliminating the need to hand-draw or sketch anything at all.
The team's approach is built on machine-learning techniques, which, in the end, give users more control over their desired edits in real-time and produce more realistic results.
"Most other approaches rely on more traditional, handcrafted techniques, which impose some limitations. For instance, these systems are either [by design] restricted to limited sets of predefined editing operations, or they are very flexible but hard to use and require experienced users to spend a considerable amount of time to perform rather basic edits," says Tiziano Protenier, lead author of the work and PhD candidate at the University of Bern. "In contrast, our system is very flexible and allows untrained users to perform complex edits within minutes using an intuitive interface."
Protenier developed the new system with collaborators Qiyang Hu, Attila Szabó, Siavash Arjomand Bigdeli and Paolo Favaro from University of Bern, and Matthias Zwicker from University of Maryland. The team will present their work at SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia. This annual gathering showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.
In their paper, "FaceShop: Deep Sketch-based Face Image Editing," the researchers demonstrate their new system via several examples. In one, a woman's nose is slightly manipulated, and a strand of hair is edited so that it is pushed away from her face, removing a dark shadow that had appeared on just one side of her face in the original photo. Another photo shows how a user is able to enhance a woman's eye make-up and bring out the color of her eyes. The examples highlighted in the paper showcase how the novel system works, intuitively, and produces high-quality, realistic results.
The team's method is based on generative adversarial neural networks (GANs), a form of artificial intelligence (AI) that, in recent years, has attracted a lot of research interest for its ability to generate realistic looking images. These GANs consist of two AIs that fight against each other. The first component tries to distinguish the generated images from genuine images, whereas the second component tries to produce images that fool the other AI. During training, the two components learn from each other, eventually resulting in a system that autonomously learned to produce realistic looking images, without any human judgement in the loop.
Unlike other AI-driven image editing methods, once the system is trained, users are more in control of the edits to their photos. "Our system features an optimum degree of control, which makes it very interesting from an application point of view," notes Portenier. "Other techniques that try to incorporate user control are either limited in the scope of possible edits or have been shown to work well on the data that was used to train the systems, but fail to produce convincing results with real user input. We are presenting a technique that mitigates this issue, resulting in a system that works surprisingly well in practice."
In future work, the researchers intend to explore additional user interaction tools to add to their framework, and consider how to leverage AI for sketch-based editing of videos.
###
For the full paper and video, visit the team's project page.
About ACM, ACM SIGGRAPH, and SIGGRAPH 2018
ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. SIGGRAPH is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2018, marking the 45th annual conference hosted by ACM SIGGRAPH, will take place from 12-16 August at the Vancouver Convention Centre in Vancouver, B.C.
To register for SIGGRAPH 2018 and hear from the authors themselves, visit s2018.siggraph.org/attend/register.
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.