https://github.com/orlyliba/face-masker

Code for automatically putting a mask on a face in photographs. The code matches features between the mask and the face in the photo and warps the mask and merges it onto the face. Additionally, the code adds ears or a hat onto the photo.

face-masker

Code (Matlab) for automatically putting a mask on a face in photographs. The code matches features between the mask and the face in the photo and warps the mask and merges it onto the face. Additionally, the code adds ears or a hat onto the photo.

In order to detect matching feature points in the faces and masks, a separate algorithm should be used (this repository does not include it). I have tested two feature detectors that worked well:

1. Face++ (http://www.faceplusplus.com/), in which feature detection is done in the cloud
2. Clandmark (http://cmp.felk.cvut.cz/~uricamic/clandmark/), an open source landmarking library

Masks should be stored as the basis image, feature points and blending mask (binary or gray scale weights). The masks can also contain ears/hat/wig. The feature points are detected by the same algorithm that you plan on using for photos, and can be shifted manually. The blending masks are best when done manually with simple programs such as paint.

Read the report and look at the poster to learn more about this project.

[1] document: https://pychubby.readthedocs.io/en/latest/source/basic_example.html

[2] 소스코드: https://github.com/jankrepl/pychubby

[3] 블로그: https://jankrepl.github.io/pychubby/

----------------------------------------------------------------------------------------------------

[1]

Basic Example

To illustrate the simplest use case let us assume that we start with a photo with a single face in it.

pychubby implements a class LandmarkFace which stores all relevant data that enable face warping. Namely it is the image itself and 68 landmark points. To instantiate a LandmarkFace one needs to use a utility class method estimate.

import matplotlib.pyplot as plt from pychubby.detect import LandmarkFace img = plt.imread("path/to/the/image") lf = LandmarkFace.estimate(img) lf.plot()

Note that it might be necessary to upsample the image before the estimation. For convenience the estimate method has an optional parameter n_upsamples.

Once the landmark points are estimated we can move on with performing actions on the face. Let’s try to make the person smile:

from pychubby.actions import Smile a = Smile(scale=0.2) new_lf, df = a.perform(lf) # lf defined above new_lf.plot(show_landmarks=False)

There are 2 important things to note. Firstly the new_lf now contains both the warped version of the original image as well as the transformed landmark points. Secondly, the perform method also returns a df which is an instance of pychubby.base.DisplacementField and represents the pixel by pixel transformation between the old and the new (smiling) image.

To see all currently available actions go to Gallery.

To create an animation of the action we can use the visualization module.

from pychubby.visualization import create_animation ani = create_animation(df, img) # the displacement field and the original image

------------------------------------------------------------------------------------------------------

[2]

PyChubby

Tool for automated face warping

Installation

pip install pychubby

If you get an error FileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake', you need to make sure cmake is installed. If you're on OSX you can install this via Homebrew with:

brew install cmake

For other platforms please consult the Cmake documentation at https://cmake.org/install/

Description

For each face in an image define what actions are to be performed on it, pychubby will do the rest.

Documentation

https://pychubby.readthedocs.io

Minimal Example

import matplotlib.pyplot as plt from pychubby.actions import Chubbify, Multiple, Pipeline, Smile from pychubby.detect import LandmarkFace img_path = 'path/to/your/image' img = plt.imread(img_path) lf = LandmarkFace.estimate(img) a_per_face = Pipeline([Chubbify(), Smile()]) a_all = Multiple(a_per_face) new_lf, _ = a_all.perform(lf) new_lf.plot(show_landmarks=False, show_numbers=False)

CLI

pychubby also comes with a CLI that exposes some of its functionality. You can list the commands with pc --help:

Usage: pc [OPTIONS] COMMAND [ARGS]... Automated face warping tool. Options: --help Show this message and exit. Commands: list List available actions. perform Take an action.

To perform an action (Smile in the example below) and plot the result on the screen

pc perform Smile INPUT_IMG_PATH

or if you want to create a new image and save it

pc perform Smile INPUT_IMG_PATH OUTPUT_IMG_PATH

Development

git clone https://github.com/jankrepl/pychubby.git cd pychubby pip install -e .[dev]

------------------------------------------------------------------------------------------------------

[3]

PyChubby - Automated Face Warping

 2 minute read

Introduction

We all know the story. You ask a random person on a street to take a photo of you and your friends. After a few moments you happily thank them and go on with your life. After some time you finally sit down for a cup of coffee and check your photos. “My god, why is none of us smiling?!”.

First of all, do not panic. It is your lucky day. If you know how to pip install things there might be hope.

pip install pychubby

Once installed, just write something like this:

import matplotlib.pyplot as plt from pychubby.actions import Multiple, Smile from pychubby.detect import LandmarkFace img_path = 'path/to/img.jpg' img = plt.imread(img_path) lf = LandmarkFace.estimate(img) a = Multiple(Smile(0.15)) lf_new, _ = a.perform(lf) lf_new.plot(show_numbers=False)

What is PyChubby

Pychubby is an automated face warping tool. Its main goal is to serve as a specialized augmentation interface for deep learning face related tasks. But it might as well be used as a silly face warping tool (see the introduction).

You might wonder why even bother when one can do similar things with Photoshop and other software. The answer is simple - automation. You do not have to locate any landmarks, move them around and then repeat the procedure on each face in every photo.

Popular image augmentation packages like imgaug are general purpose (any kind of image) and do not provide many options when it comes to geometric transformations. pychubby is specialized on human faces and allows for creation of geometric transformations that are:

1. Local (on the face)
2. Smooth (no artifacts)
3. Realistic (to an extent)

In other words one does not augment the whole image but just the faces. The augmentations are realistic and have no artifacts.

Building blocks

The logic of pychubby can be summarized in three blocks

1. Landmark Detection
   • Given a photo, a pretrained landmark detection model predicts 68 landmarks on each face.
2. Reference Space Mapping
   • The landmarks are mapped into a so called reference space. This mapping corrects for possible rotations, translations and scaling in the input image.
3. Manual Action Definition
   • Majority of pychubby actions are defined in the reference space and therefore should yield consistent warpings across different faces. Go to Gallery to see the predefined ones or feel free to define new ones.

Want to know more?

If you are interested in giving pychubby a try or just want to learn more see below a few useful links:

• GitHub
• Documentation

All potential contributors are more than welcome and any feedback is highly appreciated.

------------------------------------------------------------------------------------------------------