Code for automatically putting a mask on a face in photographs. The code matches features between the mask and the face in the photo and warps the mask and merges it onto the face. Additionally, the code adds ears or a hat onto the photo.
face-masker
Code (Matlab) for automatically putting a mask on a face in photographs. The code matches features between the mask and the face in the photo and warps the mask and merges it onto the face. Additionally, the code adds ears or a hat onto the photo.
In order to detect matching feature points in the faces and masks, a separate algorithm should be used (this repository does not include it). I have tested two feature detectors that worked well:
Masks should be stored as the basis image, feature points and blending mask (binary or gray scale weights). The masks can also contain ears/hat/wig. The feature points are detected by the same algorithm that you plan on using for photos, and can be shifted manually. The blending masks are best when done manually with simple programs such as paint.
Read the report and look at the poster to learn more about this project.
To illustrate the simplest use case let us assume that we start with a photo with a single face in it.
pychubbyimplements a classLandmarkFacewhich stores all relevant data that enable face warping. Namely it is the image itself and 68 landmark points. To instantiate aLandmarkFaceone needs to use a utility class methodestimate.
Note that it might be necessary to upsample the image before the estimation. For convenience theestimatemethod has an optional parametern_upsamples.
Once the landmark points are estimated we can move on with performing actions on the face. Let’s try to make the person smile:
frompychubby.actionsimportSmilea=Smile(scale=0.2)new_lf,df=a.perform(lf)# lf defined abovenew_lf.plot(show_landmarks=False)
There are 2 important things to note. Firstly thenew_lfnow contains both the warped version of the original image as well as the transformed landmark points. Secondly, theperformmethod also returns adfwhich is an instance ofpychubby.base.DisplacementFieldand represents the pixel by pixel transformation between the old and the new (smiling) image.
To see all currently available actions go toGallery.
To create an animation of the action we can use thevisualizationmodule.
frompychubby.visualizationimportcreate_animationani=create_animation(df,img)# the displacement field and the original image
If you get an errorFileNotFoundError: [Errno 2] No such file or directory: 'cmake': 'cmake', you need to make surecmakeis installed. If you're on OSX you can install this via Homebrew with:
pychubbyalso comes with a CLI that exposes some of its functionality. You can list the commands withpc --help:
Usage: pc [OPTIONS] COMMAND [ARGS]... Automated face warping tool. Options: --help Show this message and exit. Commands: list List available actions. perform Take an action.
To perform an action (Smile in the example below) and plot the result on the screen
We all know the story. You ask a random person on a street to take a photo of you and your friends. After a few moments you happily thank them and go on with your life. After some time you finally sit down for a cup of coffee and check your photos. “My god, why is none of us smiling?!”.
First of all, do not panic. It is your lucky day. If you know how to pip install things there might be hope.
Pychubby is an automated face warping tool. Its main goal is to serve as a specialized augmentation interface for deep learning face related tasks. But it might as well be used as a silly face warping tool (see the introduction).
You might wonder why even bother when one can do similar things with Photoshop and other software. The answer is simple -automation. You do not have to locate any landmarks, move them around and then repeat the procedure on each face in every photo.
Popular image augmentation packages likeimgaugare general purpose (any kind of image) and do not provide many options when it comes to geometric transformations.pychubbyis specialized on human faces and allows for creation of geometric transformations that are:
Local (on the face)
Smooth (no artifacts)
Realistic (to an extent)
In other words one does not augment the whole image but just the faces. The augmentations are realistic and have no artifacts.
Building blocks
The logic ofpychubbycan be summarized in three blocks
Landmark Detection
Given a photo, a pretrained landmark detection model predicts 68 landmarks on each face.
Reference Space Mapping
The landmarks are mapped into a so called reference space. This mapping corrects for possible rotations, translations and scaling in the input image.
Manual Action Definition
Majority ofpychubbyactions are defined in the reference space and therefore should yield consistent warpings across different faces. Go toGalleryto see the predefined ones or feel free to define new ones.
Want to know more?
If you are interested in givingpychubbya try or just want to learn more see below a few useful links: