1. pix2pix项目

1.1 项目介绍

通过轮廓生成对应形状实体：

https://github.com/phillipi/pix2pix

https://phillipi.github.io/pix2pix/

https://github.com/affinelayer/pix2pix-tensorflow

https://affinelayer.com/pixsrv/

https://arxiv.org/abs/1611.07004

2. face2face-demo项目

2.1 项目介绍

通过上面项目的启发，应用到视频上，则可以实现换脸技术：

https://github.com/datitran/face2face-demo

1.2 安装

1.2.1 环境介绍:

Anaconda / Python 3.5

TensorFlow 1.2

OpenCV 3.0

Dlib 19.4

整体打包到docker镜像中：

1.2.2 克隆项目：

git clone https://github.com/datitran/face2face-demo.git

1.2.3 激活环境：

mac:

conda env create -f environment.yml

linux:

windows:

1.2.4 生成训练数据:

下载脸部标签数据集：

http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2

下载视频文件：

https://dl.dropboxusercontent.com/s/2g04onlkmkq9c69/angela_merkel_speech.mp4

生成数据：

python generate_train_data.py --file angela_merkel_speech.mp4 --num 400 --landmark-model shape_predictor_68_face_landmarks.dat

训练后会生成两个文件夹original和landmarks

https://dl.dropboxusercontent.com/s/pfm8b0yogmum63w/dataset.zip

1.2.5 训练模型：

git clone https://github.com/affinelayer/pix2pix-tensorflow.git

mv face2face-demo/landmarks face2face-demo/original pix2pix-tensorflow/photos

cd pix2pix-tensorflow/

# Resize original images

python tools/process.py \

--input_dir photos/original \

--operation resize \

--output_dir photos/original_resized

# Resize landmark images

python tools/process.py \

--input_dir photos/landmarks \

--operation resize \

--output_dir photos/landmarks_resized

# Combine both resized original and landmark images

python tools/process.py \

--input_dir photos/landmarks_resized \

--b_dir photos/original_resized \

--operation combine \

--output_dir photos/combined

# Split into train/val set

python tools/split.py \

--dir photos/combined

# Train the model on the data

python pix2pix.py \

--mode train \

--output_dir face2face-model \

--max_epochs 200 \

--input_dir photos/combined/train \

--which_direction AtoB

1.2.6 导出模型

python reduce_model.py --model-input face2face-model --model-output face2face-reduced-model

python freeze_model.py --model-folder face2face-reduced-model

1.3 run demo

python run_webcam.py --source 0 --show 0 --landmark-model shape_predictor_68_face_landmarks.dat --tf-model face2face-reduced-model/frozen_model.pb

1.3 项目研究