Skip to content

Commit

Permalink
Reorganized the repository for easy installation
Browse files Browse the repository at this point in the history
  • Loading branch information
cg-yuval committed Nov 16, 2022
1 parent 242e11c commit 61746a4
Show file tree
Hide file tree
Showing 68 changed files with 83 additions and 34 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.idea
*.pyc
__pycache__
experiments/results
fsgan/experiments/results
weights
download_fsgan_models.py
17 changes: 12 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,27 @@ every pair of faces can produce a high quality face swap.
## Requirements
- High-end NVIDIA GPUs with at least 11GB of DRAM.
- Either Linux or Windows. We recommend Linux for better performance.
- CUDA Toolkit 10.1, CUDNN 7.5, and the latest NVIDIA driver.
- Python 3.6+ and PyTorch 1.4.0+.
- CUDA Toolkit 10.1+, CUDNN 7.5+, and the latest NVIDIA driver.

## Installation
- [Ubuntu installation guide](https://github.com/YuvalNirkin/fsgan/wiki/Ubuntu-Installation-Guide)
- [Windows installation guide](https://github.com/YuvalNirkin/fsgan/wiki/Windows-Installation-Guide)
```Bash
git clone https://github.com/YuvalNirkin/hyperseg
cd hyperseg
conda env create -f hyperseg_env.yml
conda activate hyperseg
pip install . # Alternatively add the root directory of the repository to PYTHONPATH.
```

For accessing FSGAN's pretrained models and auxiliary data, please fill out
[this form](https://docs.google.com/forms/d/e/1FAIpQLScyyNWoFvyaxxfyaPLnCIAxXgdxLEMwR9Sayjh3JpWseuYlOA/viewform?usp=sf_link).
We will then send you a link to FSGAN's shared directory and download script.
```Bash
python download_fsgan_models.py # From the repository root directory
```

## Inference
- [Face swapping guide](https://github.com/YuvalNirkin/fsgan/wiki/Face-Swapping-Inference)
- [Face swapping Google Colab](inference/face_swapping.ipynb)
- [Face swapping Google Colab](fsgan/inference/face_swapping.ipynb)
- [Paper models guide](https://github.com/YuvalNirkin/fsgan/wiki/Paper-Models-Inference)

## Training
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
batch_size = [24, 12]
workers = 32
pretrained = False
criterion_id = VGGLoss('../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_id = VGGLoss('../../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_gan = GANLoss(use_lsgan=True)
generator = MultiScaleResUNet(in_nc=101, out_nc=(3, 3), flat_layers=(2, 2, 2, 2), ngf=128)
discriminator = MultiscaleDiscriminator(use_sigmoid=True, num_D=2)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
batch_size = [32, 16]
workers = 32
pretrained = False
criterion_id = VGGLoss('../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_id = VGGLoss('../../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_gan = GANLoss(use_lsgan=True)
generator = MultiScaleResUNet(in_nc=101, out_nc=3, flat_layers=(2, 2, 2, 2), ngf=128)
discriminator = MultiscaleDiscriminator(use_sigmoid=True, num_D=2)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
batch_size = [48, 24]
workers = 32
pretrained = False
criterion_id = VGGLoss('../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_id = VGGLoss('../../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_gan = GANLoss(use_lsgan=True)
generator = MultiScaleResUNet(in_nc=101, out_nc=3, flat_layers=(2, 2, 2, 2), ngf=128)
discriminator = MultiscaleDiscriminator(use_sigmoid=True, num_D=2)
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
batch_size = [32, 16]
workers = 32
pretrained = False
criterion_id = VGGLoss('../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_id = VGGLoss('../../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_gan = GANLoss(use_lsgan=True)
generator = MultiScaleResUNet(in_nc=7, out_nc=3, flat_layers=(2, 2, 2, 2), ngf=128)
discriminator = MultiscaleDiscriminator(use_sigmoid=True, num_D=2)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
batch_size = [32, 16]
workers = 32
pretrained = False
criterion_id = VGGLoss('../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_id = VGGLoss('../../../weights/vggface2_vgg19_256_1_2_id.pth')
criterion_attr = VGGLoss('../../../weights/celeba_vgg19_256_2_0_28_attr.pth')
criterion_gan = GANLoss(use_lsgan=True)
generator = MultiScaleResUNet(in_nc=4, out_nc=3, flat_layers=(2, 2, 2, 2), ngf=128)
discriminator = MultiscaleDiscriminator(use_sigmoid=True, num_D=2)
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion inference/reenact.py → fsgan/inference/reenact.py
Original file line number Diff line number Diff line change
Expand Up @@ -312,7 +312,7 @@ def __call__(self, source_path, target_path, output_path=None, select_source='lo

class FaceReenactmentRenderer(VideoRenderer):
def __init__(self, display=False, verbose=0, output_crop=False, resolution=256, crop_scale=1.2,
encoder_codec='avc1', separate_process=False):
encoder_codec='mp4v', separate_process=False):
self._appearance_map = None
self._fig = None
self._figsize = (24, 16)
Expand Down
12 changes: 6 additions & 6 deletions inference/swap.py → fsgan/inference/swap.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,12 @@
parser.add_argument('-b', '--batch_size', default=8, type=int, metavar='N',
help='mini-batch size')
parser.add_argument('-rm', '--reenactment_model', metavar='PATH',
default='../weights/nfv_msrunet_256_1_2_reenactment_v2.1.pth', help='reenactment model')
parser.add_argument('-cm', '--completion_model', default='../weights/ijbc_msrunet_256_1_2_inpainting_v2.pth',
default='weights/nfv_msrunet_256_1_2_reenactment_v2.1.pth', help='reenactment model')
parser.add_argument('-cm', '--completion_model', default='weights/ijbc_msrunet_256_1_2_inpainting_v2.pth',
metavar='PATH', help='completion model')
parser.add_argument('-bm', '--blending_model', default='../weights/ijbc_msrunet_256_1_2_blending_v2.pth',
parser.add_argument('-bm', '--blending_model', default='weights/ijbc_msrunet_256_1_2_blending_v2.pth',
metavar='PATH', help='blending model')
parser.add_argument('-ci', '--criterion_id', default="vgg_loss.VGGLoss('../weights/vggface2_vgg19_256_1_2_id.pth')",
parser.add_argument('-ci', '--criterion_id', default="vgg_loss.VGGLoss('weights/vggface2_vgg19_256_1_2_id.pth')",
metavar='OBJ', help='id criterion object')
parser.add_argument('-mr', '--min_radius', default=2.0, type=float, metavar='F',
help='minimum distance between points in the appearance map')
Expand All @@ -72,7 +72,7 @@
help='number of finetune iterations')
finetune.add_argument('-fl', '--finetune_lr', default=1e-4, type=float, metavar='F',
help='finetune learning rate')
finetune.add_argument('-fb', '--finetune_batch_size', default=4, type=int, metavar='N',
finetune.add_argument('-fb', '--finetune_batch_size', default=3, type=int, metavar='N',
help='finetune batch size')
finetune.add_argument('-fw', '--finetune_workers', default=4, type=int, metavar='N',
help='finetune workers')
Expand Down Expand Up @@ -362,7 +362,7 @@ def __call__(self, source_path, target_path, output_path=None, select_source='lo

class FaceSwappingRenderer(VideoRenderer):
def __init__(self, display=False, verbose=0, output_crop=False, resolution=256, crop_scale=1.2,
encoder_codec='avc1', separate_process=False):
encoder_codec='mp4v', separate_process=False):
self._appearance_map = None
self._fig = None
self._figsize = (24, 16)
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
3 changes: 2 additions & 1 deletion models/simple_unet_02.py → fsgan/models/simple_unet_02.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,8 @@ def __init__(self, in_size, out_size, is_deconv):
self.up = nn.ConvTranspose2d(in_size, out_size, kernel_size=2, stride=2)
else:
self.up = nn.UpsamplingBilinear2d(scale_factor=2)
self.conv1d = nn.Conv1d(in_size, out_size, kernel_size=(1,1))
# self.conv1d = nn.Conv1d(in_size, out_size, kernel_size=(1,1))
self.conv1d = nn.Conv2d(in_size, out_size, kernel_size=(1, 1))

def forward(self, inputs1, inputs2):
outputs2 = self.up(inputs2)
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@


def main(input_path, output_dir=None, cache_path=None, seq_postfix='_dsfd_seq.pkl', resolution=256, crop_scale=2.0,
select='all', disable_tqdm=False, encoder_codec='avc1'):
select='all', disable_tqdm=False, encoder_codec='mp4v'):
cache_path = os.path.splitext(input_path)[0] + seq_postfix if cache_path is None else cache_path
if output_dir is None:
output_dir = os.path.splitext(input_path)[0]
Expand Down Expand Up @@ -140,7 +140,7 @@ def main(input_path, output_dir=None, cache_path=None, seq_postfix='_dsfd_seq.pk
help='selection method [all|longest]')
parser.add_argument('-dt', '--disable_tqdm', dest='disable_tqdm', action='store_true',
help='if specified disables tqdm progress bar')
parser.add_argument('-ec', '--encoder_codec', default='avc1', metavar='STR',
parser.add_argument('-ec', '--encoder_codec', default='mp4v', metavar='STR',
help='encoder codec code')
args = parser.parse_args()
main(args.input, args.output, args.cache, args.seq_postfix, args.resolution, args.crop_scale, args.select,
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,13 @@
help='display the rendering')
general.add_argument('-v', '--verbose', default=0, type=int, metavar='N',
help='verbose level')
general.add_argument('-ec', '--encoder_codec', default='avc1', metavar='STR',
general.add_argument('-ec', '--encoder_codec', default='mp4v', metavar='STR',
help='encoder codec code')

detection = base_parser.add_argument_group('detection')
detection.add_argument('-dm', '--detection_model', metavar='PATH', default='../weights/WIDERFace_DSFD_RES152.pth',
detection.add_argument('-dm', '--detection_model', metavar='PATH', default='weights/WIDERFace_DSFD_RES152.pth',
help='path to face detection model')
detection.add_argument('-db', '--det_batch_size', default=8, type=int, metavar='N',
detection.add_argument('-db', '--det_batch_size', default=4, type=int, metavar='N',
help='detection batch size')
detection.add_argument('-dp', '--det_postfix', default='_dsfd.pkl', metavar='POSTFIX',
help='detection file postfix')
Expand All @@ -77,7 +77,7 @@
help='write empty sequence lists to file')

pose = base_parser.add_argument_group('pose')
pose.add_argument('-pm', '--pose_model', default='../weights/hopenet_robust_alpha1.pth', metavar='PATH',
pose.add_argument('-pm', '--pose_model', default='weights/hopenet_robust_alpha1.pth', metavar='PATH',
help='path to face pose model file')
pose.add_argument('-pb', '--pose_batch_size', default=128, type=int, metavar='N',
help='pose batch size')
Expand All @@ -91,7 +91,7 @@
help='poses temporal smoothing kernel size')

landmarks = base_parser.add_argument_group('landmarks')
landmarks.add_argument('-lm', '--lms_model', default='../weights/hr18_wflw_landmarks.pth', metavar='PATH',
landmarks.add_argument('-lm', '--lms_model', default='weights/hr18_wflw_landmarks.pth', metavar='PATH',
help='landmarks model')
landmarks.add_argument('-lb', '--lms_batch_size', default=64, type=int, metavar='N',
help='landmarks batch size')
Expand All @@ -103,9 +103,9 @@
help='landmarks temporal smoothing kernel size')

segmentation = base_parser.add_argument_group('segmentation')
segmentation.add_argument('-sm', '--seg_model', default='../weights/celeba_unet_256_1_2_segmentation_v2.pth',
segmentation.add_argument('-sm', '--seg_model', default='weights/celeba_unet_256_1_2_segmentation_v2.pth',
metavar='PATH', help='segmentation model')
segmentation.add_argument('-sb', '--seg_batch_size', default=32, type=int, metavar='N',
segmentation.add_argument('-sb', '--seg_batch_size', default=16, type=int, metavar='N',
help='segmentation batch size')
segmentation.add_argument('-sep', '--segmentation_postfix', default='_seg.pkl', metavar='POSTFIX',
help='segmentation file postfix')
Expand Down Expand Up @@ -240,6 +240,7 @@ def process_pose(self, input_path, output_dir, seq_file_path):

if os.path.isfile(curr_pose_path):
continue
assert os.path.isfile(curr_vid_path), f'The cropped sequence video file is missing: {curr_vid_path}'
print('=> Computing face poses for video: "%s"...' % curr_vid_name)

# Initialize input video
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion utils/video_renderer.py → fsgan/utils/video_renderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ class VideoRenderer(mp.Process):
separate_process (bool): If True, the renderer will be run in a separate process
"""
def __init__(self, display=False, verbose=0, verbose_size=None, output_crop=False, resolution=256, crop_scale=1.2,
encoder_codec='avc1', separate_process=False):
encoder_codec='mp4v', separate_process=False):
super(VideoRenderer, self).__init__()
self._display = display
self._verbose = verbose
Expand Down
File renamed without changes.
26 changes: 26 additions & 0 deletions fsgan_env.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: fsgan
channels:
- pytorch
- conda-forge
- anaconda
dependencies:
- python=3.9.13
- pip=20.3.1
- cudatoolkit=11.6
- pytorch=1.12.1
- torchvision=0.13.1
- ffmpeg=4.4.1
- yacs=0.1.8
- pip:
- setuptools==58.2.0
- torch-summary==1.4.5
- opencv-contrib-python==4.5.4.60
- tensorflow==2.7.0
- tqdm==4.64.1
- matplotlib==3.6.2
- ffmpeg-python==0.2.0
- PyYAML==6.0
- pandas==1.5.1
- seaborn==0.12.1
- scipy==1.9.3
- git+https://github.com/YuvalNirkin/face_detection_dsfd.git
14 changes: 14 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import setuptools

setuptools.setup(
name="fsgan",
version="1.0.1",
author="Dr. Yuval Nirkin",
author_email="yuval.nirkin@gmail.com",
description="FSGAN: Subject Agnostic Face Swapping and Reenactment",
long_description_content_type="text/markdown",
package_data={'': ['license.txt']},
include_package_data=True,
packages=setuptools.find_packages(),
python_requires='>=3.6',
)

0 comments on commit 61746a4

Please sign in to comment.