Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIX2POSE refactor #171

Merged
merged 101 commits into from
Feb 9, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
635e029
Ignore .h5 files from repo
oarriaga Oct 15, 2021
0017ce8
Start refactoring model generator
oarriaga Oct 15, 2021
403c86a
Refactor discriminator
oarriaga Oct 15, 2021
263b646
Split model and loss utils
oarriaga Oct 15, 2021
45da37f
Refactor domain randomization processor
oarriaga Oct 15, 2021
42e85dc
Ignore .iml file in complete repository
oarriaga Oct 16, 2021
ac22307
Add quaternion backend and basic coloring scheme
oarriaga Oct 16, 2021
d49e9f9
Add scene for rendering pixel and normal image
oarriaga Oct 18, 2021
5c1c726
Start refactoring loss
oarriaga Oct 18, 2021
e49f96d
Refactor weighted foreground loss
oarriaga Oct 18, 2021
b71bf3b
Change directory name to hold generic models
oarriaga Oct 18, 2021
3e1186f
Fix bug with pipeline incorrect output shape
oarriaga Oct 18, 2021
50645db
Add a fully convolutional neural network based on KeypointNet2D
oarriaga Oct 18, 2021
524ecfc
Fix bug with predict weighted foreground loss
oarriaga Oct 21, 2021
f4a8659
Add backend functions for prediction
oarriaga Oct 25, 2021
cab9eb1
Add small comment on how to get object 3D shape
oarriaga Oct 25, 2021
f8c0dcc
Add partially tested pipeline for full inference
oarriaga Oct 25, 2021
5251fe9
Add simple processors for pix2pose inference
oarriaga Oct 25, 2021
cb610c0
Changed train script to use UNET-VGG
oarriaga Oct 25, 2021
8876c11
Add structure with video player
oarriaga Oct 25, 2021
66e8ac5
Add simple ICP computation
oarriaga Oct 25, 2021
bf385b4
Add working demo
oarriaga Oct 27, 2021
1906c4a
Refactor main pipeline
oarriaga Oct 28, 2021
a512a09
Refactor pipelines
oarriaga Oct 28, 2021
bfd1ebd
Refactor backend
oarriaga Oct 28, 2021
858cf26
Add basic processor
oarriaga Oct 28, 2021
a402331
Start ObjectHypothesis example
oarriaga Oct 28, 2021
9810748
Remove comments
oarriaga Oct 28, 2021
501f434
Refactor code
oarriaga Oct 28, 2021
46c4e2d
Refactor code to train GAN
oarriaga Nov 1, 2021
5810d8e
Add training for UNET and GAN
oarriaga Nov 1, 2021
d2b014f
Remove unecessary metrics
oarriaga Nov 1, 2021
3b85afa
Add available GAN training
oarriaga Nov 2, 2021
b9901b8
Add additional losses to pix2pose
oarriaga Nov 8, 2021
7d72dde
Add basic training with full GAN model
oarriaga Nov 8, 2021
7472cea
Add fix to PnP having to solve for less than 4 mask points
oarriaga Nov 17, 2021
74c4888
Change demo for image processing
oarriaga Nov 17, 2021
b6aeb0b
Add accessible state of failure and success of internal PnP solution
oarriaga Nov 17, 2021
ca3b26c
Remove unsued functions and processors
oarriaga Nov 17, 2021
6266d3a
Remove unnecessary files
oarriaga Nov 17, 2021
d9fe2c0
Add comments to functions
oarriaga Nov 18, 2021
acde9da
Add backend function comments
oarriaga Nov 18, 2021
2535574
Remove unecessary values from pipelines
oarriaga Nov 18, 2021
2f37602
Remove unecessary files
oarriaga Nov 18, 2021
22ea0a1
Add resize option for augmenting keypoints based on interpolation
oarriaga Nov 18, 2021
bc30c79
Add comments to missing functions
oarriaga Nov 19, 2021
a8066a4
Remove unnecessary flags for estimate keypoints pipeline
oarriaga Nov 19, 2021
646d733
Add reminder to debug incorrect shape management
oarriaga Nov 19, 2021
cf8a1d7
Add comments to loss functions
oarriaga Nov 19, 2021
f300584
Removed keras GAN examples
oarriaga Nov 19, 2021
3177930
Remove unecessary metric function
oarriaga Nov 19, 2021
0338f70
Add untested symmetric weighted loss
oarriaga Nov 19, 2021
db21312
Change space name to compute graphics convention i.e. NDC
oarriaga Nov 19, 2021
34f1565
Add NDC transforms and rotation matrix builds
oarriaga Nov 19, 2021
a5fd6a0
Add untested symmetric loss training script
oarriaga Nov 19, 2021
a9f2a27
Add predictions transformation
oarriaga Nov 23, 2021
b2aca58
Fix bug with rotation matrix creation
oarriaga Nov 23, 2021
e9ac4bb
Add python rotate image function
oarriaga Nov 23, 2021
31b063b
Refactor symmetric loss with based on rotate_image backend function
oarriaga Nov 23, 2021
0ec6aa0
Update training scripts
oarriaga Nov 23, 2021
a36763b
Add canonical coloring scheme scene
oarriaga Nov 25, 2021
2a86373
Add rotation build matrices
oarriaga Nov 25, 2021
86bec27
Add training script for canonical pose estimation
oarriaga Nov 25, 2021
0aa168b
Move canonical functions to backend
oarriaga Nov 26, 2021
049f3df
Fix bug with tensor name being overwritten
oarriaga Nov 26, 2021
6dac02b
Add scene for canonical discrete transformations
oarriaga Nov 26, 2021
5a8083d
Add training script for canonical transformation
oarriaga Nov 26, 2021
9134957
Update training canonical scripts
oarriaga Nov 29, 2021
8111c11
Add single drawing mask function
oarriaga Nov 30, 2021
042630f
Add additional pipelines for single inference visualization
oarriaga Nov 30, 2021
3cbba0d
Refactor pipelines to work independent from detector
oarriaga Nov 30, 2021
0a8b1f3
Refactor demo to work with new pipelines
oarriaga Nov 30, 2021
63bc57c
Found bug with mask drawing when box2D is given
oarriaga Dec 1, 2021
a54c3c1
Refactor pipelines for better modularity
oarriaga Dec 1, 2021
0555b94
Add MultiPoseEstimation pipeline
oarriaga Dec 2, 2021
424430b
Add multi samples in demo
oarriaga Dec 2, 2021
be139c2
Added parameters for multiple objects in the scene
oarriaga Dec 2, 2021
46492c3
Refactor training script for multiple objects
oarriaga Dec 2, 2021
efa75f7
Remove jpegs from repository
oarriaga Dec 2, 2021
0587a9e
Add rotated image
oarriaga Dec 2, 2021
1818024
Remove unused loss backend function
oarriaga Jan 19, 2022
1c9d520
Add basic tests for loss functions
oarriaga Jan 25, 2022
8a50045
Start test for backend
oarriaga Jan 26, 2022
28cf4c7
Add tests and refactor backend
oarriaga Feb 1, 2022
0c4700e
Removed mulitple pix2pose pipeline
oarriaga Feb 1, 2022
24db9d0
Move scene to legacy
oarriaga Feb 1, 2022
4472df9
Remove test
oarriaga Feb 1, 2022
8aba37d
Rearranged structure to include tested functionality
oarriaga Feb 1, 2022
c5939cb
Refactor coloring to be in backend and add pixel mask rendering for s…
oarriaga Feb 1, 2022
1959b5f
Fix bug with demo importing legacy pipeline
oarriaga Feb 1, 2022
3b0cfb6
Refactor basic training script for pix2pose RGB mask
oarriaga Feb 2, 2022
3be885f
Move metrics to legacy dir
oarriaga Feb 3, 2022
788de88
Refactor tests
oarriaga Feb 5, 2022
99c1954
Add backend and test processors
oarriaga Feb 8, 2022
7b174c4
Add comment to function
oarriaga Feb 8, 2022
59a3ea5
Add drawing callback
oarriaga Feb 8, 2022
305861b
Remove drawing of pose6D in Pix2Pose pipeline and save original image…
oarriaga Feb 8, 2022
d3f2f81
Add thickness option for drawing poses
oarriaga Feb 9, 2022
41edbf0
Change resize interpolation
oarriaga Feb 9, 2022
c081d6a
Revert demo for single image
oarriaga Feb 9, 2022
72d1400
Delete legacy file to keep pep8 master
oarriaga Feb 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move metrics to legacy dir
  • Loading branch information
oarriaga committed Feb 3, 2022
commit 3be885f36dddb082f52f558441cba429dfdfdc7a
File renamed without changes.
13 changes: 10 additions & 3 deletions examples/pix2pose/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import argparse
from datetime import datetime

import tensorflow as tf
from tensorflow.keras.utils import get_file
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import (
Expand All @@ -14,8 +15,7 @@

from scenes import PixelMaskRenderer
from pipelines import DomainRandomization
from loss import WeightedReconstruction
from metrics import mean_squared_error as MSE
from weighted_reconstruction import WeightedReconstruction

MTL_FILE = 'textured.mtl'
OBJ_FILE = 'textured.obj'
Expand Down Expand Up @@ -98,11 +98,18 @@
# building python generator
sequence = GeneratingSequence(processor, args.batch_size, args.steps_per_epoch)


# metric for labels with alpha mask
def mean_squared_error(y_true, y_pred):
squared_difference = tf.square(y_true[:, :, :, 0:3] - y_pred[:, :, :, 0:3])
return tf.reduce_mean(squared_difference, axis=-1)


# instantiating the model and loss
model = UNET_VGG16(num_channels, image_shape, freeze_backbone=True)
optimizer = Adam(args.learning_rate)
loss = WeightedReconstruction(args.beta)
model.compile(optimizer, loss, metrics=MSE)
model.compile(optimizer, loss, mean_squared_error)

# building experiment path
experiment_label = '_'.join([model.name, args.run_label])
Expand Down