Skip to content

Unable to Achieve High-Resolution Results as Shown in README Examples #7

Open
@brioanimationdev

Description

Hi,

I’m having trouble achieving the same high-resolution results as shown in the README examples. I followed the tutorial steps provided in the README closely and recorded my video using an iPhone 13 Mini. Below are the detailed steps I took:

Steps Followed:

  1. Uploaded an 8-second video of my hand (light conditions were good).
  2. Split the video into frames.
  3. Created frame_list_test.txt and frame_list_train.txt based on the frames.
  4. Ran RobustVideoMatting to generate the mask data.
  5. Ran Mediapipe to extract the keypoint data.
  6. Ran Depth-Anything-V2 to generate the depth maps.
  7. Ran InterWild to generate the mano_params data.
  8. Created the cam_param.json file using the following script:
    import math
    import json
    
    # Frame dimensions
    img_width = 720
    img_height = 1080
    
    # Principal point (center of the image)
    princpt_x = img_width / 2
    princpt_y = img_height / 2
    
    # Horizontal Field of View (HFOV) in degrees
    HFOV_deg = 73
    HFOV_rad = math.radians(HFOV_deg)
    
    # Compute focal length in pixels
    focal_length_px = (img_width / 2) / math.tan(HFOV_rad / 2)
    
    # Create camera parameters dictionary
    cam_param = {
        "focal": [focal_length_px, focal_length_px],  # Assuming fx = fy
        "princpt": [princpt_x, princpt_y]
    }
    
    # Save to cam_param.json
    cam_param_path = '/content/UHM/data/Custom/data/YourSubjectName/0/cam_param.json'
    with open(cam_param_path, 'w') as f:
        json.dump(cam_param, f, indent=4)
  9. Placed and arranged all the models as per the README instructions.
  10. Ran the training script. After training completed, I received 10 snapshot files (snapshot_10.pth through snapshot_99.pth).
  11. Tested with snapshot 99 using the following command:
    %cd /content/UHM/Avatar/main/
    !python test.py --subject_id TomerSubject1 --test_epoch 99
  12. Ran the test script again with the texture flag:
    %cd /content/UHM/Avatar/main/
    !python test.py --subject_id TomerSubject1 --test_epoch 99 --use_tex
  13. Finally, ran the test script with the fit_pose_to_test flag:
    %cd /content/UHM/Avatar/main/
    !python test.py --subject_id TomerSubject1 --test_epoch 99 --use_tex --fit_pose_to_test

Issue:

Despite following all these steps, the resulting videos/meshes are not as accurate or detailed as I had hoped. I’ve attached all the data I used for reference. Could you please review it and help me figure out what I might be doing wrong? Any tips to achieve better results would be greatly appreciated!
my results + data:
https://drive.google.com/drive/folders/1CFGv7VChYxkTQOsNI_FevL7HMAB9cc6V?usp=sharing

Thanks in advance,
Brio

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions