手持物体重建相关工作汇总(截止至2025.3)
汇总近几年的相关工作.
(CVPR 2024) HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video[page]
Zicong Fan1,2 Maria Parelli1 Maria Eleni Kadoglou1 Muhammed Kocabas1,2 Xu Chen1,2,† Michael J. Black2 Otmar Hilliges1
1ETH Z ̈ urich, Switzerland 2Max Planck Institute for Intelligent Systems, T ̈ ubingen, Germany
手持物体重建仅是其实验的一部分，
metrics
测试的指标包括: 手部姿态准确度(MPJPE, root-relative mean-per-joint error), 物体姿态和形状的准确度(CD, Chamfer Distance), F-score(F10, 10mm阈值以下被认为正确)和物体的手相对倒角距离($\textbf{CD}_{h}$, hand-relative Chamfer distance for the object)
数据集
HO3D(对比方法iHOI在训练过程中使用了测试物体的3D标注)
对比方法
iHOI, DiffHOI
(3DV 2024) Color-NeuS: Reconstructing Neural Implicit Surfaces with Color[page]
Licheng Zhong1 ⋆ Lixin Yang1,2 ⋆ Kailin Li1 Haoyu Zhen1 Mei Han3 Cewu Lu1,2 †
1Shanghai Jiao Tong University 2Shanghai Qi Zhi Institute 3National University of Singapore
测试指标
数据集
(CVPR 2023) In-Hand 3D Object Scanning from an RGB Sequence[page]
Shreyas Hampali1,3, Tomas Hodan1, Luan Tran1, Lingni Ma1, Cem Keskin1, Vincent Lepetit2,3
1Reality Labs at Meta, 2LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS, Marne-la-Vallee, France, 3Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria
将连续的RGB序列拆分成不同的片段, 并确保片段之间有重合。
YCB, Aria
对于香蕉、剪刀这种“薄”+“弱纹理”特点的物体，重建容易失败。
(ICCV 2023) CHORD: Category-level Hand-held Object Reconstruction via Shape Deformation [page]
Kailin Li, Zhewei Huang, Chen Wang, Zhiyuan Wang, Juyong Zhang
University of Science and Technology of China, Shanghai AI Lab
CHORD 方法提出了一种基于类别形状先验的变形重建方法。
Metrics
测试的指标包括: 形状重建精度 (Chamfer Distance, CD), 形状完整性 (F-score)。
数据集
COMIC (本文新构建的数据集) 和 HO3D。
对比方法
iHOI, ObMan, NDF, DDF-HO
(SIGGRAPH Asia 2022) Reconstructing Hand-Held Objects from Monocular Video[page]
- RGB video作为输入
- 相机固定不动, 需要借助背景图像来分割手和物体，再使用hand segmentation进一步将手分割出来。
测试指标
Chamfer Distance
数据集
自建数据集HOD, 35个物体
对比方法
ObMan, GF, IHOI
(CVPR 2022) What’s in your hands? 3D Reconstruction of Generic Objects in Hands.[page]
Wentian Qu1,2 Zhaopeng Cui3 Yinda Zhang4 Chenyu Meng1,2 Cuixia Ma1,2 Xiaoming Deng1,2* Hongan Wang1,2∗
1Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences 3State Key Lab of CAD&CG, Zhejiang University 4Google
(ICCV 2023) Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views
稀疏视角重建
(2021 3DV)Towards Unconstrained Joint Hand-Object Reconstruction From RGB Videos [page]
()
()
()
()
(CVPR 2022) What’s in Your Hands? 3D Reconstruction of Generic Objects in Hands [page]
Yufei Ye, Shubham Tulsiani, Abhinav Gupta
Carnegie Mellon University
该方法利用手部姿态作为条件约束，从单帧 RGB 图像中重建物体。
Metrics
测试的指标包括: 形状重建精度 (Chamfer Distance, CD) 和 F-score。
数据集
HO3D, DexYCB
对比方法
ObMan, iHOI, SDF-based methods
(ICCV 2023) HO-NeRF: Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views [page]
Wentian Qu, Jiarui Xu, Animesh Garg
University of Toronto
HO-NeRF 使用 NeRF 进行新视角合成，并优化手部和物体的位姿。
Metrics
测试的指标包括: 视角合成质量 (PSNR, SSIM, LPIPS)，姿态估计误差 (MPJPE, CD)。
数据集
DexYCB, ObMan
对比方法
iHOI, DiffHOI, NeRF-baselines
(CVPR 2022) Collaborative Learning for Hand and Object Reconstruction with Attention-Guided Graph Convolution [page]
Tze Ho Elden Tse, Zhimin Chen, Antonio Garcia-Uceda, Gregory Rogez, Edmond Boyer, Helge Rhodin
INRIA, University of British Columbia
该方法利用注意力引导的图神经网络进行手部和物体的联合重建。
Metrics
测试的指标包括: 形状重建精度 (Chamfer Distance, CD)，F-score，手部姿态误差 (MPJPE)。
数据集
HO3D, DexYCB
对比方法
ObMan, iHOI, Graph-based methods
其他的相关工作
(WACV 2025) DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing [page]
引入深度和法向先验来得到更好的3dgs与meshing
(CVPR 2025) https://github.com/facebookresearch/fast3r [page]
添加了全局fusion来加速原本的匹配过程。
(arXiv 2024) [Spann3R] 3D Reconstruction with Spatial Memory [page]
