update

page/content/papers/cagenerf/index.md

@@ -17,7 +17,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 4
+series_order: 5
 ---
 
 ## `CageNeRF`: Cage-based Neural Radiance Fields for Genrenlized 3D Deformation and Animation

page/content/papers/danbo/index.md

@@ -17,7 +17,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 1
+series_order: 15
 ---
 
 ## `DANBO`: Disentangled Articulated Neural Body Representations via Graph Neural Networks

page/content/papers/dracon/index.md

@@ -17,7 +17,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 5
+series_order: 21
 ---
 
 ## `DRaCoN` – Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars

page/content/papers/fpo/index.md

@@ -0,0 +1,60 @@
+---
+date: 2022-06-21T04:14:54-08:00
+draft: false
+params:
+  author: Nikolaos Zioulis
+title: FPO
+categories: ["papers"]
+tags: ["nerf", "deformation", "cvpr22"]
+layout: simple
+menu: #
+robots: all
+# sharingLinks: #
+weight: 10
+showHero: true
+description: "Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time"
+summary: TODO
+keywords: #
+type: '2022' # we use year as a type to list papers in the list view
+series: ["Papers Published @ 2022"]
+series_order: 20
+---
+
+## Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time
+
+> Liao Wang1∗
+Jiakai Zhangi, Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, Lan Xu
+
+{{< keywordList >}}
+{{< keyword icon="tag" >}} NeRF {{< /keyword >}}
+{{< keyword icon="tag" >}} Deformation {{< /keyword >}}
+{{< keyword icon="email" >}} *CVPR* 2022 {{< /keyword >}}
+{{< /keywordList >}}
+
+### Abstract
+{{< lead >}}
+Implicit neural representations such as Neural Radiance Field (NeRF) have focused mainly on modeling static objects captured under multi-view settings where real-time rendering can be achieved with smart data structures, e.g., PlenOctree. In this paper, we present a novel Fourier PlenOctree (FPO) technique to tackle efficient neural modeling and real-time rendering of dynamic scenes captured under the free-view video (FVV) setting. The key idea in our FPO is a novel combination of generalized NeRF, PlenOctree representation, volumetric fusion and Fourier transform. To accelerate FPO construction, we present a novel coarse-to-fine fusion scheme that leverages the generalizable NeRF technique to generate the tree via spatial blending. To tackle dynamic scenes, we tailor the implicit network to model the Fourier coefficients of timevarying density and color attributes. Finally, we construct the FPO and train the Fourier coefficients directly on the leaves of a union PlenOctree structure of the dynamic sequence. We show that the resulting FPO enables compact memory overload to handle dynamic objects and supports efficient fine-tuning. Extensive experiments show that the proposed method is 3000 times faster than the original NeRF and achieves over an order of magnitude acceleration over SOTA while preserving high visual quality for the free-viewpoint rendering of unseen dynamic scenes.
+{{< /lead >}}
+
+{{< button href="https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Fourier_PlenOctrees_for_Dynamic_Radiance_Field_Rendering_in_Real-Time_CVPR_2022_paper.pdf" target="_blank" >}}
+Paper
+{{< /button >}}
+
+### Approach
+
+{{< figure
+    src="overview.png"
+    alt="FPO overview"
+    caption="`FPO` overview."
+    >}}
+
+### Results
+
+#### Comparisons
+{{<badge label="body--NeRF" message="NeuralBody" color="coral" logo="github" link="https://github.com/zju3dv/neuralbody" target="_blank">}}
+
+#### Performance
+{{<badge label="train" message="2h" color="informational" logo="link" >}}
+{{<badge label="train" message="RTX3090" color="informational" logo="link" >}}
+{{<badge label="render" message="800_x_800" color="informational" logo="link" >}}
+{{<badge label="render" message="10ms" color="informational" logo="link" >}}

page/content/papers/gnr/index.md

@@ -17,7 +17,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 4
+series_order: 12
 ---
 
 ## `Generalizable Neural Performer`: Learning Robust Radiance Fields for Human Novel View Synthesis

page/content/papers/gpnerf/index.md

@@ -18,7 +18,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 3
+series_order: 17
 ---
 
 ## `GP-NeRF`: Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

page/content/papers/monohuman/index.md

@@ -0,0 +1,67 @@
+---
+date: 2023-06-20T04:14:54-08:00
+draft: false
+params:
+  author: Nikolaos Zioulis
+title: MonoHuman
+categories: ["papers"]
+tags: ["nerf", "smpl", "monocular", "cvpr23"]
+layout: simple
+menu: #
+robots: all
+# sharingLinks: #
+weight: 10
+showHero: true
+description: "MonoHuman: Animatable Human Neural Field from Monocular Video"
+summary: TODO
+keywords: #
+type: '2023' # we use year as a type to list papers in the list view
+series: ["Papers Published @ 2023"]
+series_order: 7
+---
+
+## `MonoHuman`: Animatable Human Neural Field from Monocular Video
+
+> Zhengming Yu, Wei Cheng, Xian Liu, Wayne Wu, Kwan-Yee Lin
+
+{{< keywordList >}}
+{{< keyword icon="tag" >}} NeRF {{< /keyword >}}
+{{< keyword icon="tag" >}} SMPL {{< /keyword >}}
+{{< keyword icon="tag" >}} Monocular {{< /keyword >}}
+{{< keyword icon="email" >}} *CVPR* 2023 {{< /keyword >}}
+{{< /keywordList >}}
+
+{{< github repo="Yzmblog/MonoHuman" >}}
+
+### Abstract
+{{< lead >}}
+Animating virtual avatars with free-view control is crucial for various applications like virtual reality and digital entertainment. Previous studies attempt to utilize the representation power of neural radiance field (NeRF) to reconstruct the human body from monocular videos. Recent works propose to graft a deformation network into the NeRF to further model the dynamics of the human neural field for animating vivid human motions. However, such pipelines either rely on pose-dependent representations or fall short of motion coherency due to frame-independent optimization, making it difficult to generalize to unseen pose sequences realistically.
+
+In this paper, we propose a novel framework MonoHuman, which robustly renders view-consistent and high-fidelity avatars under arbitrary novel poses. Our key insight is to model the deformation field with bi-directional constraints and explicitly leverage the off-the-peg keyframe information to reason the feature correlations for coherent results. In particular, we first propose a Shared Bidirectional Deformation module, which creates a pose-independent generalizable deformation field by disentangling backward and forward deformation correspondences into shared skeletal motion weight and separate non-rigid motions. Then, we devise a Forward Correspondence Search module, which queries the correspondence feature of keyframes to guide the rendering network. The rendered results are thus multi-view consistent with high fidelity, even under challenging novel pose settings. Extensive experiments demonstrate the superiority of our proposed MonoHuman over state-of-the-art methods.
+{{< /lead >}}
+
+{{< button href="https://openaccess.thecvf.com/content/CVPR2023/papers/Yu_MonoHuman_Animatable_Human_Neural_Field_From_Monocular_Video_CVPR_2023_paper.pdf" target="_blank" >}}
+Paper
+{{< /button >}}
+
+### Approach
+
+{{< figure
+    src="overview.png"
+    alt="MonoHuman overview"
+    caption="`MonoHuman` overview."
+    >}}
+
+### Results
+
+#### Data
+{{<badge label="test" message="ZJU_MOCAP" color="yellowgreen" logo="github" link="https://github.com/zju3dv/neuralbody/blob/master/INSTALL.md#zju-mocap-dataset" target="_blank">}}
+
+#### Comparisons
+{{<badge label="body--NeRF" message="NeuralBody" color="coral" logo="github" link="https://github.com/zju3dv/neuralbody" target="_blank">}}
+{{<badge label="body--NeRF" message="HumanNeRF" color="blue" logo="github" link="chungyiweng/HumanNeRF" target="_blank">}}
+{{<badge label="body--NeRF" message="NeuMan" color="white" logo="github" link="apple/ml-neuman" target="_blank">}}
+
+#### Performance
+{{<badge label="train" message="70h" color="informational" logo="link" >}}
+{{<badge label="train" message="V100" color="informational" logo="link" >}}

page/content/papers/neuman/index.md

@@ -0,0 +1,62 @@
+---
+date: 2022-10-23T04:14:54-08:00
+draft: false
+params:
+  author: Nikolaos Zioulis
+title: NeuMan
+categories: ["papers"]
+tags: ["nerf", "smpl", "monocular", "eccv22"]
+layout: simple
+menu: #
+robots: all
+# sharingLinks: #
+weight: 10
+# showPagination: true
+# showHero: true
+# layoutBackgroundBlur: true
+# heroStyle: thumbAndBackground
+description: "NeuMan: Neural Human Radiance Field from a Single Video"
+summary: TODO
+keywords: #
+type: '2022' # we use year as a type to list papers in the list view
+series: ["Papers Published @ 2022"]
+series_order: 23
+---
+
+## `NeuMan`: Neural Human Radiance Field from a Single Video
+
+> Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan
+
+{{< keywordList >}}
+{{< keyword icon="tag" >}} NeRF {{< /keyword >}}
+{{< keyword icon="tag" >}} SMPL  {{< /keyword >}}
+{{< keyword icon="tag" >}} Monocular {{< /keyword >}}
+{{< keyword icon="email" >}} *ECCV* 2022 {{< /keyword >}}
+{{< /keywordList >}}
+
+{{< github repo="apple/ml-neuman" >}}
+
+### Abstract
+{{< lead >}}
+Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences. We propose a novel framework to reconstruct the human and the scene that can be rendered with novel human poses and views from just a single in-the-wild video. Given a video captured by a moving camera, we train two NeRF models: a human NeRF model and a scene NeRF model. To train these models, we rely on existing methods to estimate the rough geometry of the human and the scene. Those rough geometry estimates allow us to create a warping field from the observation space to the canonical pose-independent space, where we train the human model in. Our method is able to learn subject specific details, including cloth wrinkles and accessories, from just a 10 seconds video clip, and to provide high quality renderings of the human under novel poses, from novel views, together with the background.
+{{< /lead >}}
+
+{{< button href="https://arxiv.org/pdf/2203.12575" target="_blank" >}}
+Paper
+{{< /button >}}
+
+### Approach
+
+{{< figure
+    src="overview.jpg"
+    alt="NeuMan overview"
+    caption="`NeuMan` overview."
+    >}}
+
+### Results
+
+#### Data
+{{<badge label="test" message="ZJU_MOCAP" color="yellowgreen" logo="github" link="https://github.com/zju3dv/neuralbody/blob/master/INSTALL.md#zju-mocap-dataset" target="_blank">}}
+
+#### Comparisons
+{{<badge label="body--NeRF" message="NeuralBody" color="coral" logo="github" link="https://github.com/zju3dv/neuralbody" target="_blank">}}

page/content/papers/relighting4d/index.md

@@ -0,0 +1,66 @@
+---
+date: 2022-10-23T04:14:54-08:00
+draft: false
+params:
+  author: Nikolaos Zioulis
+title: Relighting4D
+categories: ["papers"]
+tags: ["nerf", "smpl", "monocular", "eccv22"]
+layout: simple
+menu: #
+robots: all
+# sharingLinks: #
+weight: 10
+# showPagination: true
+# showHero: true
+# layoutBackgroundBlur: true
+# heroStyle: thumbAndBackground
+description: "Relighting4D: Neural Relightable Human from Videos"
+summary: TODO
+keywords: #
+type: '2022' # we use year as a type to list papers in the list view
+series: ["Papers Published @ 2022"]
+series_order: 19
+---
+
+## `Relighting4D`: Neural Relightable Human from Videos
+
+> Zhaoxi Chen, Ziwei Liu
+
+{{< keywordList >}}
+{{< keyword icon="tag" >}} NeRF {{< /keyword >}}
+{{< keyword icon="tag" >}} SMPL  {{< /keyword >}}
+{{< keyword icon="tag" >}} Monocular {{< /keyword >}}
+{{< keyword icon="email" >}} *ECCV* 2022 {{< /keyword >}}
+{{< /keywordList >}}
+
+{{< github repo="FrozenBurning/Relighting4D" >}}
+
+### Abstract
+{{< lead >}}
+Human relighting is a highly desirable yet challenging task. Existing works either require expensive one-light-at-a-time (OLAT) captured data using light stage or cannot freely change the viewpoints of the rendered body. In this work, we propose a principled framework, Relighting4D, that enables free-viewpoints relighting from only human videos under unknown illuminations. Our key insight is that the spacetime varying geometry and reflectance of the human body can be decomposed as a set of neural fields of normal, occlusion, diffuse, and specular maps. These neural fields are further integrated into reflectance-aware physically based rendering, where each vertex in the neural field absorbs and reflects the light from the environment. The whole framework can be learned from videos in a self-supervised manner, with physically informed priors designed for regularization. Extensive experiments on both real and synthetic datasets demonstrate that our framework is capable of relighting dynamic human actors with free-viewpoints.
+{{< /lead >}}
+
+{{< button href="https://arxiv.org/pdf/2207.07104" target="_blank" >}}
+Paper
+{{< /button >}}
+
+### Approach
+
+{{< figure
+    src="overview.png"
+    alt="Relighting4D overview"
+    caption="`Relighting4D` overview."
+    >}}
+
+### Results
+
+#### Data
+{{<badge label="test" message="ZJU_MOCAP" color="yellowgreen" logo="github" link="https://github.com/zju3dv/neuralbody/blob/master/INSTALL.md#zju-mocap-dataset" target="_blank">}}
+{{<badge label="test" message="PeopleSnapshot" color="lightblue" logo="link" link="https://graphics.tu-bs.de/people-snapshot" target="_blank">}}
+
+#### Comparisons
+{{<badge label="body--NeRF" message="NeuralBody" color="coral" logo="github" link="https://github.com/zju3dv/neuralbody" target="_blank">}}
+
+#### Performance
+{{<badge label="train" message="V100" color="informational" logo="link" >}}

page/content/papers/structural_local/index.md

@@ -0,0 +1,69 @@
+---
+date: 2022-06-21T04:14:54-08:00
+draft: false
+params:
+  author: Nikolaos Zioulis
+title: StructuredLocal
+categories: ["papers"]
+tags: ["nerf", "smpl", "monocular", "cvpr22"]
+layout: simple
+menu: #
+robots: all
+# sharingLinks: #
+weight: 10
+# showPagination: true
+# showHero: true
+# layoutBackgroundBlur: true
+# heroStyle: thumbAndBackground
+description: Structured Local Radiance Fields for Human Avatar Modeling
+summary: TODO
+keywords: #
+type: '2022' # we use year as a type to list papers in the list view
+series: ["Papers Published @ 2022"]
+series_order: 22
+---
+
+## Structured Local Radiance Fields for Human Avatar Modeling
+
+> Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yandong Guo, Yebin Liu
+
+{{< keywordList >}}
+{{< keyword icon="tag" >}} NeRF {{< /keyword >}}
+{{< keyword icon="tag" >}} SMPL  {{< /keyword >}}
+{{< keyword icon="tag" >}} Monocular {{< /keyword >}}
+{{< keyword icon="email" >}} *CVPR* 2022 {{< /keyword >}}
+{{< /keywordList >}}
+
+### Abstract
+{{< lead >}}
+It is extremely challenging to create an animatable clothed human avatar from RGB videos, especially for loose clothes due to the diffculties in motion modeling. To address this problem, we introduce a novel representation on the basis of recent neural scene rendering techniques. The core of our representation is a set of structured local radiance felds, which are anchored to the pre-defned nodes sampled on a statistical human body template. These local radiance felds not only leverage the fexibility of implicit representation in shape and appearance modeling, but also factorize cloth deformations into skeleton motions, node residual translations and the dynamic detail variations inside each individual radiance feld. To learn our representation from RGB data and facilitate pose generalization, we propose to learn the node translations and the detail variations in a conditional generative latent space. Overall, our method enables automatic construction of animatable human avatars for various types of clothes without the need
+for scanning subject-specifc templates, and can generate realistic images with dynamic details for novel poses. Experiment show that our method outperforms state-of-the-art methods both qualitatively and quantitatively.
+{{< /lead >}}
+
+{{< button href="https://openaccess.thecvf.com/content/CVPR2022/papers/Zheng_Structured_Local_Radiance_Fields_for_Human_Avatar_Modeling_CVPR_2022_paper.pdf" target="_blank" >}}
+Paper
+{{< /button >}}
+
+### Approach
+
+{{< figure
+    src="overview.jpg"
+    alt="StructuredLocal overview"
+    caption="`StructuredLocal` overview."
+    >}}
+
+### Results
+
+#### Data
+{{<badge label="test" message="ZJU_MOCAP" color="yellowgreen" logo="github" link="https://github.com/zju3dv/neuralbody/blob/master/INSTALL.md#zju-mocap-dataset" target="_blank">}}
+{{<badge label="test" message="PeopleSnapshot" color="lightblue" logo="link" link="https://graphics.tu-bs.de/people-snapshot" target="_blank">}}
+
+#### Comparisons
+{{<badge label="body--NeRF" message="NeuralBody" color="coral" logo="github" link="https://github.com/zju3dv/neuralbody" target="_blank">}}
+{{<badge label="body--NeRF" message="AnimatableNeRF" color="cyan" logo="github" link="https://github.com/zju3dv/animatable_nerf" target="_blank">}}
+
+#### Performance
+{{<badge label="train" message="25h" color="informational" logo="link" >}}
+{{<badge label="train" message="RTX3090" color="informational" logo="link" >}}
+{{<badge label="render" message="5sec" color="informational" logo="link" >}}
+{{<badge label="render" message="512_x_512" color="informational" logo="link" >}}

page/content/papers/tava/index.md

@@ -17,7 +17,7 @@ summary: TODO
 keywords: #
 type: '2022' # we use year as a type to list papers in the list view
 series: ["Papers Published @ 2022"]
-series_order: 5
+series_order: 7
 ---
 
 ## `TAVA`: Template-free Animatable Volumetric Actors