State of the Art Models
Aayush Bansal et al (CVPR 2016)
"Marr Revisited" - AKA "SkipNet"
https://arxiv.org/pdf/1604.01347.pdf
Github: https://github.com/aayushbansal/MarrRevisited
- matlab... Only uses 1449 images?
Zhang et al (CVPR 2017)
Note: This is more of a dataset introduction paper.
https://arxiv.org/pdf/1612.07429.pdf
Github: https://github.com/yindaz/surface_normal. They also provide pretrained models (in lua)
Wang Fouhey (CVPR 2015)
https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Wang_Designing_Deep_Networks_2015_CVPR_paper.pdf
Github: https://github.com/xiaolonw/caffe-3dnormal
Eigen Fregus paper - Seems the gold standard (ICCV 2015)
Uses a VGG-16 model as backbone.
https://arxiv.org/pdf/1411.4734.pdf
"Predicting depth, surface normals and semantic labels with a common multi-scale
convolutional architecture"
Chen et al (ICCV 2017)
https://arxiv.org/pdf/1704.02956.pdf
Github: https://github.com/princeton-vl/surface_normals. They also provide pretrained models in lua)
Dataset
NYU v2
Depth video dataset.
Need to convert to normals
Need to sample frames, then convert to normals. Several leading ways to do this, but three main ways.
- Ladicky, Fouhey, Wang.
Ladicky et al (ECCV 2014)
Discriminatively
trained dense surface normal estimation.
https://www.inf.ethz.ch/personal/ladickyl/normals_eccv14.pdf (referred by link)
- https://www.inf.ethz.ch/personal/ladickyl/nyu_normals_gt.zip
- https://github.com/aayushbansal/PixelNet/tree/master/models/analysis
-
Qi et al (CVPR 2018)
GeoNet (SOTA, with less data)
No found github link. Or pretrained model.
Summary from papers on the right
Metric: mean angular error
19.0 - GeoNet (Qi et al, CVPR 2018)
19.4 - "Baseline" VGG-16 model (from GeoNet paper)
19.8 - SkipNet (Aayush et al, CVPR 2016)
20.6 - SURGE (Wang et al, # from GeoNet paper)
23.7 - Multi-scale CNN (Eigen Furgus, ICCV 2015)
26.9 - Deep3D (Wang Fouhey, CVPR 2015)
Wang et al (NIPS 2016)
SURGE (basically VGG-16 + CRF)
https://papers.nips.cc/paper/6502-surge-surface-regularized-geometry-estimation-from-a-single-image.pdf
They've defined their own new metrics so that's why these numbers are off.
