Toward Practical Monocular Indoor Depth Estimation
CVPR 2022

   Data Instruction     Switch to Project Introduction

Data Download

The data are released with the purpose for (self-supervised) depth estimation with stereo cameras. The data are distributed only for non-commercial scientific research purpose and follow a single-user, non-exclusive, non-transferable, non-commercial, free of charge right along with CC-BY-NC License. By downloading the following data, you agree with the license. SimSIN: HM3D and Replica are released when the first-author internship at Meta, and VA and UniSIN are released when the first-author work at USC.

(1) SimSIN (HM3D + Replica), including stereo pairs + rendered depth: [Here, 363G], file list: SimSIN_release.txt. We also provide a split zip file to avoid directly large file download here. Run "zip -F spl.zip --out SimSIN.zip" after downloading all files.

MP3D data can be generated by first downloading their datasets here, and then install habitat-sim. We attach a script for how to generate stereo data with depth from MP3D.

(2) UniSIN: [Here, 457G], file list: UniSIN_500_list.txt. We also provide a split zip file to avoid directly large file download here. Run "zip -F Unispl.zip --out UniSIN.zip" after downloading all files.

UniSIN's validation set (1000 images): [Here, 1.1G]. See the leaderboard and call for participation in the later section.

(3) VA: [Here, 8G], file list: VA_left_all.txt

(4) Sample data for the above data can be downloaded [Here, 1.2G]

-- Camera intrinsics can be found in the dataloader code. You can project the estimated depth to 3D pointcloud by the intrinsics

-- We also provide a depth to pointcloud visualization code here. You can also find explicit parameters (cx, cy, focal length, stereo baseline) in the code.

Google drive might experience limited download quota. We're seeking other options to store data. A quick fix is to "make a copy" of the file you would like to download and download from your google drive. See here for details.

For further questions related to data, please contact choyingw@usc.edu

Data Explorer

Example SimSIN images

scales


Example VA sequences (image, depth)



Example UniSIN sequences (left, right)






Example UniSIN sequences (image, depth from ZED-2i)




Results on UniSIN sequences (image, depth from DistDepth)









Zero-Shot cross dataset evaluation on VA: VA_eval

[1] MonoDepth2: Clement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J. Brostow. Digging into self-supervised monocular depth estimation. In ICCV, 2019

[2] DepthHints: Jamie Watson, Michael Firman, Gabriel J Brostow, and Daniyar Turmukhambetov. Self-supervised monocular depth hints. In ICCV, 2019

[3] ManyDepth: Jamie Watson, Oisin Mac Aodha, Victor Prisacariu, Gabriel Brostow, and Michael Firman. The temporal opportunist: Self-supervised multi-frame monocular depth. In CVPR, 2021.

[4] DistDepth: Cho-Ying Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, Shuochen Su. Toward Practical Monocular Indoor Depth Estimation. In CVPR 2022.

Benchmark on UniSIN:
The units are in meters. We first run the median scaling of the predicted and groundtruth depth and calculate the errors. Error definition:

scales

Rank Method MAE (m) AbsRel (m) RMSE (m) RMSE_log10
1. DistDepth, trained on UniSIN [CVPR 22] 0.505
 0.130
 0.611
 0.162
2. DistDepth, trained on SimSIN [CVPR 22] 0.518
 0.135
 0.623
 0.159
3. MonoDepth2, trained on UniSIN [ICCV 19] 0.571
 0.163
 0.688
 0.200
4. MonoDepth2, trained on SimSIN [ICCV 19] 0.610
 0.175
 0.742
 0.232

[1] DistDepth: Cho-Ying Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, Shuochen Su. Toward Practical Monocular Indoor Depth Estimation. In CVPR 2022.

[2] MonoDepth2: Clement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J. Brostow. Digging into self-supervised monocular depth estimation. In ICCV, 2019

 Call for participation:
Please download data in the first part and predict depth maps from images. The folder structure should be <UniSIN_submission>: 0000.npy, 0001.npy .... 0999.npy. Directly save depth maps (in meters) into .npy files and compress the root folder into UniSIN_submission.zip.
Email choyingw@usc.edu we will run the error calculation on the server.

The website template was borrowed from Michaël Gharbi