diff --git a/README.md b/README.md index f737951454d0c818e383c82468f9dce809e0092c..7bf53453cf30cca8cb9feddb4167ebc3fa24e47e 100644 --- a/README.md +++ b/README.md @@ -206,7 +206,7 @@ __Q:__ The NeRF reconstruction of my custom dataset looks bad; what can I do? __A:__ There could be multiple issues: - COLMAP might have been unable to reconstruct camera poses. - There might have been movement or blur during capture. Don't treat capture as an artistic task; treat it as photogrammetry. You want _\*as little blur as possible\*_ in your dataset (motion, defocus, or otherwise) and all objects must be _\*static\*_ during the entire capture. Bonus points if you are using a wide-angle lens (iPhone wide angle works well), because it covers more space than narrow lenses. -- The dataset parameters (in particular `aabb_scale`) might have been tuned suboptimally. We recommend starting with `aabb_scale=16` and then decreasing it to `8`, `4`, `2`, and `1` until you get optimal quality. +- The dataset parameters (in particular `aabb_scale`) might have been tuned suboptimally. We recommend starting with `aabb_scale=16` and then increasing or decreasing it by factors of two until you get optimal quality. - Carefully read [our NeRF training & dataset tips](https://github.com/NVlabs/instant-ngp/blob/master/docs/nerf_dataset_tips.md). ## diff --git a/docs/nerf_dataset_tips.md b/docs/nerf_dataset_tips.md index 5e123a63bcc6a45c5e1e670b90fc9f9270b2daae..23f11d4eac42dec7f5fa1a48175cb438c7544933 100644 --- a/docs/nerf_dataset_tips.md +++ b/docs/nerf_dataset_tips.md @@ -26,7 +26,7 @@ It is worth checking the alignment of your cameras to this bounding box by check <img src="assets/nerfbox.jpg" width="100%"/> -For natural scenes where there is a background visible outside the unit cube, it is necessary to set the parameter `aabb_scale` in the `transforms.json` file to a power of 2 integer up to 16 (that is 1, 2, 4, 8, or 16), at the outermost scope (same nesting as e.g. the existing `camera_angle_x` parameter). See [data/nerf/fox/transforms.json](/data/nerf/fox/transforms.json) for an example. +For natural scenes where there is a background visible outside the unit cube, it is necessary to set the parameter `aabb_scale` in the `transforms.json` file to a power of 2 integer up to 128 (that is 1, 2, 4, 8, ..., 128), at the outermost scope (same nesting as e.g. the existing `camera_angle_x` parameter). See [data/nerf/fox/transforms.json](/data/nerf/fox/transforms.json) for an example. The effect can be seen in the image below: @@ -73,7 +73,7 @@ The script will run FFmpeg and/or COLMAP as needed, followed by a conversion ste By default, the script invokes colmap with the "sequential matcher", which is suitable for images taken from a smoothly changing camera path, as in a video. The exhaustive matcher is more appropriate if the images are in no particular order, as shown in the image example above. For more options, you can run the script with `--help`. For more advanced uses of COLMAP or for challenging scenes, please see the [COLMAP documentation](https://colmap.github.io/cli.html); you may need to modify the [scripts/colmap2nerf.py](/scripts/colmap2nerf.py) script itself. -The `aabb_scale` parameter is the most important `instant-ngp` specific parameter. It specifies the extent of the scene, defaulting to 1; that is, the scene is scaled such that the camera positions are at an average distance of 1 unit from the origin. For small synthetic scenes such as the original NeRF dataset, the default `aabb_scale` of 1 is ideal and leads to fastest training. The NeRF model makes the assumption that the training images can entirely be explained by a scene contained within this bounding box. However, for natural scenes where there is a background that extends beyond this bounding box, the NeRF model will struggle and may hallucinate "floaters" at the boundaries of the box. By setting `aabb_scale` to a larger power of 2 (up to a maximum of 16), the NeRF model will extend rays to a much larger bounding box. Note that this can impact training speed slightly. If in doubt, for natural scenes, start with an `aabb_scale` of 16, and subsequently reduce it if possible. The value can be directly edited in the `transforms.json` output file, without re-running the [scripts/colmap2nerf.py](/scripts/colmap2nerf.py) script. +The `aabb_scale` parameter is the most important `instant-ngp` specific parameter. It specifies the extent of the scene, defaulting to 1; that is, the scene is scaled such that the camera positions are at an average distance of 1 unit from the origin. For small synthetic scenes such as the original NeRF dataset, the default `aabb_scale` of 1 is ideal and leads to fastest training. The NeRF model makes the assumption that the training images can entirely be explained by a scene contained within this bounding box. However, for natural scenes where there is a background that extends beyond this bounding box, the NeRF model will struggle and may hallucinate "floaters" at the boundaries of the box. By setting `aabb_scale` to a larger power of 2 (up to a maximum of 16), the NeRF model will extend rays to a much larger bounding box. Note that this can impact training speed slightly. If in doubt, for natural scenes, start with an `aabb_scale` of 128, and subsequently reduce it if possible. The value can be directly edited in the `transforms.json` output file, without re-running the [scripts/colmap2nerf.py](/scripts/colmap2nerf.py) script. Assuming success, you can now train your NeRF model as follows, starting in the `instant-ngp` folder: diff --git a/src/testbed_nerf.cu b/src/testbed_nerf.cu index 0faa7767b17b201ee4ec63d777cb4b6e1faeb4f8..73296a3dafb93dc60438381de8302795b0db2dfd 100644 --- a/src/testbed_nerf.cu +++ b/src/testbed_nerf.cu @@ -45,7 +45,7 @@ NGP_NAMESPACE_BEGIN inline constexpr __device__ float NERF_RENDERING_NEAR_DISTANCE() { return 0.05f; } inline constexpr __device__ uint32_t NERF_STEPS() { return 1024; } // finest number of steps per unit length -inline constexpr __device__ uint32_t NERF_CASCADES() { return 5; } +inline constexpr __device__ uint32_t NERF_CASCADES() { return 8; } inline constexpr __device__ float SQRT3() { return 1.73205080757f; } inline constexpr __device__ float STEPSIZE() { return (SQRT3() / NERF_STEPS()); } // for nerf raymarch