Ellipsoids shape for volumetric primitives #1464
Conversation
Speierers
force-pushed
the
ellipsoids_release
branch
from
4d43420 to
062a057
Compare
There was a problem hiding this comment.
Hi Sébastien,
thank you for this PR, it looks extremely useful. I left some feedback.
High level questions: can you explain why backface culling is such an important feature when rendering gaussians, and why that excludes LLVM from using the mesh version of the plugin? Could there not be some workaround?
I noticed that there are various TODOs in the code related to backface culling.
| * Taken from "Precision Improvements for Ray/Sphere Intersection", Ray Tracing Gems 2 | ||
| * \return \c true if a solution could be found | ||
| */ | ||
| template <typename Value> |
There was a problem hiding this comment.
What is the role of this function compared to solve_quadratic above? That one also includes improvements to avoid floating point cancellation errors when resolving the two roots.
| uint32_t ray_flags, Mask active) { | ||
| dr::masked(t, !active) = dr::Infinity<Float>; | ||
| active &= is_valid(); | ||
|
|
||
| dr::masked(shape, !active) = nullptr; | ||
| dr::masked(instance, !active) = nullptr; | ||
|
|
||
| prim_index = pi.prim_index; |
There was a problem hiding this comment.
Hum, I would prefer not to do this assignment here as opposed to in the vcall. Is it possible that your mesh ellipsoids could simply divide out the face count when using this flag?
| return result; | ||
| } | ||
|
|
||
| DEVICE Vector3f prod_inv(const Vector3f &v) const { |
There was a problem hiding this comment.
Is this supposed to be a transpose product? I am not sure where _inv comes from.
| // per-mesh basis in the future, we will need to create another IAS specifically | ||
| // for meshes that request backface culling (e.g. could add Shape->enable_backface_culling()) | ||
| uint32_t flags = disable_face_culling ? | ||
| OPTIX_INSTANCE_FLAG_DISABLE_TRIANGLE_FACE_CULLING : |
There was a problem hiding this comment.
The OPTIX_INSTANCE_FLAG_DISABLE_TRANSFORM (a performance optimization) was lost with this change.
| @@ -86,10 +89,7 @@ class MI_EXPORT_LIB ShapeGroup : public Shape<Float, Spectrum> { | |||
| private: | |||
| ScalarBoundingBox3f m_bbox; | |||
| std::vector<ref<Base>> m_shapes; | |||
|
|
|||
| #if defined(MI_ENABLE_LLVM) || defined(MI_ENABLE_CUDA) | |||
| DynamicBuffer<UInt32> m_shapes_registry_ids; | |||
There was a problem hiding this comment.
Not sure I understand this change. There is no registry in scalar mode.
| - Specifies whether the SH coefficients of the primitives are defined in | ||
| sRGB color space. (Default: True) | ||
|
|
||
| This plugin implements a simple radiance field integrator for ellipsoids shapes. |
There was a problem hiding this comment.
The following would be good to point out here:
- This is not just an emissive integrator, it does handle inter-reflection.
- But the volume is only emissive.
There was a problem hiding this comment.
Perhaps also a hint of what is missing compared to the fancy version in your separate repository.
There was a problem hiding this comment.
Can you explain why one would store SH coefficients in sRGB vs RGB? Is this just to deal with external datasets? I am imagining that it wouldn't matter much in an actual optimization.
By the way -- in this case, wouldn't it be better to make the SRGB flag a shape feature, rather than a global integrator one?
| @@ -686,12 +709,17 @@ Scene<Float, Spectrum>::ray_test_gpu(const Ray3f &ray, Mask active) const { | |||
| const OptixConfig &config = optix_configs[s.config_index]; | |||
|
|
|||
| UInt32 ray_mask(255), | |||
| ray_flags(OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT | | |||
| OPTIX_RAY_FLAG_DISABLE_CLOSESTHIT), | |||
| ray_flags(OPTIX_RAY_FLAG_DISABLE_CLOSESTHIT), | |||
There was a problem hiding this comment.
This loses the OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT optimization, which seems problematic.
| // Enforce backface culling, which is only enabled on the EllipsoidsMesh IAS | ||
| // TODO this could be enabled/disabled using a flag argument to this method. | ||
| // TODO currently the logic doesn't work for a single IAS, hence the check below | ||
| if (!config.has_only_meshes) |
There was a problem hiding this comment.
The condition here is a different one when comparing to earlier code. Also, there are some TODOs.
|
|
||
| .. _shape-EllipsoidsMesh: | ||
|
|
||
| Mesh ellipsoids (:monosp:`EllipsoidsMesh`) |
There was a problem hiding this comment.
I think that "mesh ellipsoids" is a nice name to refer to these, but EllipsoidsMesh sounds kind of odd.
Can you please do a global rename of
- "EllipsoidsMesh" -> "MeshEllipsoids"
- "ellipsoids_mesh" -> "mesh_ellipsoids"?
- "ellipsoidsmesh" -> "mesh_ellipsoids"
| @@ -53,6 +53,9 @@ enum class RayFlags : uint32_t { | |||
| /// Derivatives of the SurfaceInteraction fields ignore shape's motion | |||
| DetachShape = 0x100, | |||
|
|
|||
| /// Enable backface culling | |||
There was a problem hiding this comment.
This feature seems to have some caveats that should be better explained. What exactly does it do, does it work for all shapes, which backends are supported, what about scalar mode, etc.
|
Is |
This PR adds support for two new shapes in Mitsuba that can be used to efficiently render volumetric primitives (e.g. 3D Gaussians):
ellipsoidsandellipsoidsmesh. Those shapes defines a point cloud of anisotropic ellipsoid primitivesusing specified centers, scales, and quaternions. The former employs a closed-form ray-intersection formula with backface culling while the later uses a mesh-based representation to leverage hardware acceleration ray-tracing.
This PR also provides
volprim_rf_basic, an example integrator that renders the set of ellipsoids as a radiance field, similar to 3D Gaussian splatting. The more advanced integrators described in the Don't Splat Your Gaussian paper as well as example scripts will be open-sourced in a seperate repository.Other contributions included in this PR:
Shape::eval_attribute_Xwhich returns a dynamic array of attributesParamsFlag::ReadOnlymitsuba.testing.RenderingRegressionTestfor easily create regression tests similar totest_render.pyellipsoidsmeshshape.