Ground Truth Renderer

Features

• Loads and renders different subsets of point clouds
• View the point cloud in different modes
  • Splats: high quality splat rendering of the whole point cloud
  • Points: high quality point rendering of the whole point cloud
  • Neural Network: renders neural network output and loss channel comparison
• Compare to a sparse subset of the point cloud with a different sampling rate
• HDF5 dataset generation containing color, normal and depth images of the point cloud
• Blending the colors of overlapping splats
• Phong Lighting

Neural Network View Mode:

• The neural network will be loaded from a NeuralNetwork.pt file
• A description of the input/output channels of the network must be given in a NeuralNetworkDescription.txt file
• Each entry consists of:
  • String: Name of the channel (render mode)
  • Int: Dimension of channel
  • String: Identifying if the channel is input (inp) or output (tar)
  • String: Transformation keywords e.g. normalization
  • Int: Offset of this channel from the start channel
• Example for a simple description file:

[['PointsSparseColor', 3, 'inp', 'normalize', 0], ['SplatsColor', 3, 'tar', 'normalize', 0]]

• Loss between two channels (self and target) that are set in the settings file is computed
• Screen ratio between these channel renderings can be controlled with UP/DOWN

HDF5 Dataset Generation

• There are two modes for the dataset generation with parameters in the settings file
  • Waypoint dataset [F7]:
    • Interpolates the camera between the user set waypoints
    • Press [INSERT]/[DELETE] to add/remove a waypoint for the current camera perspective
    • Holding [SPACE] shows a preview of the interpolation
    • waypointStepSize defines the interpolation value between two waypoints
  • Sphere dataset [F8]:
    • Sweeps the camera along a sphere around the point cloud
    • sphereStepSize influences the amount of viewing angles (0.2 results in ~1000 angles)
    • theta and phi min/max values define the subset of the sphere to be sweeped
    • Move the camera to the desired distance from the center of the point cloud
• Restart the engine with the ground truth renderer
• Select the density of the sparse point cloud
• Configure the rendering parameters for the splat and sparse splat view modes
• Start the generation process with [F7]/[F8]
• The file will be stored in the HDF5 directory