NVIDIA Isaac ROS EA2 Release

release-ea2: 330b740679886c968c63e28f94eb5c30908004ba

README.md

@@ -1,2 +1,123 @@
-# Isaac ROS Camera
+# Isaac ROS Argus Camera
 
+<div align="center"><img src="resources/100_library_left.JPG" width="350" title="HAWK Stereo Left"/><img src="resources/100_library_right.JPG" width="350" title="HAWK Stereo Right"/></div>
+
+This repository provides monocular and stereo nodes that enable ROS developers to use cameras connected to Jetson platforms over a CSI interface. The nodes internally use libargus, which is an API for acquiring images and associated metadata from camera devices.
+
+[Libargus API reference](https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html)
+
+This package is compatible with ROS2 Foxy and has been tested on the Jetson platfrom with off-the-shelf cameras from NVIDIA partners(see the **Reference Camera** section for more details). **Note**: x86_64 is not supported.
+
+## System Requirements
+The CSI camera device needs to be connected and running and to present the video device node (e.g. /dev/video0).
+
+### Jetson
+- [Jetson AGX Xavier and NX Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/)
+- [JetPack 4.6](https://developer.nvidia.com/embedded/jetpack)
+
+**Note:** For best performance on Jetson, ensure that power settings are configured appropriately ([Power Management for Jetson](https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/power_management_jetson_xavier.html#wwpID0EUHA)).
+
+### Docker
+Precompiled ROS2 Foxy packages are not available for JetPack 4.6 (based on Ubuntu 18.04 Bionic). You can either manually compile ROS2 Foxy and required dependent packages from source or use the Isaac ROS development Docker image from [Isaac ROS Common](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common).
+
+You must first install the [Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) to make use of the Docker container development/runtime environment.
+
+Configure `nvidia-container-runtime` as the default runtime for Docker by editing `/etc/docker/daemon.json` to include the following:
+```
+    "runtimes": {
+        "nvidia": {
+            "path": "nvidia-container-runtime",
+            "runtimeArgs": []
+        }
+    },
+    "default-runtime": "nvidia"
+```
+and then restarting Docker: `sudo systemctl daemon-reload && sudo systemctl restart docker`
+
+Run the following script in `isaac_ros_common` to build the image and launch the container:
+
+`$ scripts/run_dev.sh <optional path>`
+
+You can either provide an optional path to mirror in your host ROS workspace with Isaac ROS packages, which will be made available in the container as `/workspaces/isaac_ros-dev`, or you can setup a new workspace in the container.
+
+### Reference Camera
+The [Leopard Imaging](https://www.leopardimaging.com/product/) NVIDIA camera partner provides the below camera modules, which are compliant with the `isaac_ros_argus_camera` packages.
+
+
+| Product Name | Type                | Resolution  |
+| ------------ | ------------------- | ----------- |
+| HAWK         | Stereo Camera       | 1920 x 1200 |
+| OWL          | Fisheye Camera      | 1920 x 1200 |
+| IMX477       | 4K Monocular Camera | 4056 x 3040 |
+
+## Quickstart
+1. Create a ROS2 workspace if one is not already prepared:  
+`mkdir -p your_ws/src`  
+**Note:** The workspace can have any name; this guide assumes you name it `your_ws`.
+
+2. Clone the Isaac ROS Argus Camera repository to `your_ws/src/isaac_ros_argus_camera`. Check that you have [Git LFS](https://git-lfs.github.com/) installed before cloning to pull down all large files.  
+`sudo apt-get install git-lfs`  
+`cd your_ws/src && git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_argus_camera`
+
+3. Build and source the workspace:  
+`cd your_ws && colcon build --symlink-install && source install/setup.bash`
+
+4. (Optional) Run tests to verify complete and correct installation:  
+`colcon test`
+
+5. Launch the node:  
+`ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="mono8"`
+
+
+## Package Reference
+### isaac_ros_argus_camera_mono
+
+`ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=<device_index> -p sensor:=<sensor_index> -p output_encoding:=<encoding_string>`
+
+| ROS Argument      | Usage                                                                  |
+| ----------------- | ---------------------------------------------------------------------- |
+| `device_index`    | The video node index (e.g. `/dev/video0`)                              |
+| `sensor_index`    | The sensor mode supported by the camera sensor and driver              |
+| `output_encoding` | The output image format.  `mono8 ` and `rgb8 ` are currently supported |
+
+### isaac_ros_argus_camera_stereo
+
+   `ros2 run isaac_ros_argus_camera_stereo isaac_ros_argus_camera_stereo --ros-args -p device:=<device_index> -p sensor:=<sensor_index> -p output_encoding:=<encoding_string>`
+
+| ROS Argument      | Usage                                                                 |
+| ----------------- | --------------------------------------------------------------------- |
+| `device_index`    | The first video node index (e.g. `/dev/video0`)                       |
+| `sensor_index`    | The sensor mode supported in the camera sensor and driver             |
+| `output_encoding` | The output image format. `mono8 ` and `rgb8 ` are currently supported |
+
+**Note**: To run the stereo node, two video nodes should present for left and right sensors, respectively (e.g. `/dev/video0` and `/dev/video1`).
+
+Examples:  
+`ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="rgb8"`
+
+`ros2 run isaac_ros_argus_camera_stereo isaac_ros_argus_camera_stereo --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="mono8"`
+
+To view the output images:  
+
+ `ros2 run image_view image_saver --ros-args -r image:=/image_raw -p filename_format:="right_image.jpg" `
+
+ `ros2 run image_view image_saver --ros-args -r image:=/stereo/left/image_raw -p filename_format:="left_image.jpg" `
+
+### CameraInfo Message
+Argus nodes use the Argus Ext API to retrieve calibration parameters from the camera through the Linux device driver and convert it to [`CameraInfo`](http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/CameraInfo.html) messages.  
+Refer to [link](https://docs.nvidia.com/jetson/l4t-multimedia/classArgus_1_1Ext_1_1ISyncSensorCalibrationData.html) for the data structure of the calibration parameters. 
+
+**Note**: Each camera module should have stored the calibration parameters in the internal memory like EEPROM and the device driver supports the API to extract it. Contact your camera vendor to get the driver that supports it.
+
+## Troubleshooting
+### Argus camera nodes could stop publishing images sometimes when used with 4K high-resolution cameras in certain graph configurations on FastRTPS
+With downstream subscribers that are not able to process images at the camera frame rate, we have observed instances where the Argus camera node suddenly stops publishing new images after about 15-20 minutes.
+
+#### Solution
+We are continuing to collect data and diagnose the issue but changing the QoS settings to "Best Effort" and using a smaller frame size seem to alleviate the condition.
+
+# Updates
+
+| Date | Changes |
+| -----| ------- |
+| 2021-10-20 | Initial release  |

giistr-cla.md

@@ -0,0 +1,58 @@
+## Individual Contributor License Agreement (CLA)
+
+**Thank you for submitting your contributions to this project.**
+
+By signing this CLA, you agree that the following terms apply to all of your past, present and future contributions
+to the project.
+
+### License.
+
+You hereby represent that all present, past and future contributions are governed by the
+[MIT License](https://opensource.org/licenses/MIT)
+copyright statement.
+
+This entails that to the extent possible under law, you transfer all copyright and related or neighboring rights
+of the code or documents you contribute to the project itself or its maintainers.
+Furthermore you also represent that you have the authority to perform the above waiver
+with respect to the entirety of you contributions.
+
+### Moral Rights.
+
+To the fullest extent permitted under applicable law, you hereby waive, and agree not to
+assert, all of your “moral rights” in or relating to your contributions for the benefit of the project.
+
+### Third Party Content.
+
+If your Contribution includes or is based on any source code, object code, bug fixes, configuration changes, tools,
+specifications, documentation, data, materials, feedback, information or other works of authorship that were not
+authored by you (“Third Party Content”) or if you are aware of any third party intellectual property or proprietary
+rights associated with your Contribution (“Third Party Rights”),
+then you agree to include with the submission of your Contribution full details respecting such Third Party
+Content and Third Party Rights, including, without limitation, identification of which aspects of your
+Contribution contain Third Party Content or are associated with Third Party Rights, the owner/author of the
+Third Party Content and Third Party Rights, where you obtained the Third Party Content, and any applicable
+third party license terms or restrictions respecting the Third Party Content and Third Party Rights. For greater
+certainty, the foregoing obligations respecting the identification of Third Party Content and Third Party Rights
+do not apply to any portion of a Project that is incorporated into your Contribution to that same Project.
+
+### Representations.
+
+You represent that, other than the Third Party Content and Third Party Rights identified by
+you in accordance with this Agreement, you are the sole author of your Contributions and are legally entitled
+to grant the foregoing licenses and waivers in respect of your Contributions. If your Contributions were
+created in the course of your employment with your past or present employer(s), you represent that such
+employer(s) has authorized you to make your Contributions on behalf of such employer(s) or such employer
+(s) has waived all of their right, title or interest in or to your Contributions.
+
+### Disclaimer.
+
+To the fullest extent permitted under applicable law, your Contributions are provided on an "as is"
+basis, without any warranties or conditions, express or implied, including, without limitation, any implied
+warranties or conditions of non-infringement, merchantability or fitness for a particular purpose. You are not
+required to provide support for your Contributions, except to the extent you desire to provide support.
+
+### No Obligation.
+
+You acknowledge that the maintainers of this project are under no obligation to use or incorporate your contributions
+into the project. The decision to use or incorporate your contributions into the project will be made at the
+sole discretion of the maintainers or their authorized delegates.

isaac_ros_argus_camera_mono/CMakeLists.txt

@@ -0,0 +1,92 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.  All rights reserved.
+#
+# NVIDIA CORPORATION and its licensors retain all intellectual property
+# and proprietary rights in and to this software, related documentation
+# and any modifications thereto.  Any use, reproduction, disclosure or
+# distribution of this software and related documentation without an express
+# license agreement from NVIDIA CORPORATION is strictly prohibited.
+
+cmake_minimum_required(VERSION 3.5)
+project(isaac_ros_argus_camera_mono)
+
+# Default to C++17
+if(NOT CMAKE_CXX_STANDARD)
+  set(CMAKE_CXX_STANDARD 17)
+endif()
+
+if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
+  add_compile_options(-Wall -Wextra -Wpedantic)
+endif()
+
+execute_process(
+  COMMAND uname -m
+  COMMAND tr -d '\n'
+  OUTPUT_VARIABLE ARCHITECTURE)
+message(STATUS "Architecture: ${ARCHITECTURE}")
+
+find_package(rclcpp REQUIRED)
+find_package(ament_cmake_auto REQUIRED)
+ament_auto_find_build_dependencies()
+
+# Only build Argus nodes on Jetson platform
+if(${ARCHITECTURE} STREQUAL "aarch64")
+
+  # Find VPI dependency
+  find_package(vpi REQUIRED)
+
+  # monocular camera node
+  ament_auto_add_library(monocular_node SHARED src/argus_camera_mono_node.cpp)
+
+  target_include_directories(
+    monocular_node
+    PRIVATE /usr/lib/aarch64-linux-gnu/tegra
+            /usr/src/jetson_multimedia_api/argus/include
+            /usr/src/jetson_multimedia_api/argus/samples/utils
+            /usr/src/jetson_multimedia_api/include
+            include)
+
+  target_compile_definitions(monocular_node PRIVATE "COMPOSITION_BUILDING_DLL")
+
+  target_link_libraries(
+    monocular_node
+    /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
+    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
+    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
+    /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so
+    vpi)
+
+  target_compile_definitions(monocular_node PRIVATE "COMPOSITION_BUILDING_DLL")
+
+  rclcpp_components_register_nodes(monocular_node "isaac_ros::argus::MonocularNode")
+  set(node_plugins "${node_plugins}isaac_ros::argus::MonocularNode;$<TARGET_FILE:monocular_node>\n")
+
+  # isaac_ros_argus executable
+  ament_auto_add_executable(${PROJECT_NAME} src/argus_camera_mono_main.cpp)
+
+  target_include_directories(${PROJECT_NAME} PUBLIC include)
+
+  target_link_libraries(${PROJECT_NAME} monocular_node
+                        ament_index_cpp::ament_index_cpp vpi)
+
+  install(
+    TARGETS ${PROJECT_NAME}
+    ARCHIVE DESTINATION lib
+    LIBRARY DESTINATION lib
+    RUNTIME DESTINATION bin)
+
+  install(DIRECTORY launch DESTINATION share/${PROJECT_NAME})
+endif()
+
+if(BUILD_TESTING)
+  find_package(ament_lint_auto REQUIRED)
+
+  # Ignore copyright notices since we use custom JetPack EULA
+  set(ament_cmake_copyright_FOUND TRUE)
+
+  ament_lint_auto_find_test_dependencies()
+
+  find_package(launch_testing_ament_cmake REQUIRED)
+
+endif()
+
+ament_auto_package(INSTALL_TO_SHARE launch)

isaac_ros_argus_camera_mono/include/isaac_ros_argus_camera_mono/argus_camera_mono_node.hpp

@@ -0,0 +1,52 @@
+/**
+ * Copyright (c) 2021, NVIDIA CORPORATION.  All rights reserved.
+ *
+ * NVIDIA CORPORATION and its licensors retain all intellectual property
+ * and proprietary rights in and to this software, related documentation
+ * and any modifications thereto.  Any use, reproduction, disclosure or
+ * distribution of this software and related documentation without an express
+ * license agreement from NVIDIA CORPORATION is strictly prohibited.
+ */
+
+#ifndef ISAAC_ROS_ARGUS_CAMERA_MONO__ARGUS_CAMERA_MONO_NODE_HPP_
+#define ISAAC_ROS_ARGUS_CAMERA_MONO__ARGUS_CAMERA_MONO_NODE_HPP_
+
+#include <string>
+
+#include "image_transport/image_transport.hpp"
+#include "rclcpp/rclcpp.hpp"
+
+#define POLYNOMIAL_DISTORTION_COEFFICIENT_COUNT 8
+#define POLYNOMIAL_RADIAL_DISTORTION_COEFFICIENT_COUNT 6
+
+namespace isaac_ros
+{
+namespace argus
+{
+
+class MonocularNode : public rclcpp::Node
+{
+public:
+  explicit MonocularNode(const rclcpp::NodeOptions &);
+
+  virtual ~MonocularNode();
+
+private:
+  // Publisher used for intra process comm
+  rclcpp::Publisher<sensor_msgs::msg::Image>::SharedPtr image_pub_;
+  rclcpp::Publisher<sensor_msgs::msg::CameraInfo>::SharedPtr camera_pub_;
+
+  // Publisher used for inter process comm
+  image_transport::CameraPublisher publisher_;
+
+  sensor_msgs::msg::CameraInfo::SharedPtr camerainfo_;
+  sensor_msgs::msg::Image::SharedPtr image_;
+
+  uint32_t width_;
+  uint32_t height_;
+};
+
+}  // namespace argus
+}  // namespace isaac_ros
+
+#endif  // ISAAC_ROS_ARGUS_CAMERA_MONO__ARGUS_CAMERA_MONO_NODE_HPP_

isaac_ros_argus_camera_mono/launch/isaac_ros_argus_camera_mono_launch.py

@@ -0,0 +1,34 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.  All rights reserved.
+#
+# NVIDIA CORPORATION and its licensors retain all intellectual property
+# and proprietary rights in and to this software, related documentation
+# and any modifications thereto.  Any use, reproduction, disclosure or
+# distribution of this software and related documentation without an express
+# license agreement from NVIDIA CORPORATION is strictly prohibited
+
+import launch
+from launch_ros.actions import Node
+
+
+def generate_launch_description():
+    return launch.LaunchDescription([
+        Node(
+            package='isaac_ros_argus_camera_mono',
+            executable='isaac_ros_argus_camera_mono',
+            parameters=[{
+                'sensor': 0,
+                'device': 0,
+                'output_encoding': 'rgb8'
+            }]
+        ),
+        Node(
+            package='image_view',
+            executable='image_saver',
+            remappings=[
+                ('/image', '/image_raw')
+            ],
+            parameters=[{
+                'filename_format': 'image.jpg'
+            }]
+        )
+    ])