diff --git a/README.md b/README.md index 60ec8729..0b92c94e 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,14 @@ Dino -## Installation +## Setup -### Minimal +### Quick simulation demo +We prepared a quick-start simulation with a Jackal robot using Docker. Please check the instructions [here](docker/README.md) + +![Overview](./assets/images/sim_demo.jpg) + +### Minimal setup Clone the WVN and our STEGO reimplementation. ```shell mkdir ~/git && cd ~/git @@ -73,8 +78,8 @@ pip3 install -e ./self_supervised_segmentation ┣ πŸ“‚wild_visual_navigation_msgs # ROS1 message definitions ┣ πŸ“‚wild_visual_navigation_ros # ROS1 nodes for running WVN β”— πŸ“‚scripts - β”— πŸ“œ wvn_feature_extractor_node.py - β”— πŸ“œ wvn_learning_node.py + β”— πŸ“œ wvn_feature_extractor_node.py # Main process for feature extraction and inference + β”— πŸ“œ wvn_learning_node.py # Main process that generates supervision signals and the online training loop β”— πŸ“œ quick_start.py # Inferencing demo_data from pre-trained checkpoints ``` ### Features @@ -87,7 +92,7 @@ pip3 install -e ./self_supervised_segmentation Dino -## Experiments +## Demos ### Inference of pre-trained model @@ -105,11 +110,11 @@ python3 quick_start.py ``` ### Online adaptation [Simulation] -Instructions can be found within [wild_visual_navigation_jackal/README.md](wild_visual_navigation_jackal/README.md). +We prepared a Docker image to run a simulation with a Jackal robot, check the instructions [here](docker/README.md). ### Online adaptation [Rosbag] -#### Download Rosbags: -To quickly test out online training and adaption we provide some example rosbags ( [GDrive](https://drive.google.com/drive/folders/1Rf2TRPT6auFxOpnV9-ZfVMjmsvdsrSD3?usp=sharing) ), collected with our ANYmal D robot. +#### Download rosbags: +To quickly test out online training and adaption we provide some example rosbags ([GDrive](https://drive.google.com/drive/folders/1Rf2TRPT6auFxOpnV9-ZfVMjmsvdsrSD3?usp=sharing)), collected with our ANYmal D robot. #### Example Result:
@@ -121,7 +126,7 @@ To quickly test out online training and adaption we provide some example rosbags
-#### ROS-Setup: +#### ROS Setup: ```shell # Create new catkin workspace source /opt/ros/noetic/setup.bash @@ -150,7 +155,7 @@ source /opt/ros/noetic/setup.bash source ~/catkin_ws/devel/setup.bash ``` -After successfully building the ros workspace, you can run the entire pipeline by either using the launch file or by running the nodes individually. +After successfully building the ROS workspace, you can run the entire pipeline by either using the launch file or by running the nodes individually. Open multiple terminals and run the following commands: - Run wild_visual_navigation @@ -184,13 +189,15 @@ python wild_visual_navigation_ros/scripts/wvn_learning_node.py - The general configuration files can be found under: `wild_visual_navigation/cfg/experiment_params.py` - This configuration is used in the `offline-model-training` and in the `online-ros` mode. - When running the `online-ros` mode, additional configurations for the individual nodes are defined in `wild_visual_navigation/cfg/ros_params.py`. -- These configuration file is filled based on the rosparameter-server during runtime. +- These configuration file is filled based on the ROS parameter server during runtime. - The default values for this configuration can be found under `wild_visual_navigation/wild_visual_navigation_ros/config/wild_visual_navigation`. Dino ## Development +
+Here we provide additional details if you want to contribute. ### Install pre-commit ```shell @@ -219,7 +226,7 @@ pytest ### Open-Sourcing Generating headers ```shell -pip3 install adheader +pip3 install addheader # If your are using zsh otherwise remove \ addheader wild_visual_navigation -t header.txt -p \*.py --sep-len 79 --comment='#' --sep=' ' @@ -238,6 +245,7 @@ rosrun procman_ros sheriff -l ~/git/wild_visual_navigation/wild_visual_navigatio ```shell rosbag_play --tf --sem --flp --wvn mission/*.bag ``` +
Dino diff --git a/assets/images/sim_demo.jpg b/assets/images/sim_demo.jpg new file mode 100644 index 00000000..efca1f58 Binary files /dev/null and b/assets/images/sim_demo.jpg differ diff --git a/docker/Dockerfile b/docker/Dockerfile index a82fb53b..9ad0847c 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -123,4 +123,6 @@ RUN mkdir -p /root/catkin_ws/src \ RUN echo "source /root/catkin_ws/devel/setup.bash" >> ~/.bashrc RUN echo "source /root/env/bin/activate" >> ~/.bashrc + +COPY docker/first_run.sh first_run.sh ENV DEBIAN_FRONTEND=dialog \ No newline at end of file diff --git a/docker/README.md b/docker/README.md index 1ed75a0d..82fd8739 100644 --- a/docker/README.md +++ b/docker/README.md @@ -13,60 +13,64 @@ To build the container: docker compose -f docker-compose-gui-nvidia.yaml build ``` -## Run the container +## Run the simulation environment in the container -To run the container (terminal-based): +Start the container in detached mode: ```sh docker compose -f docker-compose-gui-nvidia.yaml up -d ``` -To launch bash on the container: +Launch a first bash terminal in the container to start the simulation environment: ```sh -docker compose exec wvn bash +docker compose -f docker-compose-gui-nvidia.yaml exec wvn_nvidia /bin/bash ``` -## Stop the container +Once in the container, run the `first_run` script to install the WVN package that is mounted automatically when the container starts: +```sh +source first_run.sh +``` -To stop the container: +Launch the Gazebo simulation environment and an RViz window as the main interface. ```sh -docker compose -f docker-compose.yaml stop +roslaunch wild_visual_navigation_jackal sim.launch ``` +If this doesn't open any window, pelase check the troubleshooting section below. -## Running Wild Visual Navigation -You can either run the following commands in 4 terminals that initialize a bash terminal in the container, or you can use VS Code with the Docker extension to instantiate terminal in the container directly. - -### Launch Jackal sim +Open a new terminal to launch WVN in the container ```sh -roslaunch wild_visual_navigation_jackal sim.launch +docker compose -f docker-compose-gui-nvidia.yaml exec wvn_nvidia /bin/bash ``` -### Launch WVN - +And then, once in the container: ```sh roslaunch wild_visual_navigation_jackal wild_visual_navigation.launch ``` -### Launch Teleop node +You can drive the Jackal robot by sending 2D Nav goals using RViz. We implemented a simple [carrot follower](../wild_visual_navigation_jackal/scripts/carrot_follower.py) that was tuned for the demo (not actually used in real experiments) -```sh -roslaunch wild_visual_navigation_jackal teleop.launch -``` -### Launch RViz window +## Stop the example + +Kill all the terminal as usual (Ctrl + D). Then, stop the container using: ```sh -roslaunch wild_visual_navigation_jackal view.launch +docker compose -f docker-compose-gui-nvidia.yaml down ``` ## Troubleshooting -If the GUI doesn't work, you might need to allow the X Server to connect before running the container: +If the GUI doesn't work, you'll see an error like: + +> No protocol specified +> qt.qpa.xcb: could not connect to display :1 + +To fix it, you might need to allow the X Server to connect before running the container: ```sh xhost +Local:* diff --git a/docker/docker-compose-gui-nvidia.yaml b/docker/docker-compose-gui-nvidia.yaml index 4d2fadc8..16708ac1 100644 --- a/docker/docker-compose-gui-nvidia.yaml +++ b/docker/docker-compose-gui-nvidia.yaml @@ -1,4 +1,3 @@ -version: "3.9" services: wvn_nvidia: extends: diff --git a/docker/docker-compose-gui.yaml b/docker/docker-compose-gui.yaml index 0cc35c5f..ee1f9c4c 100644 --- a/docker/docker-compose-gui.yaml +++ b/docker/docker-compose-gui.yaml @@ -1,4 +1,3 @@ -version: "3.9" services: wvn: extends: diff --git a/docker/docker-compose-nvidia.yaml b/docker/docker-compose-nvidia.yaml index 1bd55c43..83c1b00f 100644 --- a/docker/docker-compose-nvidia.yaml +++ b/docker/docker-compose-nvidia.yaml @@ -1,4 +1,3 @@ -version: "3.9" services: wvn_nvidia: extends: diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index 56f48cc2..9a9adbb5 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -1,4 +1,3 @@ -version: "3.9" services: wvn: build: @@ -13,10 +12,4 @@ services: - ../../self_supervised_segmentation:/root/catkin_ws/src/self_supervised_segmentation environment: - ROS_IP=127.0.0.1 - command: "/bin/bash" - # command: > - # /bin/bash -c "pip3 install -e /root/catkin_ws/src/wild_visual_navigation - # && pip3 install -e /root/catkin_ws/src/self_supervised_segmentation - # && cd /root/catkin_ws && catkin build - # && cd /root - # && source /root/catkin_ws/devel/setup.bash" \ No newline at end of file + command: "/bin/bash" \ No newline at end of file diff --git a/docker/first_run.sh b/docker/first_run.sh new file mode 100755 index 00000000..eeba87d8 --- /dev/null +++ b/docker/first_run.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +echo "pip3 install -e /root/catkin_ws/src/self_supervised_segmentation ..." +pip3 install -e /root/catkin_ws/src/self_supervised_segmentation > /dev/null + +echo "pip3 install -e /root/catkin_ws/src/wild_visual_navigation ..." +pip3 install -e /root/catkin_ws/src/wild_visual_navigation > /dev/null + +echo "catkin build ..." +catkin build > /dev/null + +echo "source devel/setup.bash ..." +source devel/setup.bash > /dev/null + +echo "Setup ready!" \ No newline at end of file diff --git a/wild_visual_navigation_jackal/README.md b/wild_visual_navigation_jackal/README.md index 50a45ad5..3c2c6d75 100644 --- a/wild_visual_navigation_jackal/README.md +++ b/wild_visual_navigation_jackal/README.md @@ -15,9 +15,7 @@ sudo apt-get update sudo apt update sudo apt install -y \ ros-noetic-jackal-simulator \ - ros-noetic-jackal-desktop \ - ros-noetic-teleop-twist-keyboard \ - ros-noetic-rqt-robot-steering \ + ros-noetic-jackal-desktop ``` ## Running diff --git a/wild_visual_navigation_jackal/launch/sim.launch b/wild_visual_navigation_jackal/launch/sim.launch index 9e95fcfc..98926656 100644 --- a/wild_visual_navigation_jackal/launch/sim.launch +++ b/wild_visual_navigation_jackal/launch/sim.launch @@ -34,4 +34,7 @@ + + + diff --git a/wild_visual_navigation_jackal/launch/teleop.launch b/wild_visual_navigation_jackal/launch/teleop.launch deleted file mode 100644 index 3ffe082f..00000000 --- a/wild_visual_navigation_jackal/launch/teleop.launch +++ /dev/null @@ -1,7 +0,0 @@ - - - - - - - \ No newline at end of file diff --git a/wild_visual_navigation_jackal/scripts/carrot_follower.py b/wild_visual_navigation_jackal/scripts/carrot_follower.py index 91e23b27..fabaa82b 100644 --- a/wild_visual_navigation_jackal/scripts/carrot_follower.py +++ b/wild_visual_navigation_jackal/scripts/carrot_follower.py @@ -23,7 +23,7 @@ MAX_LINEAR_VEL = 0.5 MAX_ANGULAR_VEL = 0.7 GAIN_LINEAR = 1.0 -GAIN_ANGULAR = 1.0 +GAIN_ANGULAR = 1.5 DIST_THR = 0.1 # 10 cms diff --git a/wild_visual_navigation_jackal/scripts/gazebo_world_publisher.py b/wild_visual_navigation_jackal/scripts/gazebo_world_publisher.py index 00038861..570d53a8 100644 --- a/wild_visual_navigation_jackal/scripts/gazebo_world_publisher.py +++ b/wild_visual_navigation_jackal/scripts/gazebo_world_publisher.py @@ -20,6 +20,9 @@ import tf.transformations as tr +last_stamp = None + + def msg_to_se3(msg): """Conversion from geometric ROS messages into SE(3) Based on Jarvis Schultz's: https://answers.ros.org/question/332407/transformstamped-to-transformation-matrix-python/ @@ -54,7 +57,11 @@ def msg_to_se3(msg): def gazebo_callback(msg): + global last_stamp stamp = rospy.Time.now() + if stamp == last_stamp: + return + T_world_base = msg_to_se3(msg.pose[1]) # this is the base_link pose in world frame (from gazebo) T_base_world = np.linalg.inv(T_world_base) @@ -74,10 +81,12 @@ def gazebo_callback(msg): br.sendTransform(t) pub.publish(marker) + last_stamp = stamp if __name__ == "__main__": rospy.init_node("gazebo_world_publisher") + last_stamp = rospy.Time.now() # Default variables rospack = rospkg.RosPack()