Adds a tutorial explaining how to use Environmental Data infrastructure. #1806

arjo129 · 2022-11-22T07:22:04Z

This PR adds a tutorial explaining how to use environmental data infrastructure. It walks users through the steps of defining their own data and loading the data into gazebo.

Signed-off-by: Arjo Chakravarty arjo@openrobotics.org

This PR adds a tutorial explaining how to use environmental data infrastructure. It walks users through the steps of defining their own data and loading the data into gazebo. Signed-off-by: Arjo Chakravarty <arjo@openrobotics.org>

Signed-off-by: Arjo Chakravarty <arjo@openrobotics.org>

codecov · 2022-11-22T08:30:46Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.39%. Comparing base (4fa1f26) to head (bfcca7b).
Report is 325 commits behind head on gz-sim7.

Additional details and impacted files

@@             Coverage Diff             @@
##           gz-sim7    #1806      +/-   ##
===========================================
+ Coverage    64.37%   64.39%   +0.02%     
===========================================
  Files          341      341              
  Lines        27132    27132              
===========================================
+ Hits         17467    17473       +6     
+ Misses        9665     9659       -6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

hidmic

First pass

hidmic · 2022-11-22T13:19:36Z

examples/worlds/environmental_sensor.sdf

-<!-- This example show cases how to load and unload environmental data. -->
+<!-- This example show cases how to use a custom sensor to sample data.
+You should be able to get data echoed on the topic
+"/world/environmental_sensor_test/model/model_with_sensor/link/link/sensor/custom_sensor/environmental_sensor/humidity"


@arjo129 meta: that's a pretty lengthy path. Can it be shortened?

Looks from the tutorial that it can be customized by <topic>, it would be worth adding that to this example file.

tutorials/environmental_data_and_visualization.md

hidmic · 2022-11-22T13:24:26Z

tutorials/environmental_data_and_visualization.md

+
+Currently gazebo is able to load data from CSV files and display them. It is
+assumed the points are sampled at a fixed grid interval. For gaps in between the
+sample points, Gazebo will interpolate the data.


@arjo129 nit: it may worth describing the interpolation techniques that are in use (or at least adding a link to the code).

hidmic · 2022-11-22T13:25:52Z

tutorials/environmental_data_and_visualization.md

+1658923063,94,36.80129505,-121.789972517,0.8
+```
+> :warning: Note that only one Environment Data file per world is allowed. Also,
+Gazebo can only load data that fits into memory.


@arjo129 meta: should we add a feature request to gz-common5 about lazy loading?

hidmic · 2022-11-22T13:26:52Z

tutorials/environmental_data_and_visualization.md

+As mentioned earlier, the data should be a CSV file with grid based spacing.
+The first line of the CSV file should be the heading for each column.
+There should be a column for time (in seconds), the x coordinate, y coordinate
+and z coordinate (in meters). You may have multiple columns of data like so:


@arjo129 nit: it may be worth listing/explaining the reference frames in which these spatial coordinates can be defined.

hidmic · 2022-11-22T13:28:53Z

tutorials/environmental_data_and_visualization.md

+    <plugin
+      filename="gz-sim-environment-preload-system"
+      name="gz::sim::systems::EnvironmentPreload">
+      <data>environmental_data.csv</data>


@arjo129 nit: it may be worth mentioning how path resolution works here.

+1
When I run

gz sim environmental_sensor.sdf

I get

[Err] [EnvironmentalSensorSystem.cc:460] No sensor data loaded [Err] [EnvironmentPreload.cc:258] No environmental data file was found at environmental_data.csv

The csv file probably needs to be installed somewhere, to build/ or install/. I don't know which one gz sim is picking up, as there are three:

$ find . -name environmental_sensor.sdf ./src/gz-sim/examples/worlds/environmental_sensor.sdf ./install/share/gz/gz-sim7/worlds/environmental_sensor.sdf ./build/gz-sim7/test/worlds/environmental_sensor.sdf

But there are only 2 CSV files, only in src/:

$ find . -name environmental_data.csv ./src/gz-sim/test/worlds/environmental_data.csv ./src/gz-sim/examples/worlds/environmental_data.csv

gz sim is probably not finding them.

Even if I copy one of the CSV files to the current working directory, now I don't get the 2nd error about file not found, but I still get the first error "No sensor data loaded."

hidmic · 2022-11-22T13:29:35Z

tutorials/environmental_data_and_visualization.md

+
+@image html files/environment_data/environment_pc_panel.png
+
+Hit the refresh button to find the pointcloud generated b y the visuallization


@arjo129 nit:

Suggested change

Hit the refresh button to find the pointcloud generated b y the visuallization

Hit the refresh button to find the pointcloud generated by the visualization

hidmic · 2022-11-22T13:29:50Z

tutorials/environmental_data_and_visualization.md

+
+Hit the refresh button to find the pointcloud generated b y the visuallization
+plugin. You can select the field you wish to view using the "Float vector"
+field. Once thats done, you should be able to see the data.


@arjo129 nit:

Suggested change

field. Once thats done, you should be able to see the data.

field. Once that's done, you should be able to see the data.

hidmic · 2022-11-22T13:30:05Z

tutorials/environmental_data_and_visualization.md

+
+# Using custom sensors to sense the data
+
+Apart from just visuallization, there are custom sensors available for sampling


@arjo129 nit:

Suggested change

Apart from just visuallization, there are custom sensors available for sampling

Apart from just visualization, there are custom sensors available for sampling

hidmic · 2022-11-22T13:30:53Z

tutorials/environmental_data_and_visualization.md

+```
+# Using C++ API to integrate with your own plugins
+
+You may want to use environmental data to model your own plugins for instance,


@arjo129 nit:

Suggested change

You may want to use environmental data to model your own plugins for instance,

You may want to use environmental data to model your own plugins. For instance,

mjcarroll · 2023-01-23T17:18:07Z

Friendly ping @arjo129 , you have some outstanding comments here.

mabelzhang

I read it in the rendered webpage (build/gz-sim7/doxygen/html/environmental_data.html) to see the images. A few notes regarding formatting issues that are different from GitHub rendering.

I still need to test it, but since that's in a different branch, I'll do it as a second pass when I switch branches...

Side note, I noticed that all the images have the reset button cut off... I saw this in other worlds on Fuel as well. It seems that our default GUI config has a box too small for the buttons, and it'll have to be fixed elsewhere... Probably don't need to remake the images here. They just don't look good, like bugs in official tutorial images...

mabelzhang · 2023-06-17T00:39:14Z

tutorials/environmental_data_and_visualization.md

@@ -0,0 +1,173 @@
+\page environmental_data Visualizing Environment Data
+
+Lot of times you may come accross environment properties that vary with time and


Suggested change

Lot of times you may come accross environment properties that vary with time and

A lot of times you may come accross environment properties that vary with time and

mabelzhang · 2023-06-17T00:39:21Z

tutorials/environmental_data_and_visualization.md

+Lot of times you may come accross environment properties that vary with time and
+space. For instance if you'd like to model properties such as gas distribution
+and water temperature one needs to have some way of defining values such that
+the values vary through out the environment. Gazebo's Environment Data


Suggested change

the values vary through out the environment. Gazebo's Environment Data

the values vary throughout the environment. Gazebo's Environment Data

mabelzhang · 2023-06-17T00:39:48Z

tutorials/environmental_data_and_visualization.md

+
+# Loading Environment Data
+
+Currently gazebo is able to load data from CSV files and display them. It is


Suggested change

Currently gazebo is able to load data from CSV files and display them. It is

Currently Gazebo is able to load data from CSV files and display them. It is

mabelzhang · 2023-06-17T03:56:27Z

tutorials/environmental_data_and_visualization.md

@@ -0,0 +1,173 @@
+\page environmental_data Visualizing Environment Data


The tutorial also needs to be linked from a new bullet in tutorials.md.in

mabelzhang · 2023-06-17T03:58:08Z

tutorials/environmental_data_and_visualization.md

+the values vary through out the environment. Gazebo's Environment Data
+infrastructure enables such visualization and sensing capabilities.
+
+# Loading Environment Data


Previewing the rendered webpage (build/gz-sim7/doxygen/html/environmental_data.html) locally shows that these headings are huge, bigger than the title. Probably need to knock all headings down a level. Same for below.

Suggested change

# Loading Environment Data

## Loading Environment Data

mabelzhang · 2023-06-17T04:01:58Z

tutorials/environmental_data_and_visualization.md

+      </dimensions>
+    </plugin>
+```
+If you would like to use Latitude and Longitude you may set the `reference` to


Suggested change

If you would like to use Latitude and Longitude you may set the `reference` to

If you would like to use latitude and longitude you may set the `reference` to

mabelzhang · 2023-06-17T04:06:52Z

tutorials/environmental_data_and_visualization.md

+
+@image html files/environment_data/environment_viz_sliders.png
+
+Next search for the "Point Cloud" tool.


This is 3 panels to add in order to visualize some data... It's a lot of steps to remember. Without a tutorial, the interface is not intuitive at all, and a user would never guess it. Is there a way to simplify this UI? Or in the meantime, maybe even just have some tooltips in the panels to tell the user which panels they need to open next?

mabelzhang · 2023-06-17T04:11:03Z

tutorials/environmental_data_and_visualization.md

+
+@image html files/environment_data/environment_data_viz.png
+
+# Using custom sensors to sense the data


"Custom sensor" is an overloaded phrase that, in the general Gazebo context, means custom sensor plugins. That is, the thing that was a lot of work to write in Gazebo-classic but made a lot easier in new Gazebo, which is an advertised feature of the new Gazebo. So I wouldn't use that phrase here. It would be very confusing if the user is familiar with that context.

Probably "custom environmental sensor" would be more appropriate.

mabelzhang · 2023-06-17T04:12:21Z

tutorials/environmental_data_and_visualization.md

+  <update_rate>30</update_rate>
+</sensor>
+```
+By default it'll echo to a topic generated by the scoped name.


By echo, do you mean publish or subscribe?

mabelzhang · 2023-06-17T04:14:34Z

examples/worlds/environmental_sensor.sdf

-<!-- This example show cases how to load and unload environmental data. -->
+<!-- This example show cases how to use a custom sensor to sample data.
+You should be able to get data echoed on the topic
+"/world/environmental_sensor_test/model/model_with_sensor/link/link/sensor/custom_sensor/environmental_sensor/humidity"


Looks from the tutorial that it can be customized by <topic>, it would be worth adding that to this example file.

mabelzhang

With the branches in #1842 and gazebosim/gz-msgs#320, I tested the Pre-loading section in this tutorial and the Via GUI section and left some comments.

I'm not able to load the CSV, so I cannot get to the Visualization section and beyond.

Maybe those branches just need to be updated? The gz-msgs PR says it is outdated. Probably both PRs need an update.
Edit: I updated both branches locally, and I still wasn't able to get the topic to show up in the drop-down list in the Point Cloud panel.

mabelzhang · 2023-06-21T01:22:23Z

tutorials/environmental_data_and_visualization.md

+## Pre-Loading
+If you are writing SDFormat files by hand or via a script you may want to
+pre-load the data. You may use the `EnvironmentPreload` plugin for this. For
+instance we could load the first file like so (by adding the environment to the


+1
And there should be a command how to run the example SDF file, like

gz sim environmental_sensor.sdf

mabelzhang · 2023-06-21T01:25:15Z

tutorials/environmental_data_and_visualization.md

+    <plugin
+      filename="gz-sim-environment-preload-system"
+      name="gz::sim::systems::EnvironmentPreload">
+      <data>environmental_data.csv</data>


+1
When I run

gz sim environmental_sensor.sdf

I get

[Err] [EnvironmentalSensorSystem.cc:460] No sensor data loaded [Err] [EnvironmentPreload.cc:258] No environmental data file was found at environmental_data.csv

The csv file probably needs to be installed somewhere, to build/ or install/. I don't know which one gz sim is picking up, as there are three:

$ find . -name environmental_sensor.sdf ./src/gz-sim/examples/worlds/environmental_sensor.sdf ./install/share/gz/gz-sim7/worlds/environmental_sensor.sdf ./build/gz-sim7/test/worlds/environmental_sensor.sdf

But there are only 2 CSV files, only in src/:

$ find . -name environmental_data.csv ./src/gz-sim/test/worlds/environmental_data.csv ./src/gz-sim/examples/worlds/environmental_data.csv

gz sim is probably not finding them.

Even if I copy one of the CSV files to the current working directory, now I don't get the 2nd error about file not found, but I still get the first error "No sensor data loaded."

mabelzhang · 2023-06-21T01:26:05Z

tutorials/environmental_data_and_visualization.md

+```
+
+## Via GUI
+To load the data, click on the 3 dots and search for the "Environment Loader":


For completeness, should instruct the user how to bring up the GUI we are starting from, e.g. I'm assuming it is

gz sim empty.sdf

Ah, with the new changes we need to preload

<plugin filename="gz-sim-environmental-sensor-system" name="gz::sim::systems::EnvironmentalSystem"> </plugin>

Otherwise nothing will work. Perhaps I should add an empty_environment_system.sdf?

mabelzhang · 2023-06-21T01:37:45Z

tutorials/environmental_data_and_visualization.md

+
+@image html files/environment_data/environment_file_loaded.png
+
+Select the appropriate fields and then click "load".


Currently, the data doesn't seem to be loading for me.

~~Right when I click the button to browse files, I get this error~~

[GUI] [Err] [EnvironmentLoader.cc:194] No environmental data file was found at

~~That's probably not supposed to happen. It seems it's trying to load too early, right when I click the browse button, before I even choose a file.~~
Update: This error disappeared after I updated the branches in the 2 PRs locally from gz-sim7 and gz-msgs9. But the following behavior is still the same.

Then after I select the environmental_data.csv in src/examples/worlds, select the right columns for the XYZ fields, and click Load, nothing happens - no more printouts. If the data loaded, I would expect a printout. If the data didn't load, I would expect an error.

Then if I proceed to the Visualization section, after I click the refresh button, there's still nothing in the drop-down list of topics.

Thanks for checking this. Were you using these two branches:

Fix enviroment system loading mechanism #1842

Adds a message that allows loading environments via a topic gz-msgs#320

Yes, those were the branches I used. I also tried after merging from gz-msgs9 and gz-sim7, as the gz-msgs PR says the branch is outdated.

Apologies for not getting back to this quicker but I've found the reason why. I've fixed it with c30639a.

mabelzhang · 2023-06-21T01:39:27Z

tutorials/environmental_data_and_visualization.md

+Select the file you would want to load by clicking on the arrow next to the
+field named "Data file path".
+
+@image html files/environment_data/environment_file_loaded.png


There should be an explanation of what the Reference fields mean - what the default ecef is, and the difference between global and spherical.

azeey · 2024-08-06T19:11:59Z

@arjo129 what's the status of this PR? Do we want to get it in for the Ionic release?

arjo129 · 2024-08-07T00:05:35Z

Yes will work on the feedback.

arjo129 requested review from mabelzhang and mjcarroll as code owners November 22, 2022 07:22

github-actions bot added the 🌱 garden Ignition Garden label Nov 22, 2022

Spelling

bfcca7b

Signed-off-by: Arjo Chakravarty <arjo@openrobotics.org>

hidmic reviewed Nov 22, 2022

View reviewed changes

arjo129 mentioned this pull request Apr 14, 2023

Fix enviroment system loading mechanism #1842

Merged

9 tasks

mabelzhang reviewed Jun 17, 2023

View reviewed changes

mabelzhang reviewed Jun 21, 2023

View reviewed changes

azeey added the beta Targeting beta release of upcoming collection label Jul 31, 2023

azeey added documentation Improvements or additions to documentation and removed beta Targeting beta release of upcoming collection labels Aug 18, 2023

mabelzhang mentioned this pull request Nov 9, 2023

Point cloud visualization plugin gazebosim/gz-gui#346

Merged

8 tasks

azeey added the beta Targeting beta release of upcoming collection label Jul 29, 2024

azeey removed the beta Targeting beta release of upcoming collection label Sep 26, 2024


		@image html files/environment_data/environment_pc_panel.png

		Hit the refresh button to find the pointcloud generated b y the visuallization

	field. Once thats done, you should be able to see the data.
	field. Once that's done, you should be able to see the data.


		# Using custom sensors to sense the data

		Apart from just visuallization, there are custom sensors available for sampling

	Apart from just visuallization, there are custom sensors available for sampling
	Apart from just visualization, there are custom sensors available for sampling

	You may want to use environmental data to model your own plugins for instance,
	You may want to use environmental data to model your own plugins. For instance,

		@@ -0,0 +1,173 @@
		\page environmental_data Visualizing Environment Data

		Lot of times you may come accross environment properties that vary with time and

	Lot of times you may come accross environment properties that vary with time and
	A lot of times you may come accross environment properties that vary with time and

	the values vary through out the environment. Gazebo's Environment Data
	the values vary throughout the environment. Gazebo's Environment Data


		# Loading Environment Data

		Currently gazebo is able to load data from CSV files and display them. It is

	If you would like to use Latitude and Longitude you may set the `reference` to
	If you would like to use latitude and longitude you may set the `reference` to


		@image html files/environment_data/environment_viz_sliders.png

		Next search for the "Point Cloud" tool.


		@image html files/environment_data/environment_file_loaded.png

		Select the appropriate fields and then click "load".

Adds a tutorial explaining how to use Environmental Data infrastructure. #1806

Are you sure you want to change the base?

Adds a tutorial explaining how to use Environmental Data infrastructure. #1806

Conversation

arjo129 commented Nov 22, 2022

codecov bot commented Nov 22, 2022 • edited Loading

Codecov Report

hidmic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjcarroll commented Jan 23, 2023

mabelzhang left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mabelzhang left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mabelzhang Jun 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mabelzhang Jun 21, 2023 • edited Loading

Choose a reason for hiding this comment

azeey commented Aug 6, 2024

arjo129 commented Aug 7, 2024

codecov bot commented Nov 22, 2022 •

edited

Loading

mabelzhang left a comment •

edited

Loading

mabelzhang left a comment •

edited

Loading

mabelzhang Jun 21, 2023 •

edited

Loading

mabelzhang Jun 21, 2023 •

edited

Loading