Realsense d435 resolution bad

[PhilipAmadasun]

How do I improve realsense d435 resolution to get better gmapping results?

[MartyG-RealSense]

Hi @PhilipAmadasun If you wish to use a higher resolution than the default setting then you can configure a custom launch resolution. For example:

ROS1
roslaunch realsense2_camera rs_camera.launch depth_width:=1280 depth_height:=720 depth_fps:=30 color_width:=1280 color_height:=720 color_fps:=30

ROS2_BETA
ros2 launch realsense2_camera rs_launch.py depth_module.profile:=1280x720x30 rgb_camera.profile:=1280x720x30

[PhilipAmadasun]

My rs_camera.launch file looks like this now.

<launch>
 <arg name="serial_no"           default=""/>
 <arg name="usb_port_id"         default=""/>
 <arg name="device_type"         default=""/>
 <arg name="json_file_path"      default=""/>
 <arg name="camera"              default="camera"/>
 <arg name="tf_prefix"           default="$(arg camera)"/>
 <arg name="external_manager"    default="false"/>
 <arg name="manager"             default="realsense2_camera_manager"/>
 <arg name="output"              default="screen"/>

 <arg name="fisheye_width"       default="-1"/>
 <arg name="fisheye_height"      default="-1"/>
 <arg name="enable_fisheye"      default="false"/>

 <arg name="depth_width"         default="1280"/>
 <arg name="depth_height"        default="720"/>
 <arg name="enable_depth"        default="true"/>

 <arg name="confidence_width"    default="-1"/>
 <arg name="confidence_height"   default="-1"/>
 <arg name="enable_confidence"   default="true"/>
 <arg name="confidence_fps"      default="-1"/>

 <arg name="infra_width"         default="848"/>
 <arg name="infra_height"        default="480"/>
 <arg name="enable_infra"        default="false"/>
 <arg name="enable_infra1"       default="false"/>
 <arg name="enable_infra2"       default="false"/>
 <arg name="infra_rgb"           default="false"/>

 <arg name="color_width"         default="1280"/>
 <arg name="color_height"        default="720"/>
 <arg name="enable_color"        default="true"/>

 <arg name="fisheye_fps"         default="-1"/>
 <arg name="depth_fps"           default="30"/>
 <arg name="infra_fps"           default="30"/>
 <arg name="color_fps"           default="30"/>
 <arg name="gyro_fps"            default="-1"/>
 <arg name="accel_fps"           default="-1"/>
 <arg name="enable_gyro"         default="false"/>
 <arg name="enable_accel"        default="false"/>

 <arg name="enable_pointcloud"         default="false"/>
 <arg name="pointcloud_texture_stream" default="RS2_STREAM_COLOR"/>
 <arg name="pointcloud_texture_index"  default="0"/>
 <arg name="allow_no_texture_points"   default="false"/>
 <arg name="ordered_pc"                default="false"/>

 <arg name="enable_sync"               default="false"/>
 <arg name="align_depth"               default="false"/>

 <arg name="publish_tf"                default="true"/>
 <arg name="tf_publish_rate"           default="0"/>

 <arg name="filters"                   default=""/>
 <arg name="clip_distance"             default="-2"/>
 <arg name="linear_accel_cov"          default="0.01"/>
 <arg name="initial_reset"             default="false"/>
 <arg name="unite_imu_method"          default=""/>
 <arg name="topic_odom_in"             default="odom_in"/>
 <arg name="calib_odom_file"           default=""/>
 <arg name="publish_odom_tf"           default="true"/>

 <arg name="stereo_module/exposure/1"  default="7500"/>
 <arg name="stereo_module/gain/1"      default="16"/>
 <arg name="stereo_module/exposure/2"  default="1"/>
 <arg name="stereo_module/gain/2"      default="16"/>
 
 

 <group ns="$(arg camera)">
   <include file="$(find realsense2_camera)/launch/includes/nodelet.launch.xml">
     <arg name="tf_prefix"                value="$(arg tf_prefix)"/>
     <arg name="external_manager"         value="$(arg external_manager)"/>
     <arg name="manager"                  value="$(arg manager)"/>
     <arg name="output"                   value="$(arg output)"/>
     <arg name="serial_no"                value="$(arg serial_no)"/>
     <arg name="usb_port_id"              value="$(arg usb_port_id)"/>
     <arg name="device_type"              value="$(arg device_type)"/>
     <arg name="json_file_path"           value="$(arg json_file_path)"/>

     <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
     <arg name="pointcloud_texture_stream" value="$(arg pointcloud_texture_stream)"/>
     <arg name="pointcloud_texture_index"  value="$(arg pointcloud_texture_index)"/>
     <arg name="enable_sync"              value="$(arg enable_sync)"/>
     <arg name="align_depth"              value="$(arg align_depth)"/>

     <arg name="fisheye_width"            value="$(arg fisheye_width)"/>
     <arg name="fisheye_height"           value="$(arg fisheye_height)"/>
     <arg name="enable_fisheye"           value="$(arg enable_fisheye)"/>

     <arg name="depth_width"              value="$(arg depth_width)"/>
     <arg name="depth_height"             value="$(arg depth_height)"/>
     <arg name="enable_depth"             value="$(arg enable_depth)"/>

     <arg name="confidence_width"         value="$(arg confidence_width)"/>
     <arg name="confidence_height"        value="$(arg confidence_height)"/>
     <arg name="enable_confidence"        value="$(arg enable_confidence)"/>
     <arg name="confidence_fps"           value="$(arg confidence_fps)"/>

     <arg name="color_width"              value="$(arg color_width)"/>
     <arg name="color_height"             value="$(arg color_height)"/>
     <arg name="enable_color"             value="$(arg enable_color)"/>

     <arg name="infra_width"              value="$(arg infra_width)"/>
     <arg name="infra_height"             value="$(arg infra_height)"/>
     <arg name="enable_infra"             value="$(arg enable_infra)"/>
     <arg name="enable_infra1"            value="$(arg enable_infra1)"/>
     <arg name="enable_infra2"            value="$(arg enable_infra2)"/>
     <arg name="infra_rgb"                value="$(arg infra_rgb)"/>

     <arg name="fisheye_fps"              value="$(arg fisheye_fps)"/>
     <arg name="depth_fps"                value="$(arg depth_fps)"/>
     <arg name="infra_fps"                value="$(arg infra_fps)"/>
     <arg name="color_fps"                value="$(arg color_fps)"/>
     <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
     <arg name="accel_fps"                value="$(arg accel_fps)"/>
     <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
     <arg name="enable_accel"             value="$(arg enable_accel)"/>

     <arg name="publish_tf"               value="$(arg publish_tf)"/>
     <arg name="tf_publish_rate"          value="$(arg tf_publish_rate)"/>

     <arg name="filters"                  value="$(arg filters)"/>
     <arg name="clip_distance"            value="$(arg clip_distance)"/>
     <arg name="linear_accel_cov"         value="$(arg linear_accel_cov)"/>
     <arg name="initial_reset"            value="$(arg initial_reset)"/>
     <arg name="unite_imu_method"         value="$(arg unite_imu_method)"/>
     <arg name="topic_odom_in"            value="$(arg topic_odom_in)"/>
     <arg name="calib_odom_file"          value="$(arg calib_odom_file)"/>
     <arg name="publish_odom_tf"          value="$(arg publish_odom_tf)"/>
     <arg name="stereo_module/exposure/1" value="$(arg stereo_module/exposure/1)"/>
     <arg name="stereo_module/gain/1"     value="$(arg stereo_module/gain/1)"/>
     <arg name="stereo_module/exposure/2" value="$(arg stereo_module/exposure/2)"/>
     <arg name="stereo_module/gain/2"     value="$(arg stereo_module/gain/2)"/>

     <arg name="allow_no_texture_points"  value="$(arg allow_no_texture_points)"/>
     <arg name="ordered_pc"               value="$(arg ordered_pc)"/>
     
   </include>
 </group>
</launch> 

The map being generated is still wonky.

[MartyG-RealSense]

Gmapping has a launch file parameter called delta for setting the map resolution, which has a default value of 0.05. It is typically defined in the Gmapping launch file like this:

<param name="delta" value="0.05"/>

My understanding is that the 0.05 value is in meters (with each square in the grid being 0.05 m or 5 cm) and decreasing the value improves the resolution of the map, though possibly at the expense of performance.

[PhilipAmadasun]

@MartyG-RealSense I changed that parameter to 0.01 and yet, there is still resolution issues although it's better but not acceptable. I also have the problem where the camera is unable to register certain kinds of objects. like table legs or objects that have large openings in mid section, it can't register the solid base or solid top of these sorts of objects. Please see image for illustration of what I mean. It also has issues registering chairs and objects that are low o the ground. How do I fix this?

[MartyG-RealSense]

If the objects that are being observed are black (as indicated on your illustration) then the camera will have difficulty capturing depth detail from them. This is because it is a general physics principle (not specific to RealSense) that dark grey or black absorbs light and so makes it more difficult for depth cameras to read depth information from such surfaces. The darker the color shade, the more light that is absorbed and so the less depth detail that the camera can obtain. Casting a strong light source onto black objects can help to bring out depth detail from them.

Is the camera is mounted within a shell on the turtlebot? If it is then please check whether parts of the shell could be obscuring the camera's view of the floor level.

[PhilipAmadasun]

@MartyG-RealSense The camera is mounted in the position shown below

I am using the depthimage_to_laserscan package for gmapping.Also, from rviz I can see there is a horizontal line that divides what part of the image is converted to laserscan data. the part of the image below the horizontal line is not used. I don't know how to change this.

[MartyG-RealSense]

A suggestion that I have seen for dealing with missing detail due to the horizontal line when using depthimage_to_laserscan is to set depth_registration to false instead of true in the 3dsensor.launch file referenced in your past case at #2425 (comment)

[PhilipAmadasun]

@MartyG-RealSense This is a what I mean by "horizontal line" . No sure if I already shared this image, basically every thing below the horizontal line does not register.

[MartyG-RealSense]

There is something resembling that white horizontal line in a depthimage_to_laserscan case at IntelRealSense/librealsense#9556 where the RealSense ROS user was experiencing problems with non-existent obstacles that were being added

When you say that everything below the line does not register, do you mean that they are not being detected as obstacles?

[PhilipAmadasun]

@MartyG-RealSense WHat I mean is, when I am gmapping(SLAM: map creation), everything below the horizontal line does not register. The horizontal line represents the laserscan data. The laserscan is created from where the camera is positioned on the robot in real life. It seems that the part depth information from that line of sight(that is, the horizontal line) is used for creating the laserscan, the laser scan information is thereofre kind of useless as it does not effeceinetly describe the environment because there are obstacles or parts of obstacles below that line of sight. Not sure how to resolve the issue.

[MartyG-RealSense]

I researched your question carefully. Some references discussed how depthimage_to_laserscan has a height parameter that determines how much of the scene the laser scan can see, with a larger height value meaning that more of the floor can be seen.

My understanding from my research is that the maximum value of 'height' is the height of the resolution that you are using - so if the image is 640x480 then height = 480, as illustrated in a discussion at ros-perception/depthimage_to_laserscan#32

[PhilipAmadasun]

@MartyG-RealSense I don't know what exactly the 'height' thing is. As far as I can see, the only parameter with the word 'heoght' in it is 'scan-height'.

[MartyG-RealSense]

My recollection from my research during my comment above is that 'height' and 'scan-height' refer to the same thing.

[PhilipAmadasun]

@MartyG-RealSense I changed it and nothing happened unfortunately.I'm at a lose. I'm trying rtabmap but thats giving me problems too.

[MartyG-RealSense]

I researched the gmapping / depthimage_to_laserscan white line further but did not find further relevant references, unfortunately.

I believe that you have already tried Intel's RealSense SLAM guide for ROS that makes use of RTABMAP?

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

[MartyG-RealSense]

Hi @PhilipAmadasun Do you have an update about this case that you can provide, please? Thanks!

[MartyG-RealSense]

Hi @PhilipAmadasun Do you require further assistance with this case, please? Thanks!

[PhilipAmadasun]

I am currently looking at other solutions and will get back to you

…

On Sun, Oct 23, 2022 at 11:40 PM MartyG-RealSense ***@***.***> wrote: Hi @PhilipAmadasun <https://github.com/PhilipAmadasun> Do you require further assistance with this case, please? Thanks! — Reply to this email directly, view it on GitHub <#2493 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AND3KDTQSULC3TMTKIQHDBDWEYVNNANCNFSM6AAAAAAQZFKUJM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[MartyG-RealSense]

Thanks very much for the update!

[MartyG-RealSense]

Hi @PhilipAmadasun Do you have an update about this case that you can provide, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.