Initial Commit

.gitignore

@@ -0,0 +1,8 @@
+# Colcon
+**/install/
+**/log/
+**/build/
+
+# VScode
+**.code-workspace
+**.vscode

CHANGELOG.md

@@ -0,0 +1,10 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+
+## [0.1.0] - 2020-03-16
+### Added
+- Initial version

LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2020, Andrej Orsula
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,57 @@
+# ROS 2 Wrapper for OpenFace
+![OS](https://img.shields.io/badge/OS-Ubuntu_18.04-orange.svg) ![ROS_2](https://img.shields.io/badge/ROS_2-Eloquent-brightgreen.svg) ![OPENFACE](https://img.shields.io/badge/OpenFace-2.2-lightgrey.svg)
+
+
+This repository contains a ROS 2 wrapper for [OpenFace](https://github.com/TadasBaltrusaitis/OpenFace), which allows to extract the following information from an RGB video, see [msgs](ros2_openface_msgs/msg).
+- 2D/3D Facial Landmarks
+- Head Pose
+- 2D/3D Eye Landmarks
+- Gaze
+  - Left Eye
+  - Right Eye
+  - Compound
+- Action Units
+
+## Installation
+Please feel free to follow the installation scripts provided in the [`scripts` folder](scripts), namely the [`ROS 2 Eloquent` installation script](scripts/install_ros2_distro_eloquent.bash) (if not already installed) and [`ROS2 OpenFace` installation script](scripts/install_ros2_openface.bash), which encapsulates the rest.
+
+
+### Requirements
+These are the build/run-time requirements for this package.
+
+#### 1) [ROS 2 Eloquent](https://index.ros.org/doc/ros2/Installation/Eloquent) with [Development Tools](https://index.ros.org/doc/ros2/Installation/Eloquent/Linux-Development-Setup/#install-development-tools-and-ros-tools)
+
+#### 2) [OpenFace 2](https://github.com/TadasBaltrusaitis/OpenFace/wiki/Unix-Installation) (tested with 2.2.0)
+Together with its dependencies, such as [dlib 9](http://dlib.net/compile.html) (tested with 9.13) and [OpenCV 4](https://github.com/opencv/opencv) (tested with 4.1.2).
+
+#### 3) [cv_bridge](https://github.com/ros-perception/vision_opencv/tree/ros2) (with support for OpenCV 4)
+
+### Building
+Build with [colcon](https://colcon.readthedocs.io/en/released).
+
+
+## Usage
+First, source the ROS 2 global installation (if not done before).
+```bash
+source /opt/ros/eloquent/setup.bash
+```
+
+Then source the ROS 2 workspace overlay (if not done before).
+```bash
+source /path/to/ros2_openface/install/local_setup.bash
+```
+
+Now it can be run with your favourite camera or video from ros2bag.
+```bash
+ros2 launch openface openface.launch.py
+```
+
+
+## Acknowledgment
+This repository is based on the amazing work of authors that contributed to OpenFace as well as its the underlying frameworks it utilises.
+
+
+## License
+This project is licensed under [BSD 3-Clause License](LICENSE).
+
+You have to respect OpenFace, dlib, and OpenCV licenses, together with licenses of the datasets used for OpenFace model training.

ros2_openface/CMakeLists.txt

@@ -0,0 +1,103 @@
+cmake_minimum_required(VERSION 3.5)
+project(openface)
+
+
+if(NOT CMAKE_CXX_STANDARD)
+  set(CMAKE_CXX_STANDARD 17)
+endif()
+
+if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
+  add_compile_options(-Wall -Wextra -Wpedantic)
+endif()
+
+
+if(BUILD_TESTING)
+  find_package(ament_lint_auto REQUIRED)
+  # the following line skips the linter which checks for copyrights
+  # uncomment the line when a copyright and license is not present in all source files
+  #set(ament_cmake_copyright_FOUND TRUE)
+  # the following line skips cpplint (only works in a git repo)
+  # uncomment the line when this package is not in a git repo
+  #set(ament_cmake_cpplint_FOUND TRUE)
+  ament_lint_auto_find_test_dependencies()
+endif()
+
+
+## Dependencies
+# ROS2 packages
+find_package(ament_cmake REQUIRED)
+find_package(rclcpp REQUIRED)
+find_package(image_transport REQUIRED)
+find_package(cv_bridge REQUIRED)
+find_package(tf2 REQUIRED)
+find_package(tf2_ros REQUIRED)
+
+# ROS2 interfaces
+find_package(std_msgs REQUIRED)
+find_package(geometry_msgs REQUIRED)
+find_package(sensor_msgs REQUIRED)
+find_package(openface_msgs REQUIRED)
+
+# OpenCV 4
+find_package(OpenCV 4 REQUIRED)
+
+# Boost
+find_package(Boost 1.5.9 REQUIRED COMPONENTS filesystem system)
+
+# OpenFace
+find_library(OPENFACE_LD_LIB LandmarkDetector PATH_SUFFIXES openface)
+find_library(OPENFACE_FA_LIB FaceAnalyser PATH_SUFFIXES openface)
+find_library(OPENFACE_GA_LIB GazeAnalyser PATH_SUFFIXES openface)
+find_library(OPENFACE_UT_LIB Utilities PATH_SUFFIXES openface)
+set(OPENFACE_LIBS ${OPENFACE_LD_LIB} ${OPENFACE_FA_LIB} ${OPENFACE_GA_LIB} ${OPENFACE_UT_LIB})
+find_path(OPENFACE_INCLUDE_DIRS LandmarkCoreIncludes.h PATH_SUFFIXES OpenFace)
+
+
+## Include dirs
+include_directories(
+  include
+  ${OPENFACE_INCLUDE_DIRS}
+  ${OpenBLAS_INCLUDE_DIR}
+  ${Boost_INCLUDE_DIRS}
+  ${Boost_INCLUDE_DIRS}/boost
+  ${OpenCV_INCLUDE_DIRS}
+)
+
+
+## Install dirs
+install(DIRECTORY
+  config launch
+  DESTINATION share/${PROJECT_NAME}
+)
+
+
+## Executables
+set(EXECUTABLE_0 openface)
+add_executable(${EXECUTABLE_0} src/${EXECUTABLE_0}.cpp)
+
+# Executable 0
+target_link_libraries(${EXECUTABLE_0}
+  ${OPENFACE_LIBS}
+  openblas
+  dlib
+  ${Boost_LIBRARIES}
+  ${OpenCV_LIBRARIES}
+)
+ament_target_dependencies(${EXECUTABLE_0}
+  rclcpp
+  std_msgs
+  geometry_msgs
+  sensor_msgs
+  openface_msgs
+  image_transport
+  cv_bridge
+  tf2
+  tf2_ros
+)
+install(TARGETS
+  ${EXECUTABLE_0}
+  DESTINATION lib/${PROJECT_NAME}/
+)
+
+
+ament_package()

ros2_openface/config/openface.yaml

@@ -0,0 +1,18 @@
+openface:
+  ros__parameters:
+    enable_face_features: true
+    enable_head_pose: true
+
+    enable_eye_features: true
+    enable_gaze: true
+
+    enable_action_units: true
+
+    enable_aligned_face: true # Only for visualisation
+    enable_hog: true # Only for visualisation
+
+    broadcast_tf: true
+    visualise: true
+    track_fps: true
+
+    use_sim_time: false

ros2_openface/launch/openface.launch.py

@@ -0,0 +1,36 @@
+"""Launch OpenFace"""
+
+import os
+from ament_index_python.packages import get_package_share_directory
+from launch import LaunchDescription
+from launch.actions import DeclareLaunchArgument
+from launch_ros.actions import Node
+from launch.substitutions import LaunchConfiguration
+from launch.actions import IncludeLaunchDescription
+from launch.launch_description_sources import PythonLaunchDescriptionSource
+from launch.substitutions import ThisLaunchFileDir
+
+
+def generate_launch_description():
+
+    config_openface = LaunchConfiguration('config_openface', default=os.path.join(get_package_share_directory(
+        'openface'), 'config', 'openface.yaml'))
+
+    return LaunchDescription([
+        DeclareLaunchArgument(
+            'config_openface',
+            default_value=config_openface,
+            description='Path to config for openface'),
+
+        Node(
+            package='openface',
+            node_executable='openface',
+            node_name='openface',
+            node_namespace="",
+            output='screen',
+            parameters=[config_openface],
+            remappings=[('camera/image_raw', 'camera/color/image_raw'),
+                        ('camera/camera_info', 'camera/color/camera_info'),
+                        ('openface/faces', 'faces')],
+        ),
+    ])

ros2_openface/package.xml

@@ -0,0 +1,39 @@
+<?xml version="1.0"?>
+<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
+<package format="3">
+  <name>openface</name>
+  <version>0.1.0</version>
+  <description>ROS 2 wrapper around OpenFace</description>
+  <maintainer email="[email protected]">Andrej Orsula</maintainer>
+  <author>Andrej Orsula</author>
+  <license>BSD</license>
+
+  <buildtool_depend>ament_cmake</buildtool_depend>
+  <test_depend>ament_lint_auto</test_depend>
+  <test_depend>ament_lint_common</test_depend>
+
+  <build_depend>rclcpp</build_depend>
+  <exec_depend>rclcpp</exec_depend>
+
+  <build_depend>image_transport</build_depend>
+  <exec_depend>image_transport</exec_depend>
+
+  <build_depend>cv_bridge</build_depend>
+  <exec_depend>cv_bridge</exec_depend>
+
+  <build_depend>tf2</build_depend>
+  <exec_depend>tf2</exec_depend>
+
+  <build_depend>tf2_ros</build_depend>
+  <exec_depend>tf2_ros</exec_depend>
+
+  <build_depend>sensor_msgs</build_depend>
+  <exec_depend>sensor_msgs</exec_depend>
+
+  <build_depend>openface_msgs</build_depend>
+  <exec_depend>openface_msgs</exec_depend>
+
+  <export>
+    <build_type>ament_cmake</build_type>
+  </export>
+</package>