Face Class Reference

This class runs the client that handles the vertices coming from the python server face_mesh_server.py to render the face landmarking. More...

#include <face.h>

Public Member Functions
	Face ()
	Constructor. More...
void	update (void)
Eigen::MatrixXd	get_vertices (void)
	Get face vertices. More...
Eigen::MatrixXi	get_surface_indices (void)
	Get the surface indices of the face mesh. More...
Eigen::MatrixXd	translation_matrix (const Eigen::Vector3d &offset, size_t vert_num)
	This function offset a set of vertices by a given offset. More...

Private Member Functions
void	setup_tcp_communication (void)
	Setup communication with python server. More...
Eigen::MatrixXd	incoming_data_callback (void)
	Read incoming data. More...

Private Attributes
std::filesystem::path	m_vertices_connections_abs_path
	Absolute name of vertices connection files. More...
std::string	m_vertices_connections_rel_filename
	Relative name of vertices connection files. More...
Eigen::MatrixXi	m_faces
	Face landmarking indices matrix. More...
std::string	m_local_ip_address = "127.0.1.1"
	Start tcp client at this ip adress (localhost). More...
const int	m_tcp_port = 5052
int	m_socket = 0
	Socket handle. More...
const int	m_buffer_size = 40000
const int	m_vertices_num = 468
	Number of vertices provided by the mediapipe. This is constant. More...
Eigen::MatrixXi	m_surf_indices
	The surface element indices of the incoming mesh. More...
std::future< Eigen::MatrixXd >	m_future_fun
	Future function handle. More...
bool	m_return_value = 0
	Termination flag for callback function. More...
Eigen::MatrixXd	m_vol_mesh_vertices
	Volume mesh data. More...
Eigen::Matrix3d	m_rot_mat = Eigen::Matrix3d::Identity()
	Face rotation matrix. More...
double	m_face_mesh_scale = 6.0
	Face scale matrix. More...
Eigen::Vector3d	m_face_offset = Eigen::Vector3d({0.0, -2.0, 0.0})
Eigen::Matrix3d	m_scale_mat

Detailed Description

This class runs the client that handles the vertices coming from the python server face_mesh_server.py to render the face landmarking.

Definition at line 25 of file face.h.

Constructor & Destructor Documentation

Face()

Face::Face

(

)

Constructor.

Construct a new Face:: Face object It initializes the tcp communcations and the asychronous functionalities. The indices of the face mesh are stored in the "vertices_connections.csv" and they are converted to an Eigen::MatrixXi.

Definition at line 9 of file face.cpp.

{
    // Get absolute file path
    std::filesystem::path absolute_path = std::filesystem::current_path();
 
    // Define absolute path of hand configuration file
    m_vertices_connections_abs_path =
        absolute_path / m_vertices_connections_rel_filename;
 
    // Read indices matrix
    Eigen::MatrixXd tmp_faces =
        Utils::load_matrix_data(m_vertices_connections_abs_path);
    m_surf_indices = tmp_faces.cast<int>();
 
    // Setup communication with python server 
    setup_tcp_communication();
 
    // Initialize asychronous function
    m_future_fun = std::async(&Face::incoming_data_callback, this);
 
    // Initialize volume vertices
    m_vol_mesh_vertices = Eigen::MatrixXd::Zero(m_vertices_num, 3);
 
    // Set rotation matrix
    m_rot_mat = EulerRotations::rotation(0, M_PI, M_PI);
 
}

Here is the call graph for this function:

Member Function Documentation

get_surface_indices()

Eigen::MatrixXi Face::get_surface_indices ( void  )

inline

Get the surface indices of the face mesh.

Returns

Eigen::MatrixXi The surface indices matrix.

Definition at line 43 of file face.h.

{ return m_surf_indices; } 

get_vertices()

Eigen::MatrixXd Face::get_vertices

(

void

)

Get face vertices.

It is the point of entry that feeds the animation loop with the face vertices data. It also manages the asynchronous threading.

Returns

Eigen::MatrixXd The face vertices.

Definition at line 41 of file face.cpp.

{
    // Set termination flag for asychronous function
    m_return_value = 1;
 
    // Get vertices
    Eigen::MatrixXd vertices =  m_future_fun.get();
 
    // Reinitialize async function
    m_return_value = 0;
    m_future_fun = std::async(&Face::incoming_data_callback, this);
 
    return vertices;
}

Here is the call graph for this function:

incoming_data_callback()

Eigen::MatrixXd Face::incoming_data_callback ( void  )

private

Read incoming data.

This is the callback function for reading asynchronously the incoming data (vertices) from the server. The data is sent as a json file, so a parser is also used for reading the x, y and z coordinates of each vertex of the face mesh.

Returns

Eigen::MatrixXd The vertex pointcloud

Definition at line 63 of file face.cpp.

{
    while(!m_return_value)
    {
        
        // Initialize buffer
        char buffer[m_buffer_size] = {0};
 
        // Read value
        int val_read = read(m_socket, buffer, m_buffer_size);
 
        // Correct parsing flag         
        bool correct_parsing;        
 
        // Initialzie volume mesh
        Eigen::MatrixXd vol_mesh_vertices;
  
        // Initialize point cloud
        nlohmann::basic_json<>::value_type x_pc, y_pc, z_pc;
 
        try
        {
            // Parse json file 
            nlohmann::json incoming_data = nlohmann::json::parse(buffer);
 
            // Pointcloud x
            x_pc = incoming_data["x"];
            y_pc = incoming_data["y"];
            z_pc = incoming_data["z"];
        
            vol_mesh_vertices.resize(x_pc.size(), 3);
        
            // Check if the parsing is correct
            correct_parsing = (x_pc.size() == m_vertices_num);
        }
        catch(nlohmann::json::parse_error& e)
        {
            // Parsing incorrect
            correct_parsing = 0;
        }
        
        if (correct_parsing) 
        {
            for (size_t i = 0; i < x_pc.size(); i++)
            {
                vol_mesh_vertices(i, 0) = x_pc[i];
                vol_mesh_vertices(i, 1) = y_pc[i];
                vol_mesh_vertices(i, 2) = z_pc[i];
            }
 
            // Rotate and scale vertices
            m_vol_mesh_vertices = (m_scale_mat * m_rot_mat *
                vol_mesh_vertices.transpose()).transpose();
 
            // Find centre of geometry
            double x_cg = m_vol_mesh_vertices.col(0).sum() / m_vertices_num;
            double y_cg = m_vol_mesh_vertices.col(1).sum() / m_vertices_num;
            double z_cg = m_vol_mesh_vertices.col(2).sum() / m_vertices_num;
 
            // Reset face offset
            Eigen::Vector3d face_offset = m_face_offset +
                Eigen::Vector3d({x_cg, y_cg, z_cg});
 
            m_vol_mesh_vertices -= (translation_matrix(face_offset,
                m_vertices_num));
        }
    }
 
    return m_vol_mesh_vertices; 
}

Here is the call graph for this function:

Here is the caller graph for this function:

setup_tcp_communication()

void Face::setup_tcp_communication ( void  )

private

Setup communication with python server.

Since the libraries for face landmarking are all written in Python, we establish an Inter-process communication (IPC), to get all the data in our C++ program. In this case the IPC is implemented with the help of sockets. The python side sets up a server and it transmits all the data to our C++ client. This method sets the tcp communication and initializes the socket and the client that listens to incoming data.

Definition at line 143 of file face.cpp.

{
    // Sock addr struct
    struct sockaddr_in serv_addr;
 
    if ((m_socket = socket(AF_INET, SOCK_STREAM, 0)) < 0)
    {
        printf("\n Socket creation error \n");
    }
 
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(m_tcp_port);
    
    // Convert IPv4 and IPv6 addresses from text to binary form
    if(inet_pton(AF_INET, m_local_ip_address.c_str(), &serv_addr.sin_addr)<=0)
    {
        printf("\nInvalid address/ Address not supported \n");
    }
 
    if (connect(m_socket, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0)
    {
        printf("\nConnection Failed \n");
    }
}

Here is the caller graph for this function:

translation_matrix()

Eigen::MatrixXd Face::translation_matrix	(	const Eigen::Vector3d &	offset,
		size_t	vert_num
	)

This function offset a set of vertices by a given offset.

This is not a homogenous transformation matrix. Instead it is used for shifting a matrix of vertices (as defined by libigl) by a given offset.

Parameters

offset	The offset to shift the matrix of vertices.
vert_num	The number of matrices.

Returns

Eigen::MatrixXd The output translation matrix.

Definition at line 176 of file face.cpp.

{
    // Initialize matrix 
    Eigen::MatrixXd t_mat = Eigen::MatrixXd(vert_num, offset.rows());
 
    // Ones vector
    Eigen::VectorXd ones_vec = Eigen::VectorXd::Ones(vert_num);
 
    for(size_t i = 0; i < offset.rows(); i++)
    {
        t_mat.col(i) = offset(i) * ones_vec;
    }
 
    return t_mat;
}

Here is the caller graph for this function:

update()

void Face::update

(

void

)

Member Data Documentation

m_buffer_size

const int Face::m_buffer_size = 40000

private

Buffer size for reading the vertices message. The number of vertices is constant and this buffer size is enough.

Definition at line 79 of file face.h.

m_face_mesh_scale

double Face::m_face_mesh_scale = 6.0

private

Face scale matrix.

Definition at line 104 of file face.h.

m_face_offset

Eigen::Vector3d Face::m_face_offset = Eigen::Vector3d({0.0, -2.0, 0.0})

private

Definition at line 105 of file face.h.

m_faces

Eigen::MatrixXi Face::m_faces

private

Face landmarking indices matrix.

Definition at line 59 of file face.h.

m_future_fun

std::future<Eigen::MatrixXd> Face::m_future_fun

private

Future function handle.

Definition at line 89 of file face.h.

m_local_ip_address

std::string Face::m_local_ip_address = "127.0.1.1"

private

Start tcp client at this ip adress (localhost).

Definition at line 66 of file face.h.

m_return_value

bool Face::m_return_value = 0

private

Termination flag for callback function.

Definition at line 92 of file face.h.

m_rot_mat

Eigen::Matrix3d Face::m_rot_mat = Eigen::Matrix3d::Identity()

private

Face rotation matrix.

Definition at line 101 of file face.h.

m_scale_mat

Eigen::Matrix3d Face::m_scale_mat

private

Initial value:

=
        m_face_mesh_scale * Eigen::Matrix3d::Identity()

Definition at line 106 of file face.h.

m_socket

int Face::m_socket = 0

private

Socket handle.

Definition at line 73 of file face.h.

m_surf_indices

Eigen::MatrixXi Face::m_surf_indices

private

The surface element indices of the incoming mesh.

Definition at line 85 of file face.h.

m_tcp_port

const int Face::m_tcp_port = 5052

private

Tcp port for socket communication (this might need to be changed if it already occupied.)

Definition at line 70 of file face.h.

m_vertices_connections_abs_path

std::filesystem::path Face::m_vertices_connections_abs_path

private

Absolute name of vertices connection files.

Definition at line 52 of file face.h.

m_vertices_connections_rel_filename

std::string Face::m_vertices_connections_rel_filename

private

Initial value:

=
        "share/vertices_connections.csv"

Relative name of vertices connection files.

Definition at line 55 of file face.h.

m_vertices_num

const int Face::m_vertices_num = 468

private

Number of vertices provided by the mediapipe. This is constant.

Definition at line 82 of file face.h.

m_vol_mesh_vertices

Eigen::MatrixXd Face::m_vol_mesh_vertices

private

Volume mesh data.

Definition at line 98 of file face.h.

---

The documentation for this class was generated from the following files:

• include/face.h
• src/face.cpp