A Fuzzy Supervisory Framework for Real-Time Optimization of Robot Output and LLM Performance in HRI

Human-robot interaction plays a vital role in pushing the capabilities of socially interactive robots by enabling them to deliver content with high emotional intelligence. This research focuses on a supervisory fuzzy framework for constantly evaluating and improving the content delivered by the robot utilizing multimodal inputs and advanced intelligent algorithms. The main reason for using fuzzy logic is that it mimics human decision-making by providing a percentage-based measure of closeness. In this project, ARI Robot is being used with an LLM integration, which enables the user to communicate with the robot. Different algorithms were integrated for the classification of multimodal inputs, BERT (Bidirectional Encoder Representations from Transformers) for the classification of content, Wav2Vec 2.0 for classifying the tone of the user while interacting with the robot, and OpenFace for classifying the facial expression of the user. All of these inputs are then supervised by a fuzzy system with predefined rules to evaluate the content delivered and provide feedback for refinement. The proposed framework ensures an overall evaluation of content delivery, providing intelligent feedback to the ARI robot to improve interaction quality. By integrating these advanced models with fuzzy logic, the system mimics human-like judgment in assessing the interaction of verbal and non-verbal indications, making the way for emotionally intelligent robots in a social world.