Postdoctoral Research
Evaluation of Socioenactive Systems
This project aims to build a conceptual framework based on experimentation with different scenarios, which provides tools for those involved with the design and development of Socioenative systems to produce technological solutions that effectively take into account the cultural context of those who participate in it, including their differences, needs, preferences, skills and values. This framework will articulate the use of new technologies of tangible, wearable and natural interfaces with the concept of pragmatic web for socially responsible design in different contexts. The knowledge produced through the project has the potential to influence the way interactive systems are understood and designed.
Accessibility in Socioenactive Systems
This project aimed to conduct a detailed investigation proposing a systemic approach to fostering accessibility in socioenactive systems. In our methodology, we firstly propose the effective participation of users in experiments to analyze various aspects related to accessibility in the use of Socioenative systems. In a second moment, it is proposed to define a set of original recommendations based on new techniques and methods that guide the application of the recommendations in the design of interactive mechanisms accessible in socioenactive systems. This plan proposes to carry out experimental validations to evaluate the methods developed and prototypes implemented in scenarios of practical applications in Socioenative systems.
Affect in Interaction Design in Socioenactive Systems
This project aimed to investigate and propose, based on the concept of Affectibility, a conceptual framework and methodology to assist designers of socioenactive systems.workshops Semi and Socially Conscious Design artifacts will guide the development of the product resulting from this research: a technological artifact for a museum in the context of a socioenactive system. Speech Act Theory, Image Act Theory, Organizational Semiotics and Universal Design will be the theoretical basis for this research and Socially Conscious Design its methodological basis.
SmartAge: Affect in Interaction Design and Technology for Elderly
This research project is located in the area of Human-Computer Interaction and aims to investigate the role of affect in the interactions of the elderly with contemporary technology and how the latter can be useful for the elderly population. In order to enhance the benefits that technology can bring to the well-being and health of users, providing greater autonomy and self-esteem, this research sought to propose and experiment with scenarios with new technologies for users over 65 years of age.
PhD Theses
A Framework for Evaluation of Equitable Access in Ubiquitous Environments
Ubiquitous computing has brought a new reality for our daily routines: wearables, tangibles, diverse sensors, and the Internet of Things (IoT) are some examples that enlarge the interaction of people-technology possibilities. Providing and evaluating equitable access in these installations is a research challenge. During my Master research we developed an evaluation instrument of equitable access named UbiAccess The instrument was based on two formal standards: W3C-WCAG (Web Content Accessibility Guidelines do World Wide Web Consortium) and the Universal Design Principles (UD). We applied UbiAcces in remote and in person scenarios. Our research also investigated a case of an Autistic Spectrum Disorder (ASD) child and another including a child diagnosed with Attention Deficit Hyperactivity Disorder (ADHD). The instrument was evaluated for other researchers. Our inicial doctoral proposal is to extend UbiAccess in the lack of coverage revealed during the studies and adding the Social area with its recommendations. UbiAccess will be part of a conceptual and practical Framework we aim to develop. This Framework should be able to present what is the dimension of equitable access in ubiquitous computing environments. In this sense, the Framework will contribute with a better understanding of the access evaluation in these environments.
A Hybrid Interface Model for Studying Socio-emotional Phenomena
The phenomenon of emotion contagion is a perspective where one individual acts upon one or more individuals influencing his/her emotional state. Behind our emotional expressions, a complex circuit is triggered in our mind and body. It involves neurological and physiological activities. The latest technological advances in literature show studies using Electroencephalogram (EEG), and Electrocardiogram (ECG) to collect neurological and physiological information to analyze, find patterns, and get inferences (Hybrid BCI) on people's emotions. Still scarce in literature are systems that consider not only the neurophysiological aspects in emotional experiences but also the environment composed by things, and other humans with their perceptual, cognitive, and sensorimotor information. Our research proposal involves studying the phenomenon of contagion from a Socioenactive systems perspective (a novel approach) which considers all the aforementioned aspects in the relationship among human's mind, body and the environment as a unity in the social world. We aim to revisit the emotion contagion phenomena to promote socio-emotional interactions. Our study explores Artificial Intelligence algorithms and methods to recognize patterns from data collected by the system. The hybrid interface consists of diverse input signals from the brain and from other physiological data, within wearable and ubiquitous systems. In this case each system in the hybrid paradigm plays different roles. In this research a hybrid interface model will be investigated and developed for these systems. Our research will be evaluated via case studies in which participants are invited to use and assess the proposed system.
Aspects of Self-Esteem in Socioenactive
The existence of new technologies based on sensors and actuators has enabled new modalities of human-computer interaction that are more physical and immersive, in which human and technological processes are more intertwined. However, if on the one hand this technological evolution and the new means of interaction have facilitated people's lives, on the other hand, several studies report that it also has the potential to provide experiences that negatively alter the psychological states of users, even affecting their self esteem. Self-esteem refers to a person's evaluation of himself or herself in such a way that it expresses an attitude of approval, or disapproval, and indicates the extent to which a person believes he or she is capable, meaningful, worthy, and successful in relation to their abilities, skills, social relationships and physical appearance, for example. Self-esteem is an individual concept, but it is developed from a person's involvement with society. The dynamic body-technology coupling of ubiquitous and pervasive technologies can be seen from the perspective of native systems, which are not based on a classic user interface, nor on a goal-based mode of interaction; the interaction is driven by the spatial presence and body involvement of a human agent without the assumption of an explicit control of the system. In this context, this work aims to propose and study whether and how a socioenactive system can positively affect the self-esteem of people, especially children with craniofacial anomalies.
Development of Computational Thinking Supported by Tangible Artifacts
…
Socioenactive Interaction and the Design of Creative Learning Environments for Children
In an increasingly technological and constantly changing world, it is increasingly important to encourage the development of skills such as creativity, teamwork and entrepreneurship. Creative people are curious and like to explore ideas, so they are more innovative in problem solving. An emerging concept is that of “creative learning”, which seeks to understand, experiment and define how methodologies and learning environments are capable of promoting creativity. Therefore, we need to explore new strategies, techniques and tools that help people develop their creativity and make sense of what they are learning. As technology advances, interaction design has required a continual overhaul of its scientific foundations and practice. The classic simplification of two systems in technological interaction, that is, man and machine, is in conflict with the concepts brought by current ubiquitous technologies. One such concept is enactive systems. Enactive systems are information systems that feature a strong and dynamic mind-technology coupling. Extending the concept of enactive systems, the socioenactive approach considers the human as a participant in the enactive system. A socioenactive system encompasses two main aspects: the enactive systems and the sociocultural aspects that involve the phenomena of the enactive experience with the technological system. The experiences that each person has, the social interactions between peers and the emotional state when using the system are some examples of the sociocultural considerations to consider for the design process. This thesis aims to explore and understand the potential of using socio-enactive systems as a support in activities that promote the development of creative thinking in children in the educational context. To achieve our goal, we performed the design and development of the tangible programming environment called TaPrEC+mBot and the wearable artifact called GloveBot, based on the socioenactive approach. Both environments were evaluated in real educational scenarios with the participation of children and teachers in the context of creative learning. The analysis of the results showed that the socioenactive interaction benefited and promoted the children's creative learning, supporting the cognitive process, promoting their initiatives and facilitating communication and collaboration among the children.
A Design Process of Communication Blocks and Components for IoT Systems: promoting People Interaction
With increasingly smaller technologies we can compose an IoT (Internet of Things) system that is not something new, but exploring ideas that were proposed in the 90s. Thus, companies and researchers begin to incorporate computing devices into objects, clothes, using communication protocols to exchange information. In the context of IoT, the forecast is that we will exceed 200 billion intelligent and interconnected “things” (entities) worldwide by the end of 2020. By 2022, almost half of the Internet data flow will be generated by the IoT. It is a time of market and research opportunities to outline the IoT. An IoT system is configured within an environment where other systems coexist, including the social system. The design of (basic) components that will integrate objects to be part of an IoT system is a challenge for designers and this extends to the creation of scenarios in which the components will be configured, either as part of objects, or as clothing or just like sensors in an environment. In an IoT system, part of the interactions no longer have the traditional forms of data inputs and outputs. It means that graphical interfaces, keyboards and mice lose space for new inputs and the availability/perception of results as output. An IoT system is classified as a socio-technical system, which is characterized as a complex system. In an IoT project, its basic components are organized as a network that communicates with each other. The system is maintained by the communication between its components that are part of objects and managed by one or more distributed software. To propose a solution to a problem with an IoT system, it will be necessary to use a process that starts with the understanding of a problem with the interested parties and goes through some steps that lead designers to propose basic components that will integrate objects to be organized in the environment in which the system will operate. Component definitions will help in proposing some kind of software solution for managing message exchanges via network communication protocols that connect the components of the IoT system. Socially conscious design starts from the assumption that a solution must be built with and by the stakeholders, seeking to articulate the problem in a way that makes sense to everyone. The process for the development of components that will compose an IoT solution needs to take into account that a software layer (management and communication solutions) and a hardware layer (hardware and network architecture) will be required. We still have as a critical and relevant point in the proposal of an IoT solution the incorporation of basic components to objects that will compose the new forms of interaction of people with the system and the system with people. In order to be able to work with all these axes (hardware, software, interaction), we will evolve the thesis in a way that the practices will allow us to compose environments to test the interaction, especially in the hospital and educational context. Thus, workshops with stakeholders will allow us to evolve the components, organizing and maturing the process.
On the Fly Design: Concept and Platform
…
Arte Factus: Study and Socially Aware Co-design of Socioenactive Digital Artifacts
This thesis investigates the concepts of enactive and socioenactive systems through the co-design of interactive art and installations. A theoretical-methodological framework called "Arte Factus" is proposed, used in three design studies reported in this thesis: InterArt, InstInt and InsTime. These studies involved the participation of 105 undergraduate and graduate students in Computer Science and Computer Engineering in the co-design of 19 facilities. The process involved the use of pervasive Do-It-Yourself (DIY) technology, and some of these installations were studied in situated practice workshops that took place in educational settings (school and exploratory science museum).
Tangible Computational Systems and Design of Communication in Inclusive Educational Contexts
The most used means of communication by human beings in their natural development is the oral language. Speech difficulties demand alternatives of different nature, including technological artifacts. Alternative and Augmentative Communication (AAC) is the denomination for a set of tools and strategies that one can use as a complement and/or replacement of speech to compensate for speech difficulties. When a student does not speak or the interlocutor does not understand what is being communicated, communication alternatives that allow a barrierless interaction inside and outside the school environment should be used, such as the AAC. In addition to being able to express themselves through speech, the students should be able to express their affective states, an essential aspect of the teaching and learning relationship. The “Grand Research Challenges in Human-Computer Interaction in Brazil” suggests that innovative technologies should be “used for the construction of systems that can be both generalized to multiple devices and at the same time specialized for different users with different needs”. Information and Communication Technologies have the potential to play an important role in promoting inclusion, improving the use of pedagogical materials and adding new social-educational resources. Nevertheless, tangible interaction technologies have not yet been widely used to meet the communication needs in inclusive environments. This thesis proposes communication systems focused on inclusive educational environments, by exploring and reconceptualizing the use of contemporary technologies (e.g. Radio Frequency Identification, Bluetooth, Smartphones) for tangible interaction. The research process of this work involved activities based on Semioparticipatory Workshops with interested parties, in several design situations, including a real context of inclusive schools. Our objective was to investigate the potential of computational technologies for the Communication scenario (Alternative, Augmentative and Affective) and propose environments based on such technologies (via tangible and touch devices, among others) that enable the communication by people and between people in inclusive educational spaces. Considering the intrinsic relationship between emotions and learning, different types of artifacts were proposed, developed and experienced: Tan2Talk, SAMobile, SAMDesk, TangiSAM, CardSAM, SAMCurve, SAMLight. Tan2Talk is a tangible environment to support alternative and augmentative communication, proposed as a supplement to the communication of students who do not make themselves understood through speech and are enrolled in regular schools (inclusive education). The SAMobile and SAMDesk systems use tangible computational artifacts (named TangiSAM and CardSAM) to enable the self-expression of affective states. SAMCurve and SAMLight were developed to understand the students' emotional experiences along different activities. The former provides the analysis of their changes in affective states over time. The latter uses a display of LEDs as a visual feedback of the affective state of different groups of students. By using the systems in inclusive educational environments, TangiSAM and CardSAM have shown effective in providing students with the expression of their affective states. Furthermore, they led to a discussion on the students¿ affective states and a reflection on the teachers' practices. The diversification of the platforms supporting these artifacts, via SamDesk and SAMobile software enable the schools to adapt the solutions to their technological infrastructure. The SAMCurve and SAMLight artifacts allowed the analysis and awareness of the teachers and the pedagogical coordination regarding the students' affective states. The results of the implementation of Tan2Talk pointed out that the children were able to communicate through the artifacts, leading to an improvement in their communication skills via idea association and social interaction. The computational systems of tangible interaction to support alternative, augmentative and affective communication have shown positive results in terms of acceptance, arousal, fun and communication capacity among the participants. Therefore, the results show their potential to be useful in educational spaces.
Investigating Natural User Interfaces (NUIs) – Technologies and Interaction in the Context of Accessibility
Natural User Interfaces (NUIs) represent a new interaction paradigm, with the promise of being more intuitive and easy to use than its predecessor, that utilizes mouse and keyboard. In a context where technology is becoming each time more invisible and pervasive, not only the amount but also the diversity of people who participate in this context is increasing. In this case, it must be studied how this new interaction paradigm can, in fact, be accessible to all the people who may use it in their daily routine. Furthermore, it is also necessary to characterize the paradigm itself, to understand what makes it, in fact, natural. Therefore, in this thesis we present the path we took in search of these two answers: how to characterize NUIs in the current technological context, and how to make NUIs accessible to all. To do so, first we present a systematic literature review with the state of the art. Then, we show a set of heuristics for the design and evaluation of NUIs, which were applied in practical study cases. Afterwards, we structure the ideas of this research into the Organizational Semiotics artifacts, and we obtain insights into how to design NUIs with Accessibility, be it through Universal Design, be it to propose Assistive Technologies. Then, we present three case studies with NUI systems which we designed. From these case studies, we expanded our theoretical references and were able to, finally, find three elements that sum up our characterization of NUI: differences, affordances and enaction.
An Analysis of the Role of Computer Systems as Mediators in Scientific Socio-Technical Networks from the Point of View of the Actor-Network Theory
This thesis aims to articulate the theoretical frameworks of the Actor-Network Theory (ANT) and Socially Aware Computing (SAC), allowing, on the one hand, to understand a scientific community from an organizational point of view, and on the other, to understand the role that different non-human elements play in mediating interests between scientists. A method was proposed to clarify the interests involving the development and use of software, and a mechanism for visualizing social groups that allows a qualitative interpretation of the joint participation of people and non-human elements. The proposal was applied to the study of a software aimed at managing research production data, mapping different scientific communities through their bibliographic productions, and building a system to support the scientific process of literature review, Quid.
The Sociotechnical Process Model for the Design of Interactive Digital TV Applications
In Brazil and in other countries, Interactive Digital TV (iDTV) faces inherent problems in its development (eg, users' lack of experience to interact with television content, user passivity inherited from analog TV and remote control limitations). At the same time, disregarding aspects of human-computer interaction can generate applications destined to failure, given the strong interactional appeal that such applications must have. This context suggests the need to create a design process for iDTV applications that aggregates both the dynamic needs of a television production chain, the fundamentals of a software process, as well as aspects of human-computer interaction. In this sense, this thesis proposes a process model that is theoretically based on Socially Conscious Computing, which is based on Organizational Semiotics and Participatory Design.
Master's Thesis
Exploring Imagination in Socioenactive Interactive Installations (qualification)
The use of technology has expanded beyond conventional environments aimed at performing well-defined tasks . In the current scenario, ubiquitous and pervasive technologies have created new forms of interaction, which expand into the physical environment. These new types of interactions challenge traditional Human-Computer Interaction methods for designing interactive systems. This requires new ways of looking at interaction design. This master's proposal aims to study the design of interactive installations that explicitly explore the imagination of the participants in a socioenactive experience. Initially, we will conduct a systematic mapping of the literature to know in depth the use of imagination in interaction design and in the context of socioenactive systems. We will then conduct an exploratory analysis in order to understand how existing interactive installations address imagination in interaction. Based on this information, we will devise recommendations for the design of facilities that want to explore children's imaginations. Finally, we will lead the design and materialize the creation of an interactive installation that develops children's imagination in a museum environment.
Equitable Access in Ubiquitous Computational Environments: Study and Evaluation Instrument
Ubiquitous computing has brought new challenges to the design of people-technology interaction scenarios. Ubiquitous computing environments may contain sensors and actuators, tangibles and other interconnected technologies, resulting in a diversity of interaction possibilities. Despite research considering formal accessibility standards, related works pay little or no attention to investigate ubiquitous computing. We, in contrast, investigate means of analyzing equitable access in ubiquitous computing environments based on two formal standards: W3C-WCAG (Web Content Accessibility Guidelines do World Wide Web Consortium) and the Universal Design Principles (UD). As a result, we generated an evaluation instrument called UbiAccess. We applied the instrument in case studies, including a simulation of a remote interaction scenario due to the Covid-19 pandemic. Our research also investigated a case of an Autistic Spectrum Disorder (ASD) child and another including a child diagnosed with Attention Deficit Hyperactivity Disorder (ADHD).
A socially-aware persuasive system design framework
Human beings are facing a variety of problems, one of which is improving our habits in relation to healthy behaviors that help to achieve our well-being. Our question in this dissertation is: how can persuasive information systems and the area of human-computer interaction help improve our relationship with healthy behaviors for well-being? On the other hand, many designers, developers, and information systems design stakeholders try to create persuasive technologies, albeit often without formal support of persuasive design approaches. The design of persuasive information systems is assuming greater importance, especially those focused on sustainability. Furthermore, research into design methodology is one of the most challenging issues in the field of persuasive technology. This research intends to discuss and expand the persuasive design frameworks, considering four main aspects: social awareness in the persuasive systems design process; inclusion of persuasive principles not only for web apps and mobile devices, but also for pervasive apps; the extension of ethical aspects in the design of persuasive information systems; and the inclusion of stakeholder behavior to influence the achievement of target behavior. Considering these key elements, in this work, we propose a Framework for designing socially conscious persuasive systems; this Framework is composed of a process with artifacts, and a tool as a guide to design persuasive systems. To validate the proposed Framework, a case study was carried out in which we tried to motivate and convince people to eat healthy foods using an information system, and on the other hand, influence restaurant owners to offer healthier foods. The case study presented illustrates the use of the Framework. The results suggest potential benefits of the proposed structure. The case study participants were a software design consulting firm, the health food restaurant and we as the researchers of this study. The Framework mentioned above has been integrated into the OpenDesign platform (a socially conscious open design platform) to be available online as a support tool for a wide audience. This support tool is composed of the Framework with artifacts and process to support the first steps in the systems design phase. The support tool was designed, developed, evaluated with another case study focused specifically on the support tool. A research group composed of designers, researchers, experts in human-computer interaction, software engineering and enactive approaches evaluated the support tool. Persuading and influencing people to achieve target behavior through technologies can be made easier using the support tool.
Guidelines for evaluation of systems in socioenactive scenarios
Contemporary computing environments intensively explore the use of various devices and sensors. These environments foster the involvement of users' body by allowing a dynamic interaction with the technology. This research is inserted in the context of a new class of systems called Socioenative Systems, in which the interaction is guided by feedback cycles between the participants and spatially present interfaces, which explore sensors and data analysis in physical and immersive interactions. In this context, the way systems design and evaluation are conducted affects the easiness of use. Despite technological advances and complexity of these systems, we need access for all people regardless of their characteristics and/or limitations. To this end, the evaluation process of design proposals plays a key role, although literature offers a small number of methods and instruments, mainly aimed at the inspection of traditional computer systems, e.g., Web systems. This Master dissertation studies an instrument to support designers to evaluate/inspect proposals for Socioenactive Systems. We aim to achieve means that assist designers in the observation and analysis of characteristics under different dimensions that define Socioenative Systems. Based on the foundations of Socioenactive Systems, we propose a set of recommendations (guidelines) organized in a methodology to guide the evaluation of Socioenactive Systems. In our approach, the principles of Universal Design and Heuristics of Natural User Interfaces are combined and mapped in the context of our instrument. We propose a prototype of how the instrument can be integrated with the OpenDesign platform. We conduct a case study with three Socioenactive Systems scenarios, in which designers utilize our instrument to inspect already designed systems. Results indicate that the instrument developed favors an analysis of proposals for Socioenactive Systems whereas allowing designers to think about universal access in these systems. This work contributes to Socioenactive Systems to move towards design for all.
Design and development of a tangible tabletop enhanced with embedded-technology objects towards a socioenactive experience
The environments constructed from contemporary technology (interactive, tangible and ubiquitous) are still challenging the mainstream HCI methods for designing computational systems, demanding new ways of considering the interaction design and its evaluation. Emerging ideas of enactive cognitive sciences can provide new perspectives to create new ways of interacting with technology, considering both the environment and the human experience as aspects of embodiment. Technologies such as tangible tabletops offer opportunities to exploit embodied forms of interaction foregrounding the role of the body in interaction and learning. Furthermore, the low cost, quantity and internet capabilities of devices offer opportunities to make physical sensing accessible for incorporation into the design of tangible tabletops, transforming physical actions into physical effects in the environment. In this dissertation, we investigate the (socio)enactive experience of interaction in the context of the ubiquity of computational systems by designing and developing TangiTime, a tangible tabletop educational exhibit enhanced with embedded-technology objects for experiencing the ‘deep time’ concept. In this dissertation work, we present the design process and the artifacts implementation. Moreover, we present three different contexts in which the exhibit was put into usage: a) a pilot use to get the first impressions of the user experience with it, b) a public exhibition to get a ‘real’ feeling of people interacting with the installation and to analyze the (socio)enactive aspects involved, and c) a workshop within a science exploratory museum to explore the role of the embedded-technology objects to support engagement and learning in the exhibit. Our results suggest that the composition of ubiquitous technologies potentializes the creation of a (socio)enactive experience by fostering multimodal perceptions and the engagement of the participants. Moreover, interaction with embedded-technology objects creates new ways of experiencing tangible tabletops toward a (socio)enactive experience.
Design and Development of Scenarios for Ubiquitous Computing Systems in the User's Perspective
This dissertation sought to investigate ubiquitous computing systems, building, for that purpose, scenarios and artifacts that used this emerging technology and that consider the user's perspective. In this work, we built four scenarios, three in the field of education, which were tried through case studies and evaluated with teachers and students from an educational institution; and one regarding accessibility that was evaluated in a case study with specialists in Human-Computer Interaction. We also investigated these four scenarios from the perspective of the Internet of Things, a concept that refers to systems that would have components connected to each other and to the Internet, sharing information. In this context, we also proposed enrichments in the design of these scenarios, so that they could evolve as scenarios of Internet of Things systems.
A study on the Evaluation of User Experience in Pervasive, Ubiquitous and socioenactive Scenarios
The omnipresent technological scenario through ubiquitous and pervasive technologies has brought new forms of interaction. These interactions became spontaneous and natural, in the sense of permeating the physical setting. User Experience (UX) brings with it new perspectives for evaluation and is currently widely recognized as an aspect to consider and understand when creating interactive technologies. This dissertation sought to investigate and understand how UX can be affected by these new forms of interaction. Particularly, we investigated the topic by carrying out UX assessments in real scenarios of ubiquitous, pervasive technology use and in scenarios with socioenactive characteristics.
Design and Development of a Low-Cost Tangible Programming Environment for Children
There are many studies that demonstrate that the contact with programming has a positive impact on the development of cognitive and socio-emotional skills in children. Programming has been discussed as a way to stimulate the development of computational thinking in children. However, there are barriers to the learning of programming; one of them is the fact that most existing programming languages are based on abstractions in the form of texts and symbols that are difficult to understand for children. A potentially facilitated way to introduce children to programming is by Tangible Interfaces. The approach of Tangible User Interfaces (TUI) proposes to embed computing elements in concrete materials, creating an educational feature that unites the advantages of physical interaction and multimedia handling provided by technology. Enriching the concrete materials, computational resources can help stimulate the computational thinking, in addition to working several senses (sight, hearing, touch). However, tangible technology is often expensive, delicate and difficult to customize. In this scenario, we propose and present the TaPrEC (Tangible Programming Environment for Children), a low cost tangible programming environment created with resistant material and easy customization, developed to teach basic concepts of programming. The environment allows children to create tangible programs organizing wooden colored blocks similar to puzzle-pieces. In this dissertation we provide technical details of the construction of TaPrEC, its evolution and the main aspects and results obtained during the Case Study conducted to evaluate it with the participation of children and teachers of primary school, which was developed over two semesters in a real educational scenario.