A Carnegie Mellon University researcher has created a system that represents household objects as virtual pets on small displays, using sensors to reflect real-world conditions. Daehwa Kim, a PhD student in Carnegie Mellon’s Future Interfaces Group, presented the project at the ACM Symposium on User Interface Software and Technology in Detroit in early November, receiving two Honorable Mention awards.
The system, called OATs (Object Agent Twins), pairs E Ink displays with sensors tracking moisture, temperature, humidity, light, occupancy and motion. The displays show digital pet avatars whose emotional states change based on sensor data—for example, a plant-shaped character appearing sad when soil moisture is low and becoming animated when watered. Kim says the approach aims to “bring this joy of caring from virtual to physical objects.”
Users configure OATs through a web portal where they name their display and select which sensors to activate for a given object. Google Gemini’s Nano Banana image-generation service creates the visual sprites showing different emotional states. According to Kim, the main hardware module costs around $20.
Kim told Wired her goal was to move AI agent interactions away from smartphone screens and into physical spaces. While she has been encouraged to sell OATs kits online, she has not decided whether to pursue commercialization.