A team from Saarland University has developed a mixed-initiative, high-resolution robotic printer that allows the user to switch seamlessly between human and robotic intervention. Raspberry Pi powers the robot’s user interface. The team hopes their work will have significant implications for the future of mixed-initiative creativity.
Team member Narges Pourjafarian got in touch to tell us about RoboSketch, which they presented at CHI 2023, a conference on human factors in computing systems which was held in Hamburg, Germany this year.
What does “mixed-initiative” mean?
You might not have come across the term “mixed-initiative” before – most of us hadn’t. In this context, it means that sometimes a person is using the machine as a tool (i.e. taking the initiative about what it should print), sometimes the machine is acting autonomously without a human user, and sometimes there is a mixture of user initiative and machine initiative.
Rethinking personal fabrication
Another term to define: “personal fabrication” means making something yourself instead of buying a commercially manufactured object, and using digital technology like computers and 3D printers to do it. Until now, personal fabrication has mostly focused on handheld tools that the user must physically control, with machines like laser cutters and 3D printers automating parts of the process that don’t need manual intervention.
Although interactive digital fabrication has already been explored as a middle ground, existing systems are limited by a very binary choice between the parts of the process that must be done by a human and the parts that are always done by a machine. Narges’s team wanted to develop something flexible that allowed for more creativity and improvisation between human-led and machine-led tasks. The goal was to offer a continuum from manual and assisted to autonomous fabrication, with a seamless transition between the different modes.
Meet RoboSketch
The team illustrated how this fluid human/robot fabrication utopia could work by building a mixed-initiative physical sketching machine. RoboSketch is a robotic printer for physically sketching high-resolution and large-scale prints.
It can be used in three ways, ranging from total human control to totally machine-led control:
- 100% human-led use (manual mode)
You can use RoboSketch as a handheld tool for freehand sketching. - 50/50 human and computer use (assisted mode)
The printer can provide interactive assistance while a human is sketching. For example, you can manually drag the robot so you’re freehand sketching, but the robot will help you keep a completely straight line. - 100% machine-led use (autonomous mode)
Completely computer-generated sketches can be produced by the printer.
The sketching machine is designed to seamlessly transition between each of those three modes with the tap of a button. It enhances the drawing experience by allowing the user to sketch freehand and then let the machine take over to do things like colour correction or enhancing sharp lines — simple jobs on which you don’t need to waste the brilliant mind of a freehand artist. If the robot isn’t sure what it should be doing, or it runs out of paper to draw on, it automatically switches to manual mode so a human can intervene to help.
How is it made?
RoboSketch is a wheeled robot with a Raspberry Pi 4 telling a high-resolution mobile printer what to do. A Raspberry Pi camera and ultrasonic sensors on the front let the robot see where it’s going, and an LCD screen communicates what is happening to the human user. The paintbrush-shaped joystick on the top is designed for a human user to grasp and use for manual sketching and control of the machine.
Conference resources
You can read the entire paper Handheld Tools Unleashed: Mixed-Initiative Physical Sketching with a Robotic Printer, which should show up in the ACM digital library very soon. That online resource also has links to projects in areas including data-driven storytelling, fostering meaningful virtual connections, games beyond gaming, IoT for social purposes, and lots more. While I don’t often like conferences due to all the speaking to people and having to sit properly, CHI 2023 sounded pretty good.
