Smartwatches can recognize a range of hand motions and identify what the wearer is doing, scientists have found, paving the way for novel health-related apps that could monitor activities such as brushing teeth or smoking a cigarette.
Researchers at Carnegie Mellon University in the US have used a standard smartwatch to figure out when a wearer was typing on a keyboard, washing dishes, petting a dog, pouring from a pitcher or cutting with scissors.
By making a few changes to the watch’s operating system, they were able to use its accelerometer to recognise hand motions and, in some cases, bio-acoustic sounds associated with 25 different hand activities at around 95 per cent accuracy.
Those 25 activities are just the beginning of what might be possible to detect.
“We envision smartwatches as a unique beachhead on the body for capturing rich, everyday activities,” said Chris Harrison, assistant professor at Carnegie Mellon.
“A wide variety of apps could be made smarter and more context-sensitive if our devices knew the activity of our bodies and hands,” Harrison said in a statement.
Just as smartphones now can block text messages while a user is driving, future devices that sense hand activity might learn not to interrupt someone while they are doing certain work with their hands, such as chopping vegetables or operating power equipment, said Gierad Laput, a PhD student at Carnegie Mellon.
Sensing hand activity also lends itself to health-related apps — monitoring activities such as brushing teeth, washing hands or smoking a cigarette.
Hand-sensing also might be used by apps that provide feedback to users who are learning a new skill, such as playing a musical instrument, or undergoing physical rehabilitation.
Apps might alert users to typing habits that could lead to repetitive strain injury (RSI), or assess the onset of motor impairments such as those associated with Parkinson’s disease.
Researchers began their exploration of hand activity detection by recruiting 50 people to wear specially programmed smartwatches for almost 1,000 hours while going about their daily activities.
Periodically, the watches would record hand motion, hand orientation and bio-acoustic information, and then prompt the wearer to describe the hand activity — shaving, clapping, scratching, putting on lipstick etc.
More than 80 hand activities were labelled in this way, providing a unique dataset.
For now, users must wear the smartwatch on their active arm, rather than the passive (non-dominant) arm where people typically wear wristwatches, for the system to work. Future experiments will explore what events can be detected using the passive arm.
“The 25 hand activities we evaluated are a small fraction of the ways we engage our arms and hands in the real world,” Laput said.
Future work likely will focus on classes of activities — those associated with specific activities such as smoking cessation, elder care, or typing.