Fingernail camera makes any object a touchpad

Mon, 31 Mar 2008 02:54:40 GMT

Fingernail camera makes any object a touchpad

Flora Graham

Using a simple webcam to observe blood flow in a person's fingers can make objects like this rock respond to touch (Video courtesy Joe Marshall, Nottingham University)

Using a camera to detect blood moving under the fingernail, a new system can make almost any object touch sensitive.

Pressure-sensitive touch interfaces are common, but they usually require a smooth surface, with sensors on or beneath its surface. This is fine for simple and robust devices such as laptop touchpads, but it makes adding touch-sensitivity to irregular or fragile objects practically impossible.

However, a new system developed at the University of Nottingham, UK, can do just that. Unlike other systems, it detects the effect of touch on the finger, not the surface being touched. It works by using a simple video camera to observe the movement of blood under a person's fingernails.

Contrasting colours

When a finger is pressed onto a surface blood shifts away from the top of the nail, making it lighter. That blood concentrates near the nail's base, which gets darker.

The camera monitors that change in contrast and can detect the fleeting change that indicates a single touch. It can also gauge how hard a finger is pressed from the intensity of the colour change.

"When you touch a surface, even lightly, the blood under the nail concentrates in one place. Surprisingly, this is relatively easy to spot with a camera," says Joe Marshall, a researcher in the Nottingham team.

Using an external camera to make objects touch sensitive could allow tactile interaction with new objects, says Marshall. "For example, historic tools in a countryside museum could be touched in order to trigger audiovisual material about their use," says Marshall.

Pet rock

The simplicity and low cost of the technology is another advantage, he adds, because all that is required is an inexpensive webcam and desktop computer.

A video (see right) shows a demonstration of the system – a touch-sensitive rock, which purrs when stroked and growls when grabbed. A webcam inside the rock's cage watches people's fingernails to cause it to react to their touch.

Marshall was impressed by the emotional response that the rock evoked in people who touched it. "It was presented as 'my pet rock' and people seemed to really engage with that," he says. "One person actually asked me if they could take it home."

Making curved or irregular 3D objects respond to touch opens up a whole new area for the way people use touch interfaces, Marshall says. "A lot of what is out there is just taking a computer desktop and saying 'let's put it on a wall or a table,'" he says. "I’m much more interested in interactive projects using objects."

Colour-blind system

Although anything that obscures the user's nails – such as gloves or nail varnish – can cause the system to fail, variations in skin colour between different users do not seem to cause problems. "Surprisingly, people's fingernails are pretty consistent. So it's colour-blind in that sense," says Marshall.

Ming Yang, at Cardiff University, UK, developed a way to make any table touch sensitive using a few microphones (with video). "This is a very interesting method which has never been explored for touch-sensitive applications," he told New Scientist.

Yang sees the simplicity of the system as its main advantage, but adds that simplicity also has drawbacks. "There are a lot of limitations that are inherited from the technology that was adapted for this system," he explains, "I think that reliability may be a big issue due to the low-cost camera that was used."

A paper on the nail-watching system will be presented in May at the Pervasive 2008 conference in Sydney, Australia.

Virtual pets can learn just like babies

Mon, 31 Mar 2008 02:45:34 GMT

Virtual pets can learn just like babies

Celeste Biever, Memphis

"SIT," says the man. The dog tilts its head but does nothing. "Sit," the man repeats.

The dog lies down. "No!" the man admonishes.

Then, unable to get the dog to sit, the man decides to teach it by example. He sits down himself.

"I'm sitting. Try sitting," he says. The dog cocks its head attentively, folds its hind legs under its body and sits. "Good!" says the man.

No, it's not a rather bizarre way to teach your pet new tricks. It is a demonstration a synthetic character in a virtual world being controlled by an autonomous artificial intelligence (AI) program, which will be released to inhabitants of virtual worlds like Second Life later this year.

Novamente, a company in Washington DC which built the AI program that controls the dog, says that the demonstration is a foretaste not just of future virtual pets but of computer games to come. Their work, along with similar programs from other researchers, was presented at the First Conference on Artificial General Intelligence at the University of Memphis in Tennessee earlier this month.

If first impressions are anything to go by, synthetic pets like Novamente's dog will be a far cry from today's virtual pets, such as Neopets and Nintendogs, which can only perform pre-programmed moves, such as catching a disc. "The problem with current virtual pets is they are rigidly programmed and lack emotions, responsiveness, individual personality or the ability to learn," says Ben Goertzel of Novamente. "They are pretty much all morons."

In contrast, Goertzel claims that synthetic characters like his dog can be taught almost anything, even things that their programmers never imagined.

For instance, owners could train their pets to help win battles in adventure games such as World of Warcraft, says Sibley Verbek of the Electric Sheep Company in New York City, which helped Novamente create the virtual pets. "It is a system that allows the user to teach the virtual character anything they want to," he says.

"Pets can be trained to help win battles in adventure games such as World of Warcraft"

So how do these autonomous programs work? Take Novamente's virtual pet, which is expected to be the first to hit the market. One way that the pets learn is by being taught specific tasks by human-controlled avatars, similar to the way babies are taught by their parents.

To do this, the humans must directly tell the pet - via Second Life's instant messaging typing interface - that they are about to teach it a task. When the pet receives a specific command, such as "I am going to teach you to sit", it works out that it is about to learn something new called "sit". It then watches the human avatar and starts to copy some of the things the teacher does.

At first it doesn't know which aspects of the task are important. This can lead to mistakes: the dog lying down instead of sitting, for example. But it soon figures out the correct behaviour by trying the task several times in a variety of ways. The key learning tool is that the pets are pre-programmed to seek praise from their owners, so they can make increasingly intelligent guesses about what they should copy, repeating adjustments that seem to make the human avatar more likely to say "good dog", and avoiding those that elicit the response "bad dog". Eventually, the pet figures out how to sit.

Learning by imitation isn't exactly a new idea. Robots in the real world are still being trained in this way. But it hasn't been easy. For example, a real robot needs sophisticated computer vision to recognise its teacher's legs, so that it can isolate their movement and copy it. But the great variation in the size and shape of legs, which depends on their motion and the angle of viewing, means it is hard to program a robot to recognise legs.

In Second Life, you can get round this problem. Characters don't see objects from a certain angle, nor from a particular distance; all they know is the 3D coordinates of the object, allowing them to recognise legs simply by their geometry. Once the pet can recognise legs, Goertzel then programs it to map the leg movements to the movement of its own legs. Obviously, the pet's own legs are a different size and shape, so the exact same motions wouldn't be appropriate. But the pets experiment with slightly different variations on the theme - and then settle on the set of movements that elicits the most praise from the avatar.

So far, Goertzel says he has successfully taught his dogs to play fetch, basic soccer skills such as kicking the ball, faking a shot and dribbling, and to dance a simple series of moves, just by showing them how (watch a video of the demo at www.novamente.net/puppy.mov).

Imitation isn't the only way the pets learn, however. They can also learn things humans may not have intended to teach them. As well as seeking praise, they are also programmed with other basic desires such as hunger and thirst, as well as some random movements and exploration of the virtual environment. As they explore, their "memory" records everything that happens. It then carries out statistical analyses to find combinations of sequences and actions that seem to predict fulfilment of its goals, such as appeasement of hunger, and uses that knowledge to guide its future behaviour. This can then lead to more sophisticated behaviour, such as a dog learning to touch its bowl when a human walks into the room, because that increases the chance of a goal being fulfilled. "It learns that going near the bowl is symbolic for food," says Goertzel. "This is a sort of rudimentary gestural communication."

Goertzel is aiming even higher. He says learning gestures could eventually form the basis for virtual pets to learn language, just as it does in young children. "Eventually we want to have virtual babies or talking parrots that learn to speak," he says (see "If only they could talk").

Deb Roy, an AI researcher at the Massachusetts Institute of Technology, worries that people will tire of training their virtual pets. "Philosophically I am on board. These are lovely and powerful ideas," he says, "But what are the results that show [Goertzel's team] are making progress compared to people who have tried similar things?"

Novamente has a few tricks up its sleeve to stop people from getting bored. For starters, the synthetic characters will learn quickly as more and more people use them. Although each pet has its own "brain", Novamente's servers will pool knowledge from all the brains. So once one pet has mastered one trick, it will be much easier for another one to master it, too.

Researchers at Novamente are not the only ones who hope to create compelling synthetic characters. Selmer Bringsjord, Andrew Shilliday and colleagues at Rensselaer Polytechnic Institute in Troy, New York, are working on a character called Eddie, that they hope will reason about another human's state of mind - potentially leading to characters that understand deceit and betrayal - and predict what other characters will do next.

The fusing of virtual worlds and AI will almost certainly be good for AI. Since the field failed to deliver on its initial promises of machines you can chat to, robotic assistants that do your housework and conscious machines (New Scientist, 23 April 2005, p 32), it has been hard to get funding to build generally intelligent programs. Instead more specific, "narrow AI" such as computer vision or chess-playing have flourished. Novamente is planning to make its pets so much fun that people will actually pay money to interact with them. If so, the multibillion-dollar games industry could drive AI towards delivering on its original promise.

If only they could talk

Could the fusion of games, virtual worlds and artificial intelligence take us closer to building artificial brains?

Novamente, the creator of virtual pets equipped with artificial intelligence, hopes its pets will learn to make common-sense assumptions like humans, which could eventually allow them to understand and produce natural language, for example.

One of the biggest challenges faced by researchers trying to imbue computers with natural language abilities is getting computers to resolve ambiguities. Take this sentence: "I saw the man with a telescope." There are three possible ways to interpret the sentence. Either I was looking at a man holding a telescope, or I saw a man through my telescope, or more morbidly, I am sawing a man with a telescope. The context would help a human figure out the real meaning, while a computer might be flummoxed.

But in an environment like Second Life, a synthetic character endowed with AI could use its immediate experience and interactions with other avatars and objects to make sense of language the way humans might. "The stuff that really excites me is to start teaching [pets] simple language," says Ben Goertzel of Novamente.

But other AI researchers doubt that virtual environments will be rich enough for synthetic characters to move towards the kind of general intelligence that is required for natural language processing. Stephen Grand, an independent researcher from Baton Rouge, Louisiana, who created the AI game Creatures in the mid-1990s, applauds the Novamente approach, but thinks there are limits to learning inside a virtual world. "Just imagine how intelligent you would be if you were born with nothing more than the sensory information available to a Second Life inhabitant," he says. "It's like trying to paint a picture while looking through a drinking straw."

News Hour: 数码

Fingernail camera makes any object a touchpad

Fingernail camera makes any object a touchpad

Contrasting colours

Pet rock

Colour-blind system

Virtual pets can learn just like babies

Virtual pets can learn just like babies

If only they could talk