How Your Smartphone Can Spy On What You Type 77
mikejuk writes "We all do it — place our phones down on the desk next to the keyboard. This might not be such a good idea if you want to keep your work to yourself. A team of researchers from MIT and the Georgia Institute of Technology have provided proof of concept for logging keystrokes using nothing but the sensors inside a smartphone — an iPhone 4 to be precise, as the iPhone 3GS wasn't up to it. A pair of neural networks were trained to recognize which keys were being pressed just based on the vibration — and it was remarkably good at it for such a small device. There have been systems that read the keys by listening but this is the first system that can hide in mobile phone malware."
Re:Wonder the accuracy rate (Score:4, Funny)
Seems like an obvious question, I wonder if you can read.
Re:Wonder the accuracy rate (Score:5, Informative)
Seems like an obscure sensing mechanismI wonder how accurate it is...
Just detecting raw keypresses didn't produce a very accurate result, but switching to picking up pairs of keywords and then using a word dictionary did produce useful data extraction. Accuracies of around 80% were achieved, but the accuracy reduced with the number of keypresses. Word recognition only achieved a 46% accuracy, but this increased to 73% if second choice words were included. Clearly semantic analysis could push the accuracy up.
Re: (Score:1)
Until you read a little closer and find out this couldn't possibly work in real life. The whole thing is a joke.
Re: (Score:2)
dunno, doesn't sound much more obscure than using the microphone to detect where the device was touched on(some years back some articles on this).
Re:Wonder the accuracy rate (Score:5, Informative)
Its not accurate at all unless you have the luxury of training the neural networks with the phone sitting in EXACTLY the same place in EXACTLY the same orientation every time, in a totally vibration damped laboratory.
You have to locate your phone two inches from the keyboard every time.
Not on a piece of paper, a book or a mouse pad, but directly on the desk.
Oh, and you have to install software on your iphone,
AND feed the data into a a couple of Neural networks external to the phone.
And nothing else can be vibrating on that desk. No radio. No mouse movements, and your computer has to be off the desk.
No air conditioning air flow, not tapping fingers, typical floor bounce from walking people.
And no typing fast.
When you start reading all of the things that will screw up this test that the authors wrote in their own study you have
to wonder how it is they even managed to keep from laughing their own study out the door.
They just proved it can't be done in the real world, yet they went ahead and put out the study anyway as if
they had discovered a real and present attack vector.
So then the recommend you keep your phone outside the room. Who does that? Why do that, when
their own study demonstrates it is totally impossible to do this?
Re: (Score:1)
Re: (Score:3)
That said, it's a neat proof-of-concept that may, eventually, find some (voluntary) applications. Need to diagnose a vibration in a car, washing machine, etc.? Something in the house making an odd noise and you can't figure it out? Water/gas line leak while you're asleep. There's potential, and, as they inadvertantly point out, (4>3GS) the sensor technology is improving.
Re: (Score:3, Insightful)
Isn't this just a proof of concept though - like most technologies start?
Their study can be used as a reference, and over time, the underlying technology and techniques can be perfected so that it can work as an additional attack vector. Do you think Acoustic Keyloggers worked right off the bat from conception to implementation? And your premise relies on the postulation that sensors in mobile phones won't improve over time as well - or that multiple technologies will just cease to improve, for that matte
Re: (Score:2)
Re: (Score:2)
It's because tin-foil hat wearing reactions to potential privacy issues are trending high on Slashdot these days.
Re: (Score:2)
Please don't confuse the idiot masses with facts, all they care about now is that your phone can track your keyboard presses and will cry outrage about privacy rights and link it to NSA conspiracies while they never for once think about turning off their phone, or just getting rid of the phone, or changing anything that might affect their behavior or basic common sense.
Re: (Score:1)
This is the hypothetical if I had any talent and a lot of free time and money. (I assume the NSA does).
You have to locate your phone two inches from the keyboard every time.
That's pretty easy to normalize using your favorite audio application...so that's an easy one to solve. If I’m just researching to see if this is possible, I would probably skip that problem.
Not on a piece of paper, a book or a mouse pad, but directly on the desk.
Using a neural network, it might be able to learn how 'soft sounds' work...not sure...harder but not insurmountable. If you break the 0 threshold of the accelerometer (the point where you’re in the realm of
Something less obscure, perhaps? (Score:2)
Damn you, Ben Franklin! (Score:1)
This whole "electricity" thing has gotten way out of hand. Look how it's being used these days!
Wow! What a vulnerability!! (Score:5, Funny)
Re: (Score:1)
Or you know, maybe the developer of the malware could train it before distribution?
Sure, I'll just send them my phone, keyboard, and desk so they can build the training set. Then when they send it back, they can use it to monitor my keystrokes. Wake me up, when this is a serious attack instead of a parlor trick.
Re: (Score:3, Insightful)
First you need to download and install a neural network program in your smartphone, train it with loads and loads of data. Then turn it on and leave it running. Then it can become a keystroke logger. At this point it worse than the proverbial unix virus, "You got a unix virus. It works on honor system. Please forward this mail to all addresses in your .mailrc and sudo \rm -rf / Thank you."
You know, the same smartass attitude was held by our government officials regarding the "hollywood" possibility of hackers gaining control over power grids, missile launch systems, water distribution systems, etc. And then Stuxnet showed up, and took out a key element of a country's nuclear weapons program. It is exceptionally arrogant to say because you can't see a problem, one doesn't exist.
This is a proof of concept; It demonstrates that such an attack is now possible. Everything Stuxnet achieved, it did
Re: (Score:2)
First you need to download and install a neural network program in your smartphone, train it with loads and loads of data.
You know, the same smartass attitude was held by our government officials regarding the "hollywood" possibility of hackers gaining control over power grids, missile launch systems, water distribution systems, etc. And then Stuxnet showed up,
Not the same, Stuxnet and even .bat files are run by default on a MicroSoft OS. To this day I have to disable auto-run, the largest
most over looked backdoor into a system.
Auto-run being on by default is most likely because people would be inconvenienced or not having a clue what to do next.
This is a proof of concept; It demonstrates that such an attack is now possible.
Proof of concept of something I've known since the early 90's that a computer system gives off electromagnetic energy
and you can read that energy through a wall (apartment). They just made it smaller and moved it closer.
Re: (Score:2)
Secondly, none of that has exactly dick to do with what's being discussed -- which is the use of motion sensors to capture vibration, which is then via a complex software application, recreates the keystrokes entered from a nearby keyboard.
Yep - I read the summery, replied to your reply, then read the article and thought damn ----------- missed it by that much.
Re: (Score:2)
It's easily done if you give someone the right motivation.
Remember the jailbreak worm that relied on people leaving the
Re: (Score:1)
Ahem. The proverbial unix virus [wikipedia.org] existed, and didn't depend on any kind of honor system. Honestly, you'd think Slashdotters should know a tiny bit of history.
Kinesis Advantage with buzzer (Score:3)
I wonder what this little app would do with the keyboard I am typing on. First of all, it is a mechanical keyboard with cherry MX browns. Second, I have the "buzzer" function on that simulates "clicky"-ness, since it is not a real clicking switch (though tactile) like e.g. the blue one or the real buckled ones.
I am not saying it would work better or worse, just curious if it would work on a Kinesis and how much the "clicker" and the totally odd shape of the keyboards would disrupt the functionality.
Anyway.... my phone is usually on airplane mode when I enter the house and is redirected to a landline that has an Asterisk box on it... then the Asterisk box' FXO is carefully disconnected, so no calls in, no calls out:). That's the way I like to handle phones and phone calls.
Re: (Score:3)
Re: (Score:2)
Unless you remove the battery, is your phone really ever off?
Re: (Score:2)
Re: (Score:2)
my phone is usually on airplane mode when I enter the house and is redirected to a landline that has an Asterisk box on it... then the Asterisk box' FXO is carefully disconnected, so no calls in, no calls out:). That's the way I like to handle phones and phone calls.
I prefer this approach: my family/friends get my personal number but know to text rather than call (auditory processing disorder makes it a bitch to understand them), and everyone else gets my Google Voice number so I can get the transcribed messages via email. This way, I don't need to deal with being pestered via phone, but can have full access to all of its useful apps & functions.
Over-generalisation? (Score:5, Insightful)
We all do it — place our phones down on the desk next to the keyboard.
I love a good over-generalisation.
Re: (Score:2)
My phone is generally in my pocket. Maybe people who are constantly on the phone do this? I don't like those people anyway - let them get their data stolen; I don't care.
Re: (Score:2)
I think most people do it simply because smartphones have so many uses. I only talk to mine if I have no other choice, but it sits in a businesscard holder when I'm at my desk just so I can glance at it to check the time, my to-do list, and so forth.
Re: (Score:1)
we all do
Even worse... (Score:4, Insightful)
it can spy on what you say!!!
Seriously, if my phone is compromised, everything else is pretty much moot.
Re: (Score:1)
If two criminals want to communicate securely with each other by cell phones, they can do so if they keep their conversations short and by using prepaid phones such as trac phones bought with cash. Their CIA, NSA, KGB or whoever can listen in on their conversations, but they don't know who is talking.
Reminds me... (Score:2)
Re:Reminds me... (Score:4, Interesting)
. . . of the little scheme someone I knew cooked up to read data transmissions from watching the lights flash on a Hayes modem - from a distance, of course :) Not that I would ever do anything spurious like that, tho.
The vibration trick seems a bit of a stretch to be useful, but it does fall into a class of things like you said. There are so many holes in the technology created by accident or on purpose that it is a wonder that anything is secure. I was at a COMDEX once a long time ago and was chatting with an engineer ( a friend ) that worked for a modem company about my companies dial up customer service sytem and complaining that it hung up on customers some times. He asked me for my dial up number and I provided it. I assumed he was going to see if it hung up on him. He proceeded to enter a long string of characters and took control of our modem, went into configuration and changed a parameter that set a hang up delay on inactivity. It fixed the problem, but that was creepy. Obviously that was long ago before the internet, but I have never trusted any system since then unless it was open source and open hardware, and even then I am not sure because I have seen spooks at the chip fab and I am sure they weren't there to get coffee.
I watched some videos from DEFCON and became even more certain that we live in a silicon dioxide house and it is subject to fracture on impact, so it would be advisable to avoid conflict with projectiles.
Re: (Score:1, Insightful)
Obviously that was long ago before the internet, but I have never trusted any system since then unless it was open source and open hardware, and even then I am not sure because I have seen spooks at the chip fab and I am sure they weren't there to get coffee.
Having the source, or the blueprints, does you little good if you do not know how to read and use them, and if you stopped to go through these things for every item you own, you would turn grey and cold long before completing this epic assignment. Technology is advancing at a breakneck pace and it simply isn't possible for any one person, or even a small group of people, to retain adequate working knowledge of all the technologies we come in contact with on a daily basis enough to provide viable protection
Re: (Score:2)
Countermeasures (Score:1)
s/Your/A/ (Score:1)
Re: (Score:2)
Um, no they can't. Not without having done so previously, and trained for your keyboard.
calling this bs (Score:1)
This is ridiculous. For the phone to run through the learning phase, the user has to type in the exact words with the phone in about the same position to calibrate the neural network. Even if you use frequency analysis to determine it, there has to be along enough time to get enough samples. So let's see it in action.
If you have mobile malware (Score:1)
On your phone, you have bigger problems than someone listening to the sound of your keystrokes on a keyboard. Everything I have read, is that iPhones are particularly resistant to getting malware on them.
Re: (Score:2)
With a known conference room or free wifi cafe this method might get interesting for pure data entry by a person.
MI5 episode (Score:5, Interesting)
There was an episode of MI5 [imdb.com] (aired as "Spooks" in the UK) that had this many years ago.
They gave a foreign agent a document to type, and had an eavesdropping device in his office. By recording the keyclicks of the known document, they were able to train the system to decode keyclicks for subsequent documents.
It didn't seem farfetched at the time, it doesn't seem farfetched today.
Re: (Score:2, Interesting)
I think Viktor Suvorov mentions in one of his books how spies used to write with a pencil, rather than use a typewriter, because of the danger of decoding a text from listening to the typing. That was in the time when people still used typewriters. So, yes, this stuff does pre-date smartphones.
Re: (Score:2)
I've been told most of those spying techniques relied on listening to the radio noise that CRT monitors give off. I guess if you owned a monitor of the same brand and model as the target you could train your snooping device on your monitor and then use that setting.
I have no idea if it's true or if it's hyperbole, but it's often said that the most sensitive snooping devices could pick up the signal from across the street.
Wired keyboards give off a much weaker radio signal that you can try to snoop on in cas
Re: (Score:2)
You still see the comments about needing to get into the device, OS protections and how keystrokes are magically encrypted.
As for this, it reminds me of early TEMPEST efforts and selling the world on end to end security with a tiny gidt in every crypto machine
Who knows where this tech will go, but great to see people experimenting and publishing real world results
Old news... (Score:2)
You should see this wireless keyboard! (Score:2)
No power, no electronics, just a bunch of keys with springs. The microphone in the computer reads the keypresses.
Re: (Score:2)
No power, no electronics, just a bunch of keys with springs. The microphone in the computer reads the keypresses.
I've already replied, so I can't mod. This sounds like an awesome idea actually. No more replacing batteries in your wireless keyboard. Build it, I will buy one.
Re: (Score:2)
This is a fantastic id4589074VTJIL4D5QX3T9JFDCGJea.
Sorry, my C3409TOIKJERC2RIOKFSOI GJRIOT cat just jumped on and off my desk.
Re: (Score:2)
This is a fantastic id4589074VTJIL4D5QX3T9JFDCGJea.
Sorry, my C3409TOIKJERC2RIOKFSOI GJRIOT cat just jumped on and off my desk.
Sounds like you need better software then.
Related! Iphone 4 can control your computer (Score:1)
Nothing new (Score:5, Funny)
Misunderstanding of the Vulnerability (Score:2)
Many of the early posts seem to misunderstand the vulnerability issue here.
This is not about your phone getting infected with malware that allows it to detect your PC keyboard typing.
This is about me putting the vibration-detection app on my own phone, and then going to someone else's desk and recording them logging in.
So, imagine me going to my local AT&T store, bank, or my boss's computer, and casually setting my phone down while they log in to check my account or whatever.
Granted, some of those syste
Re: (Score:2)
Many of the early posts seem to misunderstand the vulnerability issue here.
This is not about your phone getting infected with malware that allows it to detect your PC keyboard typing.
This is about me putting the vibration-detection app on my own phone, and then going to someone else's desk and recording them logging in.
So, imagine me going to my local AT&T store, bank, or my boss's computer, and casually setting my phone down while they log in to check my account or whatever.
Granted, some of those systems will require more than just a password (I might need their username, or the URL to log in, or perhaps their firewall only accepts certain IPs), but it's still a considerable weakness if this application is reliable and gets out in the open.
I can imagine keyboards that are "vibration silent" or special "vibration absorption" pads that will prevent this from happening. Either that, or customer service reps will start saying "Please remove your phone from my desk while I access your account."
1. Place underneath ATM.
2. Use any existing method of obtaining user's card #
Would be slightly less obvious than putting something over the buttons themselves. For RFID equipped cards, the entire setup could be out of sight.
Now if only the iPhone's battery could be hacked to last long enough to make this plausible...