Using acoustic echoes to reveal and map your surroundings

[hoodlum]

If you haven't read my thread here or watched the video here I suggest you go and do that to get a better understanding of the tangent I'm about to go on. This is probably a post that no one will care about but I still took the time to type all of this up for some unknown reason.

In modern society technology is something people consider as a useful tool and it is seldom that security and anti-surveillance is the first thing that jumps in to a persons mind when they are considering choosing a new phone. Marketing is all about what a phone can do for you and how many apps it has available but someone with a bit more insight might realise that each and every app available is potentially a disguised attack and everything the phone can do (a high resolution camera, a microphone that samples over 44000 times per second and an accelerometer) can be used against you to invade your privacy at every level. Not only do spies sift through your internet history anymore and your browsing habbits but now they construct a full profile of you. Every intimate detail about your life is more easily accessed by them as technology grows (think smartphones) and in the modern day they even go so far as to take pictures of inside your home & work (once again see my post here) which can even give them hand-written notes that were just sitting around and never came in to contact with technology.

This aforementioned technology allows them to construct a 3D model of your surroundings. It allows them to retrieve sensitive information and otherwise unknown data but also allows anyone to potentially estimate the size of the room based on known object sizes (such as A4 sheets of paper) to give a scale and shadows to predict distances. Obviously when this is being used where exact measurements need to be provided (think targeted raids) it is unreliable.

Now if there was a microphone in the room, even better if it was moved around the room along with some way to listen for the echo's then we would be able to precisely map things... oh wait, there is - your phone. Thanks to Ivan Dokmanića, Reza Parhizkara, Andreas Walthera, Yue M. Lub, and Martin Vetterlia there is now an algorithm and a predefined test method to be able to map the geometry of a room based on echos. Using your phone to emit sounds and capture the responses (or in the future just capture responses) could be the next step in surveillance to map target's surroundings. Due to the hardware nature of a phone there are some limitations and things that don't tie in exactly with the paper below but research can be expanded exponentially.

The first thing that people jump to is thinking that this noise emitted would need to be audible for a smartphone to pick up however Zoosh proved that the microphone was able to pick up sounds that humans are unable to hear (see here) in an effort to bring NFC technology to phones without it being a native built in option through hardware. This means a phone could emit sounds a human could not hear in order to map surroundings and then by combining this data with other sources can create a pretty invasive and complete picture.

Abstract:
"Imagine that you are blindfolded inside an unknown room. You snap your fingers and listen to the room’s response. Can you hear the shape of the room? Some people can do it naturally, but can we design computer algorithms that hear rooms? We show how to compute the shape of a convex polyhedral room from its response to a known sound, recorded by a few microphones. Geometric relationships between the arrival times of echoes enable us to “blindfoldedly” estimate the room geometry. This is achieved by exploiting the properties of Euclidean distance matrices. Furthermore, we show that under mild conditions, first-order echoes provide a unique description of convex polyhedral rooms. Our algorithm starts from the recorded impulse responses and proceeds by learning the correct assignment of echoes to walls. In contrast to earlier methods, the proposed algorithm reconstructs the full 3D geometry of the room from a single sound emission, and with an arbitrary geometry of the microphone array. As long as the microphones can hear the echoes, we can position them as we want. Besides answering a basic question about the inverse problem of room acoustics, our results find applications in areas such as architectural acoustics, indoor localization, virtual reality, and audio forensics."

Acoustic echoes reveal room shape
Published 17/05/2013
Direct Link: http://www.pnas.org/content/110/30/12186.full

Supporting Information - Dokmani
c et al. 10.1073/pnas.1221464110
http://www.pnas.org/content/suppl/2013/06/12/1221464110.DCSupplemental/pnas.201221464SI.pdf

 

[TriVegas]

I'll bite, In trying to keep my reply short and concise here it goes. IF you garnered enough attention to warrant this kind of monitoring from Big Brother there are only two plausible outcomes.

1. You find out AFTER being extracted from your life and either placed in detention or "removed" for lack of stating the obvious.

2. You're already ahead of the curve and go off grid with the finances and strategy already in place to do so. That requires no tech living and short of living in a cave your are in isolation. Living in a faraday cage should you have any modern tech as simple as a radio for news. Anything electronic beyond that or a self contained system with no gps, Internet capabilities, cell phone, etc. would be a no-no. Unless you have the resources of living off the land or the network and finances to move money(lots of it) and keep secrets pre-1900's style, you are going to get found.

 

[hoodlum]

See I completely disagree TriVegas.
Remember this could be used by private enterprise or criminals just as much as it could by the government, it would allow them to map certain restricted areas and gather detailed plans of tunnel systems and areas previously thought to be secure. Let's assume you want to stop the possibility of this rather than just react once your already detained, I don't think you need to live a completely tech-less life. Now the obvious way of this is by changing how the microphone and speaker is activated. Obviously there are many software (and a few firmware) fixes for this but as easily as they roll out, they are circumvented malisciously. There is one way though to really reduce the amount of data being transmitted to a level where it is unusable, especially combined with other software based modifications. Assuming the target has an iPhone, the easiest way would be to hardwire the power of the camera & microphone to the toggle switch on the upper left of the phone previously used to control vibrate/outdoor ringing profiles. This vibrate profile could be used to kill all power to the selected accessories and thus greatly improve security of the targets phone without causing any major loss of functionality. As soon as the target needed to use these devices he could just flick the switch and they would operate as per normal within a very short time. There are many other things that can be done but that's one useful suggestion which doesn't require throwing all technology out of the window.

 

[TriVegas]

See I completely disagree TriVegas.
Remember this could be used by private enterprise or criminals just as much as it could by the government, it would allow them to map certain restricted areas and gather detailed plans of tunnel systems and areas previously thought to be secure. Let's assume you want to stop the possibility of this rather than just react once your already detained, I don't think you need to live a completely tech-less life. Now the obvious way of this is by changing how the microphone and speaker is activated. Obviously there are many software (and a few firmware) fixes for this but as easily as they roll out, they are circumvented malisciously. There is one way though to really reduce the amount of data being transmitted to a level where it is unusable, especially combined with other software based modifications. Assuming the target has an iPhone, the easiest way would be to hardwire the power of the camera & microphone to the toggle switch on the upper left of the phone previously used to control vibrate/outdoor ringing profiles. This vibrate profile could be used to kill all power to the selected accessories and thus greatly improve security of the targets phone without causing any major loss of functionality. As soon as the target needed to use these devices he could just flick the switch and they would operate as per normal within a very short time. There are many other things that can be done but that's one useful suggestion which doesn't require throwing all technology out of the window.

Pretty much any area worth securing has not allowed personal electronics within their confines since the invention of the camera phone and the USB thumb drive.

 

[hoodlum]

TriVegas I am going to have to disagree with you again on that one.
I understand there is a possibility of top secret locations having a ban in place on mobile phones, places like missile silo's perhaps. Even in these cases I have doubt that they would restrict all electronics because if the people inside take the proper precautions then there is no risk and if the people are corrupted then the type of spy gear they could bring in to a place like this would be far smaller and spy-specific and a phone wouldn't be used for espionage type activities, as there would be a better alternative.
But the fact is that for organised crime and for terrorists having the ability to map private locations causes a very serious risk. When I talk about a private location, I'm not talking about a missile silo but rather something along the lines of a major utility building, such as a main electrical substation for a city. If you don't want to imagine a substation then imagine a major water treatment plant or part of the telecommunications network (where phones are certainly allowed). All of these places are private and have restricted access for a reason. If terrorists were able to map these types of places they could effectively cause mass chaos for an entire city with a few very stealthy attacks, which would likely go undetected and give enough time to safely escape the country. If they knew where certain electrical feeds were coming from they could target them and destroy the hardware used to handle and distribute it meaning the affected vicinity loses all electrical power and can take a long time to fix, in some cases days. The danger here isn't really your good overheating but the backup systems in places like hospitals are only designed to last a short time so people on life support and that need to undergo critical operations are at risk of death. Then by targeting the broadcasting network they can instil mass-fear as society is reliant on technology and people wouldn't be able to get the message to each other that they were okay and to stay safe. At present without having diagrams or mapping existing networks the type of attack needed to cause such devastation would be extremely large however if they could pinpoint the major connections in the existing networks they could cripple them very easily. Now even though you might put them under the same blanket, organised crime could use the same information but by blackmailing companies and extorting money from them.

 

[TriVegas]

TriVegas I am going to have to disagree with you again on that one.
I understand there is a possibility of top secret locations having a ban in place on mobile phones, places like missile silo's perhaps. Even in these cases I have doubt that they would restrict all electronics because if the people inside take the proper precautions then there is no risk and if the people are corrupted then the type of spy gear they could bring in to a place like this would be far smaller and spy-specific and a phone wouldn't be used for espionage type activities, as there would be a better alternative.

Personal friend was chemist with low level security clearance rating at Los Alamos National Labs. No personal electronics in controlled environments, no exceptions. Phone, ipod, etc. stayed in the car. Anything in car or workspace open to inspection and confiscation without notice or cause. There was other measures in place he couldn't mention and wasn't aware of details as well, this was ten years ago.

 

[hoodlum]

We're getting a bit off topic of the mapping and seeming to be discussing potential attacks more. I agree with you, there are certain places where you aren't allowed a phone and I never said they're aren't. What I said was that there are plenty of places that do allow phones which house sensitive data (be it building schematics and layouts or even something as simple as a person of interest wall inside a police station). If you want to discuss the plausibility of mapping using echos I'm more than happy too but from here I'm going to refrain from discussing specific attacks because it could only bring more harm than good.

 

[TriVegas]

While the mapping is viable, the reasonability of it being employed by the average citizen or company against another "common" person or enterprise is minimal at best.

It would be easier and less pain staking to acquire blueprints on public record or get someone directly inside. The expense and time vested would not be practical when interception of data is easier from afar via web, cellular or other transmissions. Beyond those means you venture into areas again that are supported by Big Brother antics or billion dollar enterprise. Getting the technology inside such place would be just as difficult as getting an individual who is working for you.

For the average joe to plan a robbery of a local store or the rich guy's housed down the street, this technology is a LONG way off from practicality. While the technology itself is impressive and scary on the same level, it's just not noteworthy enough for the average person to be leaving their cell phones and iPads in a electronically shielded box in the garage just yet.

 

[shortz]

I am not sure how much this should be considered, but this is one reason I like Apple. They do not allow third party apps to be used in their app store or their phones. They also went on record saying that out of the hundreds of times they have been asked to hand over info, that they have only done so a few times, and required a warrant for it. Google and FB have already admitted they are in full cooperation with the government, meaning, nothing is safe on sites or any software that they had their hands on.

 

[BigGameHunter]

Thank you for the info