A few days ago I blogged about the Carnegie Mellon study on cellphones in aircraft, wondering if it wasn’t total bunk. Today I found out I was right: it doesn’t prove anything. The IEEE article on the study admits it right in this paragraph (emphasis is mine):

There is no smoking gun to this story: there is no definitive instance of an air accident known to have been caused by a passenger’s use of an electronic device. Nonetheless, although it is impossible to say that such use has contributed to air accidents in the past, the data also make it impossible to rule it out completely. More important, the data support a conclusion that continued use of portable RF-emitting devices such as cellphones will, in all likelihood, someday cause an accident by interfering with critical cockpit instruments such as GPS receivers.

So the researchers didn’t actually prove anything, they just said it might be true. Let me apply their conclusion to a different scenario, equally truthful as their own:

Nonetheless, although it is impossible to say that cows may someday build rockets and colonize the moon, the data also make it impossible to rule it out completely.

Yes, cattle seem to lack skills in building rockets. Their social skills don’t extend to much more than an occasional “moo.” Yet its impossible to rule out completely that one day they could build rockets and colonize the moon.

Other snippets of their article are also dubious. For instance, researchers are shocked, shocked to learn that some people try to use their phones in flight!

Our research shows clearly that, in violation of FCC and FAA rules, calls are regularly made from commercial aircraft.

News Flash: people sometimes break rules. Especially when the rules are arbitrary. These are rules that the FCC implemented not for safety reasons, but to keep an airborne phone from lighting up several cell towers. And the one the FAA adopted it not for security but for the sake of convenience or comfort to the passengers (or if you’re of a more conspiratorial nature, to drive use of the expensive AirPhones)?

The researchers later contradict themselves, at least partially (all emphasis mine):

Furthermore, PCS is regulated separately from cellular; the FCC does not restrict airborne use of PCS wireless handsets.

That’s right, the FCC does not restrict the use of your PCS phone in flight. If you’ve got Sprint or Verizon service, you can gab with Aunt Martha until the signal fades. Your flight attendants might get annoyed, the FAA may beat you down, but the FCC won’t care.

The researchers do touch on the scariest part of interference – that occuring to the GPS system. The article states:

Our measurements also found emissions from other onboard sources—devices used by passengers—in the frequency used by GPS.

Pretty vague, huh? Emissions were found, but from where? How do they know it came from devices used by passengers? It certainly doesn’t say that this interference came from phones. The article cites a NASA technical memorandum (PDF) about a particular phone, the Samsung SPH-N300, but did not test this phone itself. It basically took the memo at face value with no testing of its own at all.

In the case of the Samsung phone, the reports to NASA were from the general aviation community. These are smaller aircraft than airliners. The Samsung phone has a GPS receiver built in, meaning it could interfere with other nearby GPS receivers – if placed right next to or on top of them! Certainly this is a different scenario from one where the phone is somewhere in the cabin, far away from the cockpit. While the NASA test seems sound, a RF-proof lab is a far different environment than an actual aircraft. The fact that a phone radiates does not in itself prove it interferes with avionics.

The most damning evidence of a fraud is this:

Ours was a conservative estimate, since a call made at the other end of the cabin from the instrumentation would be below the threshold we could observe.

Uh, come again? You had a sensitive broadband antenna and frequency analyzer in the overhead bin – a bin separated from the cabin by a flimsy plastic door – and you could not detect a call made from the other end of the cabin? You mean to tell me these phones are so powerful as to overwhelm shielded electronics located in the cockpit behind a steel-reinforced cabin door, yet you couldn’t detect them fifty feet away using an oversized antenna? Are the passengers flying pigs, by chance?

The report then goes on to cite the ASRS database, a database aircrews “and others” use to report strange behavior in the aircraft they fly. This database is flawed for many reasons. Number one, the entries are anonymous, meaning no followup can occur. Number two, if so-called emissions experts with a fancy spectrum analyzer can’t detect a cellphone in back of the cabin, how likely are the aircrews to positively identify the source of the interference? Aircrews aren’t trained in the science of radio. They are trained to either fly the plane or to hand out peanuts. If something odd occurs on the flight, guess what’s going to get blamed – those spooky electronics. Captain Bob ain’t leaving his seat to hunt down a naughty PDA. This data is anything but scientific.

One scary scenario the article cites comes from the dubious ASRS database. A 30-degree navigational error was supposedly corrected when a passenger turned off his DVD player. DVD players aren’t intentional radiators: they do not by nature transmit. There might be some interference caused by them by their intermodulating frequenies (IF) but its highly unlikely that those weak signals would be strong enough to overwhelm a VOR receiver in the cockpit. And that’s assuming the IF was anywhere near the VOR frequencies, which is unknown.

The model of the so-called offending DVD player (called “the new DVD players” in this 1999 report) is not stated. The plane was an ancient B727, apt to have other troubles due to age, and its wiring and antennas were not subsequently inspected. The radio signals coming from the ground (including the VOR transmitter the plane was supposedly near) are far more powerful than any a DVD player could produce. One incident does not constitute scientific proof. Even so, for inexplicable reasons the researchers cite this case (submitted anonymously, remember) without bothering to recreate it themselves for testing purposes.

Sorry, guys. You’ve gone round and round and you haven’t proven a thing. You can’t show that any interference occured; you can’t identify the source of any signals you did measure; you base your conclusions largely on anonymous, non-scientific, self-reported data; and you can’t even detect a cellphone in the cabin fifty feet away.

I’m gonna make a call here myself, and I call bs.

[Update] Another blogger takes a hard look at the CMU data and also has issues with the report.