The NSA's plans to track everything you do

Bamsford again.

On a remote edge of Utah’s dry and arid high desert, where temperatures often zoom past 100 degrees, hard-hatted construction workers with top-secret clearances are preparing to build what may become America’s equivalent of Jorge Luis Borges’s “Library of Babel,” a place where the collection of information is both infinite and at the same time monstrous, where the entire world’s knowledge is stored, but not a single word is understood. At a million square feet, the mammoth $2 billion structure will be one-third larger than the US Capitol and will use the same amount of energy as every house in Salt Lake City combined.

Unlike Borges’s “labyrinth of letters,” this library expects few visitors. It’s being built by the ultra-secret National Security Agency—which is primarily responsible for “signals intelligence,” the collection and analysis of various forms of communication—to house trillions of phone calls, e-mail messages, and data trails: Web searches, parking receipts, bookstore visits, and other digital “pocket litter.” Lacking adequate space and power at its city-sized Fort Meade, Maryland, headquarters, the NSA is also completing work on another data archive, this one in San Antonio, Texas, which will be nearly the size of the Alamodome.

Just how much information will be stored in these windowless cybertemples? A clue comes from a recent report prepared by the MITRE Corporation, a Pentagon think tank. “As the sensors associated with the various surveillance missions improve,” says the report, referring to a variety of technical collection methods, “the data volumes are increasing with a projection that sensor data volume could potentially increase to the level of Yottabytes (1024 Bytes) by 2015.”[1] Roughly equal to about a septillion (1,000,000,000,000,000,000,000,000) pages of text, numbers beyond Yottabytes haven’t yet been named. Once vacuumed up and stored in these near-infinite “libraries,” the data are then analyzed by powerful infoweapons, supercomputers running complex algorithmic programs, to determine who among us may be—or may one day become—a terrorist. In the NSA’s world of automated surveillance on steroids, every bit has a history and every keystroke tells a story.

With a special bonus:

Unfortunately for General Douglas MacArthur, the codebreakers were able to read the communications of Spain’s ambassador to Tokyo and other diplomats, who noted that in their discussions with the general, he made clear his secret hope for all-out war with China and Russia, including the use of nuclear weapons if necessary. In a rare instance of secret NSA intercepts playing a major part in US politics, once the messages were shown to President Truman, MacArthur’s career abruptly ended.

Well, well.

On a related note:

General Curtis LeMay remembered correctly that the JCS had earlier concluded that atomic weapons would probably not be useful in Korea, except as part of “an overall atomic campaign against Red China.” But, if these orders were now being changed because of the entry of Chinese forces into the war, LeMay wanted the job; he told Stratemeyer that only his headquarters had the experience, technical training, and “intimate knowledge” of delivery methods. The man who had directed the firebombing of Tokyo in 1945 was again ready to proceed to the Far East to direct the attacks. (11) Washington was not worried that the Russians would respond with atomic weapons because the US possessed at least 450 bombs and the Soviets only 25.

On 9 December MacArthur said that he wanted commander’s discretion to use atomic weapons in the Korean theatre. On 24 December he submitted “a list of retardation targets” for which he required 26 atomic bombs. He also wanted four to drop on the “invasion forces” and four more for “critical concentrations of enemy air power.”

In interviews published posthumously, MacArthur said he had a plan that would have won the war in 10 days: “I would have dropped 30 or so atomic bombs . . . strung across the neck of Manchuria.” Then he would have introduced half a million Chinese Nationalist troops at the Yalu and then “spread behind us – from the Sea of Japan to the Yellow Sea – a belt of radioactive cobalt . . . it has an active life of between 60 and 120 years. For at least 60 years there could have been no land invasion of Korea from the North.” He was certain that the Russians would have done nothing about this extreme strategy: “My plan was a cinch.” (12)

Cobalt 60 has 320 times the radioactivity of radium. One 400-ton cobalt H-bomb, historian Carroll Quigley has written, could wipe out all animal life on earth. MacArthur sounds like a warmongering lunatic, but he was not alone. Before the Sino-Korean offensive, a committee of the JCS had said that atomic bombs might be the decisive factor in cutting off a Chinese advance into Korea; initially they could be useful in “a cordon sanitaire [that] might be established by the UN in a strip in Manchuria immediately north of the Korean border.” A few months later Congressman Albert Gore, Sr. (Father of former VP and 2000 Democratic candidate Al Gore, Jr., and subsequently a strong opponent of the Vietnam war) complained that “Korea has become a meat grinder of American manhood” and suggested “something cataclysmic” to end the war: a radiation belt dividing the Korean peninsula permanently into two.

Jesus fucking christ.

Holy crap.

It is scary how close we can come to nuclear war.

During the Cuban missile crisis, it came down to 3 Russian submarine officers arguing about whether to carry out standing orders to launch one at the U.S. if they were attacked (they were accidentally attacked.) Had they carried out their orders, the U.S. would have launched a full nuclear attack on Russia.

Another president may have given MacAuthur his head. Who knows, around the world, how close we’ve come?

I would have dropped 30 or so atomic bombs . . . strung across the neck of Manchuria.

Holy fuck. It scares the shit out of me that men like MacArthur and LeMay had the kind of power they did. Heh, Truman deserved the Nobel peace prize for firing MacArthur and not being out of his fucking mind.

Is this going to make me safer? And what about mah chilluns? Will they be safer too!? Oh lawdy lawdy…

Dude was a General. Obviously he knows what he’s doing and we should have taken his advice without thinking twice on it. Have you people learned nothing from the wingnuts this last decade?

Generals come only after Jesus in the keyboard warrior foodchain. They are even modifying Jesus to make sure that Jesus was always right. What considerate chaps they are.

If you have a modded Jesus, don’t you risk it being bricked when the next official patch is released?

A yottabyte of storage. Seriously? You know how many watts that would draw? How much that would mass, let alone cost?

A terabyte is 10^12 bytes. So a yottabyte, 10^24, is a trillion terabyte drives. Imagine that by some miracle, by 2015 a terabyte only costs a dollar… If that terabyte by another miracle weighs only a gram, you’re still talking a million metric tons of storage medium to hold that yottabyte… How much silicon do you think Taiwan actually cranks out in any given year?

Yeah, I understand the author first mentions some supposed new NSA site, and then separately cites some public MITRE report that has nothing to do with the NSA site, but the fact he mentions this ridiculous number at all in connection with it is very strange.

Other stuff in this article is also hyped. A “million square feet” sounds like a huge expanse until you think about it for a second or two… I’m sure there are a great many industrial sites that are 333 yards on a side. And it only costs $2 billion? Pssh.

Edited to retract a statement impugning the writer’s intentions. I guess he’s got a pretty sound reputation. But this MITRE report and his citation of it seems like it’s only there to daze the reader with numbers.

As for MacArthur/LeMay and the nuclear war with China, yeah, that was always MacArthur’s plan for bringing the war into Manchuria, and also why Truman stopped it and fired him. Not really news. As for the Korean War sans Chinese intervention, there never was any target large enough to nuke; every North Korean city had already been destroyed into rubble in the first year of the war.

Bruce Cumings, the author of that piece, has written some important works on Korean history, but also some very very wrong ones, especially on North Korea - he wrote, among other things, how North Korea didn’t start the Korean War with Soviet help a year before Soviet archives proved they did, and how North Korea was an agricutural wonderland a year before their manmade famine. Very, very, very hardcore leftist, who sees the US as a worse tyranny than the DPRK.

Wasn’t China at this stage mostly pitchforks and the odd gun?

Hahahahaha MacArthur owned. Jesus Christ.

Um. No.

MacArthur may have been a loon, but it seems he and I play Civ2 pretty similarly.

Yottabytes sound crazy, but Bamsford’s pretty reliable.

I knew about MacArthur, but I didn’t know about the idea for a goddamn radioactive neutral zone. Was Magneto on the joint chiefs?

Cuming’s background is an amusing surprise.

I knew this chick in high school named Yotta, but she spelled it differently.

Best P&R post EVAR!

You know at 1st I was going to jump on the bandwagon and call bullshit on the yottabyte thing. However after doing the math I don’t think it is that far off.

The amount of storage per dollar has been doubling every 11 months for the last oh 10 years, IIRC. So that’s 6.5 doublings between now and the end of 2015, which means we’ll get ~x90 as much storage per dollar. That means that we will be close to 1 TB/dollar (like Miramon was saying). Now the NAS/SAN/whatever system will cost more than that, but let’s work with the 1 TB/dollar figure. That means that this system would cost one trillion dollars. Now that is pretty spendy even for the NSA, but not totally unreasonable, it is really only off by an order of magnitude. Given the growth rates of storage (yes, these rates could break down at any time, it is not like they are a phsyical law) then within 3 more years or so (2018) the cost would be down to 100B which is crazy expensive, but possible.

No idea how you would process such a volume of information, but I think it will be possible to store it, in time.

Also, I am dubious that e-mail/phone calls/recipts would be sufficient to fill up a yottabyte.

Tons of hi-def video surveillance OTOH, now that I can see filling it up.

The distinct impression I get from my reading is that the NSA keeps getting sold on these stupid, error-prone data mining algorithms by the sleazy intelligence industry, and thinking that IF ONLY THEY HAD MORE DATA they’d totally be able to find the terrorists.

I guess I was wrong about tons of hi-def video filling that up. Decent high def is 20Mb/sec (that is better than what you get on your blue ray player). That only works out to 78TB/year per camera (and really, you would think that given the problem: a camera focused on the same location for a really long time, we could come up with better compression than H264, but whatever).

So you would need 13 BILLION hi-def cameras to fill that up in one year. Maybe you are thinking about the long term and you want to plan for 10 years that’s still 1 billion cameras. That is a lot. I don’t have a good grasp on how many street corners there are in America, but I doubt that there are that many (I mean, heck there are only 300 million people here). I bet you, even if you filled each and every office/place of business/home with cameras you would be below one billion. And, clearly, you could place a camera on each person and record everything that they do and only use 300M of them.

Hey somebody has to foot the bill for this kind of basic research, it can’t all be done by the financial industry! I’m fully in favor of the government spending oodles on data mining and advancing the art (although, maybe spending it on Yotta bytes of storage is not the best way to improve the art).