Thursday, February 09, 2006

Enigma

My favorite museum in the Washington DC area is . . . well, okay, that would be the National Air and Space Museum. And my second favorite would have to be the National Gallery of Art. But aside from those two, my favorite Washington museum is actually located some distance away from the city, out the Balto-Wash Parkway. I am talking, of course, about the National Cryptologic Museum.

I visited there a few years ago when I gave a talk at the University of Maryland. The museum building was once the Colony 7 Motel, located at the freeway interchange just beside the National Security Agency's headquarters at Fort Meade. As I heard the story, the NSA found that several motel rooms were more or less permanently occupied by folks from the Soviet Embassy. They therefore bought the motel and closed it down. But then what could they do with the building? So they made a museum out of it. (This story may not be exactly true, of course, but it is too good not to tell.)

The museum is not particularly large, but despite its size it has some of the most remarkable exhibits I have ever seen. There are collections of Renaissance texts on cryptography, Civil War signal flags and code wheels, early model telephone scramblers (the size of whole desks) and so on. The most wonderful display when I was there was the Enigma machine.

Everyone has heard of Enigma, I suppose. It was the electromechanical cipher system used by Nazi Germany, hideously complex, supposedly undecipherable. Only it wasn't, quite. Thanks to early work by the Poles, the mathematical genius of the British (led by Alan Turing himself) and American technological wizardry, the Allies broke Enigma. In so doing, they built the foundations of modern information theory and computer science.

They also, just possibly, won the War. The Enigma breakthrough, code-named "Ultra", was extremely secret and was not generally revealed until almost thirty years after the War ended. So the conventional accounts of the War, like Churchill's own multivolume history, necessarily left out one enormous element. Consider the Battle of the Atlantic, which depended on finding U-boats before they could destroy supply convoys heading for Britain. A complete account of that could not be given without the crucial fact that in February, 1942, the German Navy upgraded to new, more sophisticated Enigma machines, thereby "blacking out" the Allies from their communications for most of that year. This happened at a period when the Germans were sinking nearly a million tons of shipping a month.

There is some dispute about the impact of Enigma/Ultra on the outcome of the War. Richard Overy's splendid study Why the Allies Won, for example, mentions it only in passing. But there can be no denying that the ability to read the enemy's radio traffic -- to look over the shoulder of the German generals and admirals -- was a dramatic advantage. And cracking the Enigma system was not the only achievement of Allied cryptanalysis during the War, and not the only signals intelligence coup that arguably changed the course of history. There was also the elegant unlocking of the German Lorenz teleprinter system and the remarkable solution of the Japanese Purple code. There was the U.S. Navy's "Magic" group, which broke the Japanese naval ciphers. There are exhibits about all these at the National Cryptologic Museum, but of course Enigma is the name to conjure with, the one that stands for all.



My own field of science is quantum information, which makes me a peculiar hybrid of information theorist and quantum physicist. One set of my intellectual forebears spent the War in shabby temporary buildings in Bletchley Park and Arlington Hall, penetrating the secrets of the enemy. The other set toiled in equally ramshackle laboratories in Chicago and Los Alamos and Oak Ridge and Hanford, penetrating the secrets of the nucleus. So for me, to touch a real Enigma machine at the National Cryptologic Museum -- to set its rotors and actually use it to encrypt my own name -- was like messing with the controls of Fermi's reactor under the squash court at Stagg Field.



And then there was Venona.

Venona started in 1943 as an attempt to read encrypted cables between Moscow and various Soviet diplomatic missions, which had been intercepted and copied since 1939. It took a long time for the Americans to begin to break into the traffic. Even when they did, they were only able to read a small fraction of the messages, years after they were sent. But the messages were astounding. Many of the cables were in fact KGB and GRU communications dealing with Soviet agents in the United States and elsewhere. The agents were discussed by code name, so that many of them remain unidentified today. But it was often possible to use circumstantial detail to identify the referents of the code names.

At about the same time, several Soviet agents defected and told their stories to the FBI. Whittaker Chambers and Elizabeth Bentley were American former Communists who had done secret work for Soviet intelligence. Igor Gouzenko was a cipher clerk for the GRU at the Soviet embassy in Canada. The information they provided supported and filled out the data gleaned from the decrypted cable traffic. Piece by piece, a fragmentary picture was built up of Soviet clandestine activities in the West. That picture was, by necessity, only known to a few.

The Venona project was only made public about ten years ago, after the end of the Cold War. Nowadays, you can read the decrypted and translated Venona messages online. But for decades it was among the darkest secrets in the secret world. And so the conventional narratives about the Cold War that people told each other for years were inevitably incomplete.



Senator Joseph McCarthy was a bad fellow -- nothing but a demagogue by the end (and probably in the beginning as well), ignorant of and utterly indifferent to the truth -- a man who carelessly inflicted great harm on the body politic. He is justly infamous. But I think that the greatest harm he did may have been to make the idea of Communist spies ridiculous in the eyes of enlightened Americans. This was bad because the Communist spies really did exist. We know this from Venona.

Consider the notorious Rosenberg case. Julius and Ethel Rosenberg were arrested, accused of stealing atomic bomb secrets, and tried for espionage. In 1953 they were executed. For decades afterward, it was an article of faith among the enlightened Left that the Rosenbergs were innocent. Only we now know that this was not the case. Julius Rosenberg was a Soviet agent code-named LIBERAL, mentioned many times in the Venona messages, who was part of a ring that passed atomic secrets on to Moscow. (While Ethel is not herself mentioned in the decrypted messages as a spy, it seems overwhelmingly likely that she knew of her husband's activities.) U.S. counter-espionage officials knew this for a fact well before the Rosenbergs were arrested.

But this secret certainty could not affect the conventional narrative -- i.e., that the Rosenbergs were railroaded simply because they were Jews who espoused progressive politics. And this narrative, though exploded by the Venona revelations, still exerts influence today, still colors the conventional thinking among cultural elites about the meaning of the Cold War and the history of the 1950's. (Don't believe me? Check this out.)

Or take the case of Alger Hiss. (Was ever a man more unfortunately named?) Hiss was a noted diplomat, a member of the U.S. delegation to Yalta, a key figure in the founding of the U.N. and president of the Carnegie Endowment for International Peace. He was pure Establishment, of the best and most liberal sort, the sort that had made the New Deal and guided the War. He attended the best schools (Johns Hopkins, Harvard Law) and moved in the best circles. Then, in 1948, he was accused by Whittaker Chambers of having been a Communist spy. Sensational hearings and trials followed, in which Hiss unsuccessfully sued Chambers for libel and then was convicted himself of perjury.

Hiss, too, was widely regarded to be a guiltless victim of anti-Communist hysteria. California congressman Richard Nixon, the prime mover against Hiss in the Congressional hearings, was villified by the liberal intelligensia. (This, perhaps, was one source of Nixon's well-known distrust of intellectuals.) Chambers's own memoir of the case, Witness, was dismissed as a pack of lies. Hiss always stoutly proclaimed his own innocence; and in return, in some circles, he became something of an icon of the horrible "McCarthy era".

He was also a genuine Communist spy, who appears under the codename ALES in the Venona messages. Chambers and Nixon turn out to have been right after all.



We therefore know that accepted accounts of historical events are not necessarily correct, because some things remain concealed even years after the fact. The accounts can be incomplete, like the histories of the Second World War before the Enigma/Ultra story came out. Or they can be just wrong, like the widespread belief in the innocence of Hiss and the Rosenbergs. When those historical narratives get mixed up in political or ideological debate, they can persist even when later evidence should make them untenable. The Left in this country -- growing a bit gray these days but still influential in cultural and academic circles -- has long defined itself by its opposition to "McCarthyism". To say now that the anti-Communists may have known a thing or two, would be to attack the very legitimacy of the Left. Such a reassessment will be strongly resisted or (more likely) simply ignored.

(As an aside, I believe that the self-image of the elite Left in the United States involves three historical factors from the 1950s and 1960s. The Left opposed the anti-Communist "witch hunts", supported the civil rights movement, and opposed the war in Vietnam. All three of these elements have proved to be remarkably durable. For instance, the grim consequences of American withdrawl from Southeast Asia had little or no effect on the Left's self-image.)

All of which raises disturbing questions. Given that the revelation of secret information can alter our view of history, what secrets remain concealed even now? Which of our accepted stories will someday be known to be incomplete, or just plain wrong? What heroes are really villains, and vice versa? Which myths have become so entangled with political identities that they will still persist, even if later proven false?

These are disturbing questions because, once you depart from the officially accepted standard stories about things, you enter a country without borders. The nearer parts of that country can be intriguing and thought-provoking, but the territories farther out are inhabited only by lunatics. Beware. This way madness lies.



In our post-X-Files era, when the wackiest conspiracy theories have diffused into mainstream culture, we all understand the idea of hidden history. The ostensible history of the last century, we are told, has been just a facade, behind which a secret story has unfolded. Shadowy groups -- be they Jews, Freemasons, Opus Dei, the Priory of Sion, CIA, KGB, Majestic-12, the Trilateral Commission, or what have you -- plot and vie for power over the decades and the centuries. We only see the faint, visible traces of a gigantic subterranean struggle.

Some of these ideas are pernicious and lead to murder and worse. (How many have died because of the Protocols of the Learned Elders of Zion?) But viewed purely as imaginative games, they can sometimes be great fun. (Think National Treasure -- which, no kidding, my daughter recently watched in history class.) And most of us know that it is mostly nonsense. (In my view, nobody can take The DaVinci Code seriously who has first read Foucault's Pendulum.) Yet the notion of a hidden history, of secret causes behind public events, is far from nonsense. Enigma and Venona teach us this much. It is probable that there are other secrets, unknown to us, that are even today shaping the world we see.

I am a person with a definite taste for this sort of modern mythology, at least in some of its flavors. It has long been a hobby of mine. I happily read about the Knights Templar, the Roswell flying saucer crash, and the latest cryptozoological developments. This predilection of mine does call for a bit of caution, though. These are very dodgy epistemological neighborhoods, and when you visit such places it is a good idea to keep a firm grip on your mental balance. Yet you cannot honestly do this simply by asserting that "Such things are all tommyrot." History does tell us that such theories mostly are tommyrot. But history also tells us that important real parts of history may be concealed from view.

Where exactly is the edge of paranoia? Here is my own working rule: Doubt is not paranoia, but some kinds of belief are. It is not paranoia to believe that your picture of the world is likely incomplete. It is not paranoia to believe that some of the visible history of the world may be driven by hidden forces and secret events. In short, it is not crazy to suspect that some conspiracies may exist. It is crazy to be convinced that the Bilderbergers are tapping your IPod.



We have certainly been living, as the (apocryphal) Chinese proverb has it, in interesting times. Over the past few years, there have been any number of strange and terrifying incidents. The obvious example is the attacks of September 11, 2001. But 9/11 was embedded in a wider web of events, some of them still without a really satisfactory explanation. I have lost track of the number of news stories, initially pregnant with significance, that have led nowhere. Of course, first news reports are often not accurate, so that later on you find out the real shape of things. But what about those stories whose true shape is never made clear at all?

Many of these loose ends have involved Iraq. At the risk of venturing into conspiracy land, let me mention six questions that either have never been answered, or whose accepted answers are not as solid as I would like:

Who is Ramzi Yousef? Yousef, you'll recall, was the key man behind the 1993 World Trade Center bombings, who managed to skip the country and evade capture for several years afterward, participating in several other terrorist plots, until he was finally nabbed in 1995 in Pakistan. He's now in the SuperMax slammer in Colorado, along with the Unabomber and Terry Nichols. But the biography we have been given of Ramzi Yousef may be nothing more than a "legend". There seems to be some reason to suspect that he was actually an Iraqi intelligence agent.

Did Nichols and McVeigh have foreign terror connections? Terry Nichols, one of the two men convicted of the 1995 Oklahoma City bombing, spent a great deal of time in the Philippines in the time leading up to the attack. There are coincidences in place and time between these visits (for the ostensible purpose of obtaining a young Philippine bride) and the activities of Ramzi Yousef and other members of terrorists groups there. A number of people (including Tim McVeigh's court-appointed attorney) have become convinced that the Oklahoma City plot had foreign connections. Why did this not come up in the trials? U.S. investigators and prosecutors were intent on creating an airtight and uncomplicated case against McVeigh and Nichols. In other words, the investigation was pursued as a criminal case (where the important thing is to secure convictions of the bad guys) rather than an intelligence operation (where the important thing is to make all the connections and assess the threat).

Did Mohammed Atta meet with an Iraqi spy in Czechoslovakia? Czech intelligence officials claim that the ringleader of the 9/11 attacks met more than once with an Iraqi intelligence official who operated under diplomatic cover in Prague. U.S. intelligence officials said that they were not convinced this ever happened. If Atta did have such a meeting, why does the U.S. government dismiss the story? (Surely a bona fide link between 9/11 and Iraq would be a political philosopher's stone for the Bush administration!) If Atta did not have such a meeting, why do the Czechs stick to the story to this day?

Who was behind the 2001 anthrax attacks? A few weeks after 9/11, letters were sent to media organizations and congressional offices containing very effectively weaponized anthrax spores. Though the attacks were small in scale, they were the most sophisticated biological terrorist attacks ever conducted. Several people got sick, and some died. A massive investigation ensued. Many people speculated at the time that Iraq might be behind the letters, since the Iraqis were known to be experimenting with anthrax as a weapon before the mid-1990s. After a while, though, the anthrax story sort of disappeared. The attacks stopped. No one was arrested. The attacks remain, at least in the public view, a mystery.

Who was being trained at Salman Pak? One of the creepiest places in Saddam's Iraq was the apparent terrorist training facility at Salman Pak. It included, among other things, an entire parked airliner, evidently for use in practicing operations on planes. The camp evidently trained some hundreds or thousands of people over the years. Salman Pak was unarguably real -- you could see it in the satellite images -- but as far as I know, nobody has given a trustworthy account of just what was going on there.

What really happened to Iraq's WMDs? Saddam Hussein certainly acted like he was concealing weapons of mass destruction before March, 2003. He seemed to be willing to risk his regime to hide . . . what? The widespread conclusion that Saddam still possessed WMDs was made more plausible by Saddam's previous extraordinary efforts to acquire and conceal them. The Americans and British clearly expected to face such weapons in the war and find stocks of them afterward. But instead . . . almost nothing. (True, the ISG did conclude that Saddam had never abandoned his interest in WMDs and had worked to maintain core capabilities in secret. But almost no actual weapons have come to light.)

Here is the accepted wisdom about these six questions, as near as I can judge. Ramzi Yousef was an Islamist terrorist born in Pakistan and raised in Kuwait, probably with family ties to al Qaeda's Khalid Shaikh Mohammed. He had nothing to do with Iraq. Nichols and McVeigh had no foreign contacts or assistance, so Oklahoma City is unconnected to everything else. The story of Atta meeting the Iraqi spy in Prague is spurious, mere bad data. The anthrax attacks were the work of a lone resourceful weirdo, probably an American, but of course nobody knows for sure. Salman Pak was . . . well, there were lots of odd things in Iraq. Saddam was a bad guy, so who knows? But the UN inspections and Bill Clinton's 1998 bombing campaign had shut down the Iraqi WMD programs for good. So by 2003, either Saddam was bluffing on a massive scale and trying to make his neighbors think he was armed with WMDs even when he wasn't, or else the pre-war intelligence reports of Iraqi WMDs were just flat wrong and likely distorted by American political goals. Or perhaps both.

Here are the central points of the conventional story: Saddam never attacked the United States. There were no connections between Saddam and al Qaeda. And there were no WMDs in Iraq.



In 1939, Winston Churchill famously described Stalin's Russia as "a riddle wrapped in a mystery inside an enigma." In an open society, it is really hard to keep a big secret for a long time. (The cryptographical victories of the mid-20th Century, secret for so many years, are a striking exception to this. There were serious leaks, even so.) In a totalitarian dictatorship, almost everything of importance is a secret, forever. As Churchill noted, that makes the behavior of such a regime extremely difficult to understand from the outside. This was true for Josef Stalin; and it was also true for Stalin's great admirer, Saddam Hussein.

But after the dictators are gone, we sometimes have the opportunity to turn the enigma inside-out, to see the evidence and read the archives and understand what happened. The archives, especially, can be crucial, because totalitarian states have a passion for record-keeping. We were able to do put Nazi Germany under the microscope after 1945 and do the same, to a far lesser extent, with Soviet Russia after 1989. I expected that this would happen with Saddam's Iraq, that once Baghdad fell we would begin to unwrap the riddles of his regime. But have we?

No -- or at least not in public. There are some formidable difficulties, of course. The main one seems to be language. The fighting in Iraq and the larger War on Terror continue, so there is an accute shortage of Arabic speakers to wade through the millions of pages of seized documents that could illuminate the workings of Saddam's regime. There may also be political reasons not to expose things too much, too soon. To take a wholly hypothetical example, suppose Russia assisted Iraq in its WMD programs after 1991, in violation of about a zillion treaties and UN resolutions. In 2006, we very much want Russian cooperation in dealing with Iran's nuclear program and in fighting various Islamist groups in Asia. So we may not be all that anxious just now to advertise the extent of Russia's aid to Saddam.

I'm amazed, though, by the lack of apparent curiosity on the part of journalists. They have for the most part accepted the conventional story about Iraq and run with it. The conventional story -- Saddam contained, no link with al Qaeda, no WMDs -- is the basis for interpreting all of the events associated with the Iraq War. It is also a defining doctrine of the anti-war Left. But is that story really as well established as most people seem to believe?



It is not hard to construct an alternative narrative. Here is one. Ramzi Yousef was trained and supported by the Iraqis as part of an unconventional attack on the US in the wake of the Gulf War. The 1993 WTC attack was therefore an Iraqi attack on U.S. soil. The Oklahoma City bombing, on the other hand, was not -- although Nichols may have gained some technical expertise from contacts with Islamists in the Philippines. The story of the Atta visit to Prague is indeed spurious, a case either of mistaken identity or of an unreliable informant. Saddam played no role in 9/11 The anthrax for the 2001 letter attacks did come from the Iraqi biological weapons program, although the actual agents in the U.S. may not even have known that they were working for Saddam (as Yousef's accomplices did not). Salman Pak was part of the larger Iraqi plan for unconventional operations against the coalition that had defeated them in 1991 and imposed such harsh sanctions. We are to this day fighting Iraqis trained there. Stocks of WMDs, certainly chemical and possibly biological, did exist in Iraq until early 2003. Some of these were moved to Syria in the months leading up to the invasion; others were hidden within Iraq, in places that have yet to be discovered. Saddam did not use WMDs in the Iraq war because (a) he was counting on external political pressures to constrain the Americans and the British, and (b) events during the invasion moved too fast for him to change plan. Finally, a fair amount of this is known to the U.S. government. Indeed, Saddam's continuing unconventional warfare was a major impetus for American and British action against him.

True? I have no idea. It seems about as plausible to me as the conventional story. Spin the tumblers and create your own version of hidden history.



I anticipate objections. For some, I must appear to have "drunk the kool-aid" utterly and lost my reason. I am evidently so anxious to defend the Bush administration that I am willing to spin fantasies about terrorist plots and secret undiscovered caches of WMDs. (Actually, if there really are WMDs still hidden somewhere in Iraq, or if we did let them slip away into Syria, I do not see how this fact would be much to the credit of the present administration. But I digress.)

To which I respond, that isn't my point at all. The "alternative narrative" above is not my theory about what happened. The standard story about Iraq, in fact, may be pretty near the truth. I am even willing to stipulate that it is the most likely story. But we must also consider three things.

First, the conventional story is not problem-free. We do not yet know how some pieces of data fit into it. The story looks shaky on a few points. Why should we not be very curious about the loose joints and the rough edges?

Second, because our country is still in the midst of a wider war, we can be pretty sure that the general public has not been told all of the important facts about events and how they fit together. Countries at war keep secrets. (Even if you pooh-pooh the idea of a genuine "War on Terror", there are lots of people in the Bush administration who take it pretty literally, and are acting accordingly. That suffices for my point.) We must accept that information not yet in our possession may change our picture considerably.

Third, we should be wary of the circumstance that the conventional Iraq narrative is woven so firmly into our political and ideological debates. This distorts our ability to complicate the story as we learn more. Let me be concrete. There is good evidence for long-standing links between Iraq and al Qaeda, although these links may never have included cooperation on actual terrorist operations. But in the rhetoric of the present day, this complicated truth disappears behind the simple slogan that "Iraq had nothing to do with al Qaeda". Why? Because that slogan is a stouter stick with which to beat up George W. Bush.

The opponents of the Bush administration and the war in Iraq find themselves in a situation like the one that, maybe, the FBI and the US Attorney faced when prosecuting Nichols and McVeigh. There are lots of strange leads, odd circumstances, suggestive details. On the other hand, the defendent is, as they believe, a wicked man. The thing to do is to try to present the simplest possible case to the jury, because complications may lead them to doubts, and then the wicked man might not be convicted. Since any complicating evidence discovered must eventually become part of the trial, it is best not to dig into the side-issues in the first place. Keep it simple. Don't turn over any unnecessary stones. Only a naive person thinks that the point of a trial, or of a political debate, is to arrive at the whole truth.



A final thought-experiment. It is possible that within the next few months we will learn a great deal more about, say, the disposition of Saddam's weapons of mass destruction. Suppose we learn that six tons of VX nerve agent was moved with Syrian help to the Bekaa Valley in late 2002, where it remains in the custody of Hezbollah. Suppose also that a couple of dozen enhanced-range Scud missiles with warheads designed for biological weapons are found buried in concealed bunkers within Iraq. Would that information really change the conventional narrative? Or, like the innocence of Julius Rosenberg, is that story too firmly entrenched to be altered by mere evidence?

4 Comments:

Blogger aram harrow said...

There's a nearly unlimited number of loose threads and conspiracy theories that we might seriously consider. So which ones to pick?

These papers suggest that our tendency to self-deceive means we do best (in terms of accuracy, though not comfort) considering theories that contradict our priors.

7:20 AM  
Blogger Ben said...

Aram:

I followed your link, which references a NY Times article on MRI studies of bias and an interesting rational choice theory analysis of self-deception. I'm afraid I do not see how either one supports your comment that "we do best ... considering theories that contradict our priors." The example that I cite in my post -- the obstinacy of belief in the innocence of Hiss and the Rosenbergs on the part of the Left, in spite of pretty definitive recent evidence -- may suggest otherwise.

What conspiracies to pick? That isn't the point, or at least not my point. When we have reason to believe that we lack crucial data, we should hold our opinions without an undue degree of certainty. (Elsewhere, I call this a dysfunction of doubt .) In the particular situation I discussed, I think there is reason to believe that we lack crucial data about Iraq and its activities in terrorism and WMDs. Some measure of doubt in the present "standard model" account of the Iraq war is therefore rational and appropriate. No?

10:31 AM  
Blogger aram harrow said...

Hi Ben,
I was referring to the last paragrph of the Cowen article (quoted below), but the point is kind of implicit elsewhere in the paper.
-----------------
Finally, and perhaps most importantly, we should be wary of what our intuitions tell us about political proposals. The self-deception hypothesis implies that when an institutional failure is present, we also tend to feel good about the status quo. It is therefore difficult to spot and market potential policy improvements. So if we are looking to improve the world, through better policy, we should look closely where we might otherwise would look last. We should look precisely at those policies we feel best about.
-------------------

Anyway, since we lack information about a lot of things, we do best (in terms of accuracy) by speculating (this is I guess your 'dysfunction of doubt' point?). But of course we need to think about how plausible our speculations are. And since some speculations comfort us (Hiss was innocent, Israel was behind 9/11, Saddam really did have WMDs and/or ties with al-Qaeda) more than others, we are more likely to find those speculations plausible. So I think that it's not enough to say that we need to doubt the standard model; not all doubts are equal, and our semi-conscious desire for self-deception means that if we ignore this we risk indulging in wishful thinking (Hiss was innocent, etc...).

12:24 PM  
Blogger Joe said...

the defendant is, as they believe, a wicked man. The thing to do is to try to present the simplest possible case to the jury, because complications may lead them to doubts, and then the wicked man might not be convicted.

I would argue that this statement applies to the communications of the Bush administration as much as the opposition. Torie Clarke was on The Daily Show recently, and she made the point that, by trying to keep too much "secret", the administration has essentially lost the credibility needed to keep anything secret. Questions where the answers really are "war on terror" have become indistinguishable from questions where the most facile answer is "war on terror." Ultimately, that increases the appearance of "loose joints and rough edges."

Which, I think, supports your main thesis: it's the responsibility of the citizenry (and, in our system, the media) to keep a healthy skepticism. If the politicomedia class believed that its consumers would keep considering all potentials, perhaps it would be less doctrinaire and more forthcoming.

10:15 PM  

Post a Comment

<< Home