The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography

“[T]he science of secrecy is largely a secret science.”

The author notes at the very outset that even a long and detailed book such as this will inevitably fall short by the simple fact that the subject matter is itself a matter of secrecy and privacy. The most informative details, and the most up-to- date information, are necessarily going to be kept hidden by the only people that know them, and so a book of this nature is only going to be able to divulge so much information.

“[I]n parallel with the development of steganography, there was the evolution of cryptography, derived from the Greek word kryptos, meaning ‘hidden.’”

The very first attempts at passing along secret information involved the mere hiding of the message. The message itself was written in the vernacular, with no attempt to hide the meaning of the written words, but the message itself was hidden somehow. This rudimentary mode of secrecy was quickly outmoded by the actual hiding of the message within the message itself, substituting the letters and characters for others.

“Cryptanalysis could not be invented until a civilization had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics and linguistics.”

The evolution of cryptology could only be achieved by a culture and civilization that had advanced to a level where multiple scientific and academic disciplines could work in concert with one another to achieve the desired result of manipulating language in mathematical ways. The more advanced the civilization, the more advanced its methods of encryption and, conversely, its methods of deciphering encrypted messages.

“Technically, a code is defined as substitution at the level of words or phrases, whereas a cipher is defined as substitution at the level of letters. Hence the term encipher means to scramble a message using a cipher, while encode means to scramble a message using a code.”

A code and a cipher differ in the level to which substitutions are made. Codes substitute larger phrases and whole words: For example, a reverse code system would tell the recipient to “leave at sunset” by writing “come at sunrise,” replacing whole words with their opposite meaning. A cipher, more properly, is a substitution of individual letters, which makes it much more complex.

“Each European power had its own so-called Black Chamber, a nerve center for deciphering messages and gathering intelligence.”

Once the necessity of devoting substantial resources to encryption and decryption became apparent, national powers created specifically designated government offices and teams to work on nothing other than the science of cryptology. When a nation would devote full-time resources and time in this area, it would automatically see results that outstripped their previous, haphazard attempts at espionage.

“The development of the telegraph, which had driven a commercial interest in cryptography, was also responsible for generating public interest in cryptography. The public became aware of the need to protect personal messages of a highly sensitive nature.”

Prior to the development of publicly available communications technology, the public’s interest in cryptography was largely confined to short instances of concealing their intentions in written letters, and even then mostly in simply keeping their correspondence private (hiding letters, etc.). Now that the telegraph was a normal means of communication, however, new methods were devised due to the fact that a third person would need to be introduced to the process. Nobody wanted a telegraph operator to read a private message, and so the message would have to be encrypted from the very beginning.

“In Britain, one of the finest writers of cryptographic fiction was Sir Arthur Conan Doyle. Not surprisingly, Sherlock Holmes was an expert in cryptography.”

In addition to the public’s growing interest in cryptological matters thanks to the telegraph, they were also being exposed to the subject thanks to a plethora of popular literature and stories that involved the act as well. Sherlock Holmes is generally considered to be one of the first and most popular characters to make use of the art of cryptography in the popular mystery novels.

“The mixed blessings of radio—ease of communication and ease of interception—were brought into sharp focus at the outbreak of the First World War. All sides were keen to exploit the power of radio, but were also unsure of how to guarantee security.”

With the telegraph, a message was confined to the telephone line. It went directly from the sender to the recipient. With the invention of radio, however, the message being sent could be picked up by anyone able to intercept the radio signal. This was terrifying new territory in the field of espionage, and so the problem became how to disguise a message that absolutely anyone could discover and read.

“Sun-Tzu, author of the Art of War, a text on military strategy dating from the fourth century B.C., stated that: ‘Nothing should be as favorably regarded as intelligence; nothing should be as generously rewarded as intelligence; nothing should be as confidential as the work of intelligence.’”

The information age was not unique in its prizing of intelligence information. For thousands of years, military strategists have prized intelligence and information above all other things; intelligence concerning an enemy force is the absolute most valuable thing in war and negotiations, and so keeping one’s information secret while having another’s information readily available is the ultimate leverage.

“The onetime pad offers a guarantee of secrecy: the Holy Grail of cryptography.”

Any consistent cipher that is used over and over again carries with it the possibility of being solved, thanks to its consistency and repetition. If every time a message was sent, a new cipher was used, however, it could never be solved since there would be no way to analyze it. There would be a complete lack of anything with which to compare a single message, and thus it would be indecipherable. This is technically possible with a one-time cipher pad, where each message skips to the next cipher, but it is almost impossible to use in practice due to how impractical and unwieldy such a system would be.

“Enigma had arrived, and as the number of Enigma machines increased, Room 40’s ability to gather intelligence diminished rapidly. The Americans and the French also tried to tackle the Enigma cipher, but their attempts were equally dismal, and they soon gave up hope of breaking it. Germany now had the most secure communications in the world.”

With the invention of the Enigma machine in Germany, most of the Allied forces gave up trying to crack its code, as their attempts at deciphering the cipher had been fruitless. Since their enemies had given up attempting to solve the Enigma cipher, Germany felt more and more confident as the war dragged on that they were able to communicate in absolute secrecy.

“After the First World War, Poland reestablished itself as an independent state, but it was concerned about threats to its newfound sovereignty.”

While most of the other Allied forces had given up on attempting to crack the Enigma cipher, Poland continued to make the attempt. Poland had a unique perspective and motivation since they existed in a highly precarious situation as newly established in between two major global powers that historically have been quite hostile to the nation of Poland. As such, Poland was more highly motivated than its allies in finding a way to gain a tactical advantage.

“Enigma was a mechanical cipher, and the Biuro Szyfrów reasoned that a more scientific mind might stand a better chance of breaking it. The Biuro organized a course on cryptography and invited twenty mathematicians, each of them sworn to an oath of secrecy.”

Highly invested in cracking Enigma, Poland expanded its horizons concerning who was recruited to crack the German cipher. Rather than stick to trained cryptologists and linguists, it began to recruit among the best and brightest mathematical minds available, to great success.

“Turing realized that Bletchley was accumulating a vast library of decrypted messages, and he noticed that many of them conformed to a rigid structure. By studying old decrypted messages, he believed he could sometimes predict part of the contents of an undeciphered message, based on when it was sent and its source.”

Alan Turing was the genius behind what came to be known as the Turing Machine, a decoding machine specifically designed to work on deciphering the Enigma machine of the German high command. One of the ways that Turing began to decipher the messages was by focusing on the dates and times of the messages. In this way, he could focus on the human element of the decryption process, which was most likely to contain repetitive patterns and even mistakes.

“For the Navajos, committing everything to memory was trivial because traditionally their language had no written script, so they were used to memorizing their folk stories and family histories.”

Employing Navajo code talkers during the Second World War was a stroke of genius intended to create a real-time encryption of information on the battlefield without having to resort to complicated and time-consuming encryption machines or processes. The use of the Navajo was also made easier thanks to the character of their culture as one that was oral based, giving great importance to the ability to memorize and transmit information orally and in person.

“Lucifer was so strong that it offered the possibility of an encryption standard that was probably beyond the codebreaking capabilities of the NSA; not surprisingly, the NSA did not want to see an encryption standard that they could not break.”

Lucifer was a computer algorithm developed by IBM and threatened to be an impenetrable method of public encryption. The National Security Agency did not like the idea of there being a publicly available method of encryption that they could not crack due to security concerns, especially considering its possible implementation by criminals attempting to circumvent official surveillance by law enforcement.

“The whole problem of key distribution is a classic catch-22 situation. If two people want to exchange a secret message over the phone, the sender must encrypt it. To encrypt the secret message the sender must use a key, which is itself a secret, so then there is the problem of transmitting the secret key to the receiver in order to transmit the secret message.”

The concept of using a secret method to exchange information is a good one: A message can be kept hidden and concealed from prying eyes. The problem, of course, is that in order to establish a secret method of communication, there has to be prior communication to establish the subsequent method of communication. Theoretically, this could go on to an infinite regress, so the question becomes how to privately communicate the means by which one will privately communicate. At some point, in traditional cryptology, communication will have to start off in an unencrypted mode, only to later move to encrypted means of communication.

“Suddenly, the mists began to clear and he had a revelation. He spent the rest of that night formalizing his idea, effectively writing a complete scientific paper before daybreak. Rivest had made a breakthrough.”

The science and evolution of cryptology and new ciphers is not a linear and predictable one. Centuries will pass without any major development, and all of a sudden, a new development might be stumbled upon in the span of a single night; the spark of insight rooted in years and years of research and contemplation can happen in the blink of an eye.

“[T]he success of the Information Age depends on the ability to protect information as it flows around the world, and this relies on the power of cryptography.”

Never before has the world relied so heavily on the ability of people to communicate at any time, in any way, and in any location. The problems are not different in kind, but they are different in degree to such an extent that the 21^st century seems to open a wholly new horizon for encryption to serve the common person in their everyday life.

“As well as protecting the communications of law-abiding citizens, encryption also protects the communications of criminals and terrorists.”

This describes the very heart of modern debates on the ethics of encryption. The dilemma does not concern the act of encryption itself, but the circumstances of those involved in doing the encryption. The tension lies between those who believe all encryption should be available and possible, no matter what, and those who would like to see some kind of restrictions or third-party involvement to ensure that rightful law enforcement bodies and political authorities can ensure public safety, even at the expense of some manner of invasion of privacy.

“In short, there is no reason why we cannot change our policy to suit the political, economic and social climate. The deciding factor will be whom the public fears the most—criminals or the government.”

The debate on the ethics of encryption may take one shape in a neutral or peaceful environment and may take on a wholly other shape when taken up in a hostile environment. The public’s mind might change depending on their immediate concerns. A climate of political oppression will favor the viewpoint of the civil libertarians and absolute freedom of speech and privacy; a climate of political stability in the face of criminal activity might shift the balance toward a more moderate view, whereby the public would sacrifice privacy for safety.

“Only a small fraction of the information flowing around the world is securely encrypted, and the remainder is poorly encrypted, or not encrypted at all.”

Even today, when encryption has never been more widely accepted, available, or implemented, it remains a minority method in reality. The sheer quantity of information flowing back and forth around the globe—primarily thanks to the internet—ensures that most of it remains unsecured or poorly secured. In the future, this is likely to change as the means of encryption become even more accessible, and even automated in many instances.

“If scientists could build a quantum computer, it would be able to perform calculations with such enormous speed that it would make a modern supercomputer look like a broken abacus.”

Quantum technology, at this point in time, is more of a thought experiment and theoretical possibility than it is a real, practical possibility for real-world use. Theoretically, however, the technology would automatically make obsolete almost every mode of encryption and computer process that currently exists. The system of quantum mechanics would create a situation in which the technology would begin to outstrip the most efficient systems of encryption in use today, primarily by making previously unthinkable quantities of calculations speedily possible.