The Defence of the Realm Read online

  Praise for The Defence of the Realm

  ‘The most complete history of the agency ever published’ Time

  ‘Andrew’s scholarship is meticulous and extensive. MI5 could not have wanted a better historian than him. He has captured every important detail of the Service, but also its ethos and its place in England as an institution. Buy it’ National Post

  ‘Illustrates through the story of the security service, the way the values of our society and our politics have changed over 100 years’ Jonathan Powell, New Statesman, Books of the Year

  ‘Authoritative history’ The Globe and Mail

  ‘As complete and thorough as such a history may be and as engrossing as any spy novel’ Tim Rutten, Los Angeles Times

  ‘Engagingly successful in bringing the spirit and the personality of the service’s culture and members through its complete survey. The common thread of keen intellect is evident . . . For those with the slightest interest in the intelligence world of the present and indeed the last 100 years, the book is assuredly essential reading’ Edmonton Journal

  ‘MI5 is the first major security or intelligence service in the world to give a historian free range of its records . . . it has been well worth the effort. The Defence of the Realm throws new light on an important area of the running of the country . . . It will be enthusiatically scrutinised by historians, intelligence buffs and conspiracy theorists’ Stella Rimington, Financial Times

  ‘Interesting, engaging . . . A fascinating read for lovers of espionage and security issues . . . Canada makes an appearance in Andrew’s narrative in the well-known story of Soviet cipher clerk Igor Gouzenko who defected in Ottawa . . . Andrew has a sense of humour and has fun describing the early recruits to the service . . . His descriptions could have been the basis for a Monty Python skit’ Winnipeg Free Press

  ‘Compelling . . . an important book’ Irish News

  PENGUIN CANADA

  THE DEFENCE OF THE REALM

  CHRISTOPHER ANDREW is Professor of Modern and Contemporary History and former Chair of the Faculty of History at Cambridge University. He is also Chair of the British Intelligence Study Group, founding Co-Editor of Intelligence and National Security, former Visiting Professor at Harvard, Toronto and the Australian National University, and a regular presenter of BBC Radio and TV documentaries. His fifteen previous books include The Mitrokhin Archive volumes 1 and 2, and a number of path-breaking studies on the use and abuse of secret intelligence in modern history.

  MI5’s self-image at the end of 1917 on a Christmas/New Year card designed by its deputy head, Eric Holt-Wilson, and drawn by the leading illustrator, Byam Shaw. MI5, in the guise of a masked Britannia, impales the loathsome figure of Subversion with her monogrammed trident before he can stab the British fighting man in the back and prevent him achieving ‘Mankind’s Immortal Victory’ – MIV (MI5 in pseudo-roman form).

  (opposite) The Security Service’s all-seeing eye with a slightly unorthodox interwar Latin motto intended to mean ‘Security is the reward of unceasing vigilance.’

  CHRISTOPHER ANDREW

  The Defence of the Realm

  The Authorized History of MI5

  PENGUIN CANADA

  Published by the Penguin Group

  Penguin Group (Canada), 90 Eglinton Avenue East, Suite 700,

  Toronto, Ontario, Canada M4P 2Y3 (a division of Pearson Canada Inc.)

  Penguin Group (USA) Inc., 375 Hudson Street, New York, New York 10014, U.S.A.

  Penguin Books Ltd, 80 Strand, London WC2R 0RL, England

  Penguin Ireland, 25 St Stephen’s Green, Dublin 2, Ireland (a division of Penguin Books Ltd)

  Penguin Group (Australia), 250 Camberwell Road, Camberwell, Victoria 3124, Australia (a division of Pearson Australia Group Pty Ltd)

  Penguin Books India Pvt Ltd, 11 Community Centre, Panchsheel Park, New Delhi – 110 017, India

  Penguin Group (NZ), 67 Apollo Drive, Rosedale, North Shore 0632, New Zealand (a division of Pearson New Zealand Ltd)

  Penguin Books (South Africa) (Pty) Ltd, 24 Sturdee Avenue, Rosebank, Johannesburg 2196, South Africa

  Penguin Books Ltd, Registered Offices: 80 Strand, London WC2R 0RL, England

  First published in a Viking Canada hardcover by Penguin Group (Canada),

  a division of Pearson Canada Inc., 2009.

  Published in this edition with updated material, 2010

  1 2 3 4 5 6 7 8 9 10 (WEB)

  Copyright © Crown copyright, 2009, 2010

  All rights reserved. Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise), without the prior written permission of both the copyright owner and the above publisher of this book.

  Set in 10.13/13.8 pt. Sabon

  Typeset by TexTech International

  Manufactured in Canada.

  Library and Archives Canada Cataloguing in Publication data available upon request to the publisher.

  ISBN: 978-0-14-317458-5

  Except in the United States of America, this book is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, re-sold, hired out, or otherwise circulated without the publisher’s prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser.

  Visit the Penguin Group (Canada) website at www.penguin.ca

  Special and corporate bulk purchase rates available; please see

  www.penguin.ca/corporatesales or call 1-800-810-3104, ext. 2477 or 2474

  Contents

  List of Illustrations

  Foreword by the Director General of the Security Service

  Preface

  Acknowledgements

  Section A

  The German Threat, 1909–1919

  Introduction: The Origins of the Secret Service Bureau

  1 ‘Spies of the Kaiser’: Counter-Espionage before the First World War

  2 The First World War: Part 1 – The Failure of German Espionage

  3 The First World War: Part 2 – The Rise of Counter-Subversion

  Section B

  Between the Wars

  Introduction: MI5 and its Staff – Survival and Revival

  1 The Red Menace in the 1920s

  2 The Red Menace in the 1930s

  3 British Fascism and the Nazi Threat

  Section C

  The Second World War

  Introduction: The Security Service and its Wartime Staff: ‘From Prison to Palace’

  1 Deception

  2 Soviet Penetration and the Communist Party

  3 Victory

  Section D

  The Early Cold War

  Introduction: The Security Service and its Staff in the Early Cold War

  1 Counter-Espionage and Soviet Penetration: Igor Gouzenko and Kim Philby

  2 Zionist Extremists and Counter-Terrorism

  3 VENONA and the Special Relationships with the United States and Australia

  4 Vetting, Atom Spies and Protective Security

  5 The Communist Party of Great Britain, the Trade Unions and the Labour Party

  6 The Hunt for the ‘Magnificent Five’

  7 The End of Empire: Part 1

  8 The End of Empire: Part 2

  9 The Macmillan Government: Spy Scandals and the Profumo Affair

  10 FLUENCY: Paranoid Tendencies

  11 The Wilson Government 1964–1970: Security, Subversion and ‘Wiggery-Pokery’

  Section E

  The Later Cold War

 
Introduction: The Security Service and its Staff in the Later Cold War

  1 Operation FOOT and Counter-Espionage in the 1970s

  2 The Heath Government and Subversion

  3 Counter-Terrorism and Protective Security in the Early 1970s

  4 The ‘Wilson Plot’

  5 Counter-Terrorism and Protective Security in the Later 1970s

  6 The Callaghan Government and Subversion

  7 The Thatcher Government and Subversion

  8 Counter-Terrorism and Protective Security in the Early 1980s

  9 Counter-Espionage in the Last Decade of the Cold War

  10 Counter-Terrorism and Protective Security in the Later 1980s

  11 The Origins of the Security Service Act

  Section F

  After the Cold War

  1 The Transformation of the Security Service

  2 Holy Terror

  3 After 9/11

  Conclusion: The First Hundred Years of the Security Service

  Appendix 1: Directors and Director Generals, 1909–2009

  Appendix 2: Security Service Strength, 1909–2009

  Appendix 3: Nomenclature and Responsibilities of Security Service Branches/Divisions, 1914–1994

  Notes

  Bibliography

  Index

  List of Illustrations

  Plates

  1 Vernon Kell (Hulton Deutsch Collection/Orbis)

  2 Major (later Brigadier General) James Edmonds (National Army Museum)

  3 William Le Queux with his publisher (Frederic G. Hodsoll/National Portrait Gallery, London)

  4 William Melville (By kind permission of Andrew Cook)

  5 Gustav Steinhauer (in disguise) (Steinhauer, The Kaiser’s Master Spy: The Story as Told by Himself, John Lane/The Bodley Head Ltd, 1930)

  6 Winston Churchill, Sidney Street Siege, 1911 (Hulton Archive/Getty Images)

  7 William Hinchley Cooke in German military uniform (Service Archives)

  8 MI9 Chemical Branch staff testing for secret writing (KV 1/73)

  9 Carl Lody (Queer People by Basil Thompson, Hodder & Stoughton Ltd, 1922)

  10 Karl Müller (Popperfoto/Getty Images)

  11 Maldwyn Haldane with Registry staff, 1918 (Service Archives)

  12 Vernon Kell with heads of branches, 1918 (Service Archives)

  13 Staff celebrating the Armistice on the roof of Waterloo House, 1918 (Service Archives)

  14 Letter from Vernon Kell to staff on Armistice Day (Service Archives)

  15 Maxwell Knight (Norman Parkinson Archive)

  16 Jane Archer, 1924 (family archives)

  17 Percy Glading, 1942 (© Metropolitan Police Authority 2009)

  18 Melita Norwood, 1938 (Service Archives)

  19 Melita Norwood, 1999 (Tony Harris/PA Archive/Press Association Images)

  20 Message to Melita Norwood from her wartime controller, 1999 (by kind permission of David Burke)

  21 Christopher Draper with Adolf Hitler, 1932 (The Mad Major by Christopher Draper, Air Review Ltd, 1962)

  22 Christopher Draper flying under Westminster Bridge (The Mad Major by Christopher Draper, Air Review Ltd, 1962)

  23 Wolfgang zu Putlitz’s passport in the name of William Putter, 1938 (Service Archives)

  24 Jona ‘Klop’ Ustinov, 1920 (Service Archives)

  25 Dick White, c. 1939 (Service Archives)

  26 Staff relaxing at Wormwood Scrubs, 1940 (Service Archives)

  27 Wormwood Scrubs office, November 1939 (Mary Evans Picture Library/Illustrated London News)

  28 Vernon Kell at Wormwood Scrubs, 1940 (Service Archives)

  29 Folkert van Koutrik, c. 1940 (Service Archives)

  30 Anthony Blunt in military uniform, 1940 (Service Archives)

  31 Surveillance photograph of John Gollan, 1942 (Service Archives)

  32 Camp 020 (Imperial War Museum HU66759)

  33 Robin ‘Tin-eye’ Stephens (Service Archives)

  34 J. C. Masterman (Service Archives)

  35 Thomas Argyll ‘Tar’ Robertson (Service Archives)

  36 Juan Pujol (GARBO) with MBE, 1984 (© Solo Syndication/Associated Newspapers Ltd)

  37 Tomás ‘Tommy’ Harris (Service Archives)

  38 Mary Sherer (Service Archives)

  39 Nathalie ‘Lily’ Sergueiev (TREASURE) with her Abwehr case officer, Major Emil Kliemann, Lisbon, March 1944 (TNA KV 2/466)

  40 Nathalie ‘Lily’ Sergueiev’s dog, Babs (TNA KV 2/466)

  41 Guy Liddell with his brother, David Liddell (Service Archives)

  42 Victor Rothschild, c. 1940 (Service Archives)

  43 German bomb hidden in a crate of onions, February 1944 (KV 4/23)

  44 Klaus Fuchs (Service Archives)

  45 ‘Jim’ Skardon and Henry Arnold (Service Archives)

  46 Sir John Shaw (Copyright unknown, courtesy of Harry S. Truman Library)

  47 Aden, 1963 (Former member of staff, private collection)

  48 Roger Hollis (Service Archives)

  49 Jomo Kenyatta at Lancaster House, 1963 (PA/PA Archive/Press Association Images)

  50 Gordon Lonsdale (Service Archives)

  51 Harry Houghton and Ethel Gee, 1960 (Service Archives)

  52 Charles Elwell, 1960 (Service Archives)

  53 Evgeni Ivanov, 1961 (Service Archives)

  54 The Soviet service attachés’ address written in lipstick (Service Archives)

  55 Evgeni Ivanov sketched by Stephen Ward (Camera Press, London)

  56 Milicent Bagot with CBE, 1967 (family archives)

  57 Bert Ramelson and Lawrence Daly, 1969 (Service Archives)

  58 Betty Reid, 1972 (Service Archives)

  59 An MI5 observation post, c. 1970 (Service Archives)

  60 Oleg Lyalin, 1971 (Service Archives)

  61 Soviet intelligence officers leaving the UK after Operation FOOT, 1971 (© Mirrorpix)

  62 Registry staff carrying out ‘look-ups’ in the card index, c. 1970 (Service Archives)

  63 Patrick Walker and Stephen Lander, 1984 (former member of staff, private collection)

  64 Cricket score card, 23 June 1984 (Service Archives)

  65 Oleg Gordievsky, 1982 (Service Archives)

  66 Mr and Mrs Arkadi Guk (© Solo Syndication/Associated Newspapers Ltd)

  67 Václav Jelínek, Czech illegal known as Erwin Van Haarlem, 1988 (Service Archives)

  68 Van Haarlem’s kitchen at the time of his arrest, 2 April 1988 (© Metropolitan Police Authority 2009)

  69 PIRA mortar attack on Downing Street, February 1991 (© Metropolitan Police Authority 2009)

  70 Donal Gannon and Gerard Hanratty, Operation AIRLINES, 1996 (Service Archives)

  71 Siobhan O’Hanlon, Gibraltar, February 1988 (Service Archives)

  72 Ceremonial Guard, Gibraltar (Service Archives)

  73 Moinul Abedin, Operation LARGE, 2000 (Service Archives)

  74 Omar Khyam and Mohammed Momin Khawaja, Operation CREVICE, 2004 (Service Archives)

  75 Dhiren Barot, Operation RHYME, 2004 (© Metropolitan Police Authority 2009)

  76 Muktah Said Ibrahim and Ramzi Mohammed, Operation HAT, July 2005 (Solo Syndication/Associated Newspaper Ltd)

  77 Yassin Hassan Omar, Operation HAT, July 2005 (© Metropolitan Police Authority)

  78 Ramzi Mohammed and Yassin Omar at a training camp, Cumbria, 2004 (© Metropolitan Police Authority 2009)

  79 Bilal Abdulla purchasing a gas canister, 2007 (© Metropolitan Police Authority 2009)

  80 Gas canister in failed bomb attack, London, 2007 (© Metropolitan Police Authority 2009)

  81 Operational training (Service Archives)

  82 Jonathan Evans in the Intelligence Operations Centre, 2009 (Service Archives)

  With thanks to the Metropolitan Police for supplying pictures 17, 68, 69, 75, 77, 78, 79 and 80.

  Integrated Illustrations

  Frontispiece: ‘The Hidden Hand’, New Year card, 1918 (Service Archives)

  Title page: The Security Service’s all-seeing eye (Service
Archives)

  xx Cartoon, Spectator, 29 November 1986 (© Michael Heath. Courtesy of the British Cartoon Archive, University of Kent)

  xxxvi MI5 Headquarters, 1909–2009 (all photographs from Service Archives except for 1 and 3, courtesy of the City of Westminster Archives)

  16 German espionage in Essex, 1908 (The Graphic, 15 July 1908)

  22 Memorandum recording Vernon Kell’s appointment to the Secret Service Bureau, 1909 (TNA WO 106/6292)

  24 Vernon Kell’s letter of acceptance, 1909 (TNA WO 106/6292)

  57 William Hinchley Cooke’s War Office Pass, 1914 (Service Archives)

  57 William Hinchley Cooke’s Alien Registration Certificate, 1917 (Service Archives)

  60 Maldwyn Haldane and Registry staff (Joseph Sassoon, Service Archives)

  62 ‘Miss Thinks She is Right’ (P. W. Marsh, Service Archives)

  62 ‘The Lost File’ (By kind permission of R. H. Gladstone)

  64 ‘The Latest Recruits’ (By kind permission of R. H. Gladstone)

  114, 115 Invitation to the March 1919 MI5 Victory celebrations and ‘Hush-Hush’ Revue, 1919 (By kind permission of R. H. Gladstone)

  141 Liberty and Securitý, New Year card, 1920 (India Office Library MSS Eur. E. 267/10b © The British Library Board. All Rights Reserved)

  165 Extract from the Red Signal and DPP memo, 1933 (TNA KV 4/435)

  169 Arnold Deutsch (Service Archives)

  173 Extract from CUSS minute book (Service Archives)

  233 Letter referring to staff at Keble College, 1941 (By kind permission of the Warden and Fellows of Keble College, Oxford)

  243 ‘Susan Barton’s’ letter to ‘Dorothy’ (Dick) White, 1939 (Service Archives)

  271 The Official Secrets Act signed by Guy Burgess (Service Archives)

  290, 291 Extract from a Report to the Prime Minister on Activities of Security Service, 1943 (Service Archives)

  295 GARBO’s fictitious network of agents (Service Archives)

  301-3 Notes on the code used by TREASURE (TNA KV 2/464)

  303 Diagram of the code used by TREASURE (TNA KV 2/464)

  306 German map with false location of Allied forces in the UK, 15 May 1944 (US National Archives)

  307 Map with actual deployment of Allied forces in the UK, 15 May 1944 (US National Archives)