Share with your friends


Analytics Magazine

The CIA’s new transparency

Analytics plays a role in opening access.

douglas samuelsonBy Douglas A. Samuelson

The CIA wants the public to know much more about what it’s doing, and analytics plays a key role in the agency’s new approach. This development is not just a few pronouncements to build public support, but a major change in how the CIA and other intelligence agencies will function.

Surprising as this may seem to long-time watchers of the intelligence community, the Central Intelligence Agency (CIA) is serious about changing its policies, practices and organizational structure, trying to move toward open information sharing with other agencies and the public. The intelligence community’s mission is expanding from keeping and collecting secrets to winning at information warfare. Disseminating the story the U.S. wants told is  emerging as a key mission.

Last October, the CIA created a new Directorate within the agency, the Directorate for Digital Innovation. This is the first such organizational structure change for the CIA in more than 50 years. Along with this came the creation of a new position within the agency, “transparency officer.” These efforts are aligned with the ”Principles of Intelligence Transparency” [1] issued this past summer by the Director of National Intelligence, who oversees all the intelligence agencies. The major component agencies of the intelligence community are the CIA, Defense Intelligence Agency (DIA), National Security Agency (NSA), National Geospatial Intelligence Agency (NGA) and National Reconnaissance Office (NRO). The Federal Bureau of Investigation (FBI), state and local law enforcement, and various parts of the Department of Homeland Security are affected and involved as well, perhaps to a lesser degree. This list is not complete.

“The rationale was that technology is moving incredibly quickly, and we need to keep up,” explains Ben Huebner, the CIA’s Privacy and Civil Liberties officer who is also the lead officer for implementing the Principles of Intelligence Transparency. “Now the information we get is not necessarily on paper and can be voluminous, so it’s hard to see how the rules apply.”

Homeland security and analytics, CIA transparency, intelligence community, CIA Directorate for Digital Innovation, Ben Huebner, John Brennan

The historically secretive CIA is moving toward open information sharing with other agencies and the public.
Photo Courtesy of | ruskpp

One striking example of the difference is the intelligence community’s longstanding preference for its people to keep a low profile and its policy of excluding from its premises any electronic devices that could be used to move data from inside to outside. “Most new hires these days have been living on cell phones and social media all their lives,” Huebner notes. “They’re not happy about being told to check their cell phones at the front door and pick them up when they leave. At the same time, we need to know more and more about what’s happening on social media. So we’re reexamining what those rules ought to be.”

Deciding how the rules apply is critical. “We need to distinguish between release and classification,” Huebner says. “Release is actually the easier problem of the two. CIA has an established and pretty effective process on the release side – when we’re looking at release, we have subject matter and classification experts reviewing, and we give them a number of resources to determine what has been classified and released previously. For example, we still need to protect sources and methods.”

So some seemingly innocuous information may need to be kept classified and not released. And many analysts and reporters have had the experience of being asked or directed to redact some parts from reports about what were identified as unclassified materials, as innocuous pieces of information can still be compromising when combined.

Release continues to be monitored and controlled carefully. Even former directors and deputy directors of CIA must submit their proposed opinion pieces to the Agency’s Publications Review Board. “We have folks here at CIA whose full-time job is to make clear and unbiased decisions about whether something is classified or not, and those decisions are binding,” Huebner says. “I spoke to those folks before this interview [with OR/MS Today].”

Ben Huebner

Ben Huebner

Not surprisingly, the CIA does not share the view of some people, at one point even including then-Attorney General Eric Holder, that there was some public service benefit in Edward Snowden’s revelations about the extent of government intelligence collection and potential for misuse. CIA Director John Brennan, in a wide-ranging mass media interview several months ago [2], “disagreed vehemently” with that assessment. No one person, even the director of an intelligence agency, would have all the information needed to make good decisions about what information could safely be released.

“But everyone agrees that we do have a problem of over-classification,” Huebner adds. “The incentives are such that when in doubt, people assign a higher classification level. This affects day-to-day work, because to the degree that we have over-classification, it prevents us from sharing that intelligence even within the intelligence community. This has been one of the biggest lessons learned in the past 15 years.”

Steps to combat this problem include training. “We have made classification training a priority, from day one and ongoing,” Huebner states. “We teach that over-classification can be just as big a problem as under-classification. This topic is on the small list of things everyone has to know here.”

Another step in this area is a new research program: trying to automate classification decisions to some extent. The CIA is working with National Archives and the University of Texas to develop software to suggest classification level using machine learning and big data. “We have had a lot of interest around the intelligence community in how this is going,” Huebner adds. “CIA is doing the pilot, but we’re keeping a large number of people informed.”

Improved classification addresses some of the difficulties now associated with release. “Information is classified at birth because of its content or because of possibly revealing sources and methods,” Huebner explains. “There can be instances when information is classified but not properly marked; that’s a training issue. There is a process, by executive order, for reclassifying material that’s found to be more sensitive than we’d realized – but that’s done only in the narrowest of circumstances.”

As for whether increased pressure and scrutiny from Congress and the news media may have made people within the intelligence agencies more risk-averse and secretive, Huebner responds, “Quite the opposite. It’s been the overriding theme in every task force that the intelligence community needs to find better ways to share information and leverage resources, and at the same time protect privacy and civil liberties. This has been a loud and clear signal to the intelligence community.”

Another interesting set of issues is the rapid growth of information gathering and analysis capabilities in the private sector. “It certainly is the case,” Huebner acknowledges, “that government is differently positioned than the private sector. Amazon and Google have tremendous resources, but they don’t have government power. Government plays by a different set of rules, and should. The intelligence agencies have broader constraints, by statute and executive orders. There are very particular requirements about how we acquire information, what is retained, who has access, and what is disseminated and how and to whom. Those rules apply across the board no matter how we get the information – including buying it from the private sector.”

Of course the new presidential administration may change some policies and practices. Longtime readers of OR/MS Today may recall our coverage of the ideas proposed by then-Maj. Gen. Michael Flynn and two associates, in charge of intelligence efforts in Afghanistan, to overhaul the intelligence gathering and analysis system, encouraging much more integration and cross-training [3]. Now a retired lieutenant general having moved on from Afghanistan to command DIA for two years, Flynn is President-Elect Trump’s pick for national security advisor. Quite understandably, Huebner made it clear that neither he nor anyone else he knows at CIA wishes to make any comment whatsoever about the transition. Nevertheless, without any such comments, we are free to surmise that Flynn’s influence over the intelligence processes will be interesting to watch.

No matter what the new administration decides, the changing nature of the intelligence agencies’ missions and resources, and of the world at large, seem most likely to drive the intelligence community further in its new direction. Hence the CIA’s transparency and digital innovation initiatives appear to be here to stay, of great importance, and of particular interest to analytics professionals who can help automate the summarizing, connection and classification assessment of vast amounts of intelligence data.

Douglas A. Samuelson ( is president and chief scientist of InfoLogix, Inc., an R&D and consulting company in Annandale, Va. He has worked in cybersecurity and cyber-counterintelligence in national security, among other practical applications. He is a frequent contributor to Analytics and OR/MS Today, and a longtime member of INFORMS.


  3. Douglas A. Samuelson, 2010, “Changing the War with Analytics: Top U.S. Intelligence Officer in Central Asia Recommends Massive Overhaul of How Information is Gathered and Utilized,” OR/MS Today, June 2010.

Analytics data science news articles



Related Posts

  • 40
    Organizations spend an average of 5.6 percent of the overall IT budget on IT security and risk management, according to the most recent IT Key Metrics Data from Gartner, Inc. However, IT security spending ranges from approximately 1 percent to 13 percent of the IT budget and is potentially a…
    Tags: security
  • 33
    Tom Willemain, a software entrepreneur, statistics professor and well-known figure in the operations research community, volunteered for a year-long sabbatical tour of duty in the National Security Agency (NSA). He recounts his experience in the recently released book, “Working on the Dark Side of the Moon: Life Inside the National…
    Tags: agency, security
  • 32
    Security analytics solutions are delivering deeper visibility into organizations’ security data than ever before, but deployment and day-to-day usage remain challenging, according to a new Ponemon Institute survey sponsored by SAS.
    Tags: security
  • 30
    According to a recent research report, 88 percent of business, security and IT professionals can't access the data needed to do their jobs effectively. Sponsored by Devo Technology, the report surveyed 400 business, IT and security decision-makers in order to examine what it means to be data-driven in the digital…
    Tags: security


Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →



INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to