[OSI] Media as the Open Source

Had really wanted to attend the Open Source 101 training they had available–would have made the conference much more valuable to me–and so I requested it when I sent them my information. I was denied, and I tried to wrangle my way in, but to no avail. Instead, I went to this session, with the following presenters:
* Mark Mansfield, Director of Public Affairs, Central Intelligence Agency
* David E. Kaplan, President, Kaplan and Associates
* Dr. Chris Westcott, Director, BBC-Monitoring
* Arnaud de Borchgrave, Director, Transnational Threats, Center for Strategic and International Studies

Just a very quick overview on this one. De Borchgrave gave a stirring condemnation of modern journalism and the disappearance of foreign correspondents from the landscape. He claimed that we had entered a new era of “journalism of assertion” rather than verification, brought about by new technology. An argument that would be stronger, I suspect, if he didn’t write for NewsMax. Maybe he wants to be ahead of the curve.

Westcott provided a nice, structured overview of BBC Monitoring, which employs about 450 people to gather news from local sources. Indeed, this seems to be an interesting answer to the fall off in foreign correspondents: hire locals. He notes that it is important that the people monitoring are close to the news they are reporting on–both physically and in terms of cultural literacy–and that they monitor over time rather than just helicoptering in.

Kaplan begins by arguing that journalists and intelligence analysts are not really that different in terms of what they do and how they do it. He says the major difference is that journalists work in an open system [debatable…] while the intelligence community is wholly insular. A few years ago, analysts didn’t have email or access to the web–some of them still don’t. They don’t talk to anyone outside their narrow community. This is bad, he says: “working in a closed system in the 21st century will kill you.” As it stands, new analysts, weened on the web, are told to “leave your connectedness, your network worthiness, at the door.”

Was hoping for a clear refutation of this from Mansfield. His claim was that “a lot has changed” over the last few years, and that analysts could talk to outsiders, but had to forward press queries to the press office. This is common in most large organizations–let alone the CIA–but it seems that this forestalls any discussion outside the Agency. Mansfield also noted that the Agency hosted conferences and did other things to encourage opening up the discussion, but did not elaborate much in that direction.

At the end, there was a discussion about the fall off of research offices among news organizations, and the moderator noted that the DNI believes more librarians were desperately needed in the intelligence community.

OK, I’ve not blogged the two plenaries. They had a whole gaggle of cameras filming those–will be interesting to see if they release the video over the web…

Posted in Uncategorized | Tagged | 1 Comment

[OSI] Technology: Improving the Use of Open Sources

Presenters (from the Session blurb):
* Steve Selwyn, Deputy Associate Director of Transformation, Office of the Chief Information Officer, Office of the Director of National Intelligence
* Joe Markowitz, Independent Scholar
* Brian Kettler, ISX Lab Chief Scientist and Principal Research Engineer, Lockheed martin Advanced Technology Labs
* Troy M. Pearsall, Executive Vice president of Technology Transfer, In-Q-Tel

I’ll just try to wrap up some of the presentations briefly. Selwyn talked a bit about the process of integrating the various systems available to various agencies, and ways of reducing redundancy.

Kettler spoke on the application of Web 2.0 (take a drink) to open source intelligence issues. He made a strong push for Tapscott’s Wikinomics (which I have yet to read, but someone recently told me was “Benkler Light”). He talked a bit about Technorati, blogosphere analysis, splogs, emotional machine coding, and some of the projects in the TREC conferences.

Markowitz talked a little bit about integrating data and research at different levels. Beginning at the lowest level (open), and allowing queries and results (and machine translation) to propagate to more secure levels in which the analyst can be a bit more detailed in her querries, etc.. This “one-way transfer” allows for insulation against information backwash (my term, not his!). He talks a bit about how it might be possible to link backwards, taking a piece of cleared information and match it with information you can distribute at the open level, to get feedback from non-cleared sources.

Current system allows for a kind of “watched update,” where the analyst can indicate items that are of interest, and a static copy of the page can be moved upward (presumably with the analysts notes), as can a “watch request” that will note changes in those pages. Have to admit, I’m not following much of this as it is outside of my domain.

Q&A

A couple of questions relating to how to get the gov to pay for licenses, there was the beginning of an interesting discussion here on how to value intelligence data, but didn’t get very far.

Someone asked: What specific technologies are coming down the pike now to deal with the phenomenal internet growth problem? Pearsall noted that new search engines are about assigning themes or facets to search results, moving from keyword-only to more contextualized search. There is a subsidiary problem. Search engines tend not to be enterprise software [but what about Google Enterprise?], but rather ad-driven, making it harder to acquire the technology.

Markowitz says this is a challenge to those who aim for unfettered access. The underlying assumption when you share is that the signal will increase more than the noise, and the internet is a counterexample of this. He went on to suggest it may not be worth mining that data [!].

Qestion from the audience: Is there a room for amateur analyst? E.g., an open Intellipedia? [Isn’t that called Wikipedia ;)] Markowitz notes, to the amusement of the group, that “every policymaker is his own analyst… as we’ve learned.” Early Bird represents both the promise and problem here. On the one hand, you can get the raw data out there fast, but then what’s the value added?

(In a later talk, Mr. Naquin, Director of the Open Source Center, noted that the Center articulates with outside experts, including professors and folks like IntelCenter. It seems silly to me to ignore the possibility that good analysis–especially open source analysis–is already happening outside the government intelligence community. )

Is collection actually needed, when the internet is already collecting? Can you do analysis in place? Tools are getting more sophisticated, and so it will bubble up, maybe. Also, remember, that you’ve got to do work to get at the deep web. Markowitz suggests that real analysis usually follows something where you have a hypothesis and then look to confirm it. Google doesn’t do that, but it might be more helpful to have a hypothesis testing search engine.

Someone asked about what happens when people get, e.g., an IP redirect (e.g., al Jazeera looks different for someone coming from a US IP address than it does for someone coming from the Middle East.) I think what he was actually asking was the degree to which the process of collecting open source material might lead to a traffic footprint that suggests the current interests of the US intelligence community, but I may be reading my own question into the question. Not a lot of useful discussion here, but–if indeed that was the intended question!–I would have liked to have heard an answer.

Someone from Booz Allen asks whether there is any intention to have something like Yahoo! Pipes on the secure networks so that analysts can build, deploy, and share their own software tools? Great question! The panelists talked a bit about sandboxes for new product, and the difficulty of trusting code of unknown origin.

Someone from California Dept. of Homeland Security has logins for seven or eight secure sources of “sensitive but unclassified” material. Are they seeking one login to rule them all? Yes, moving toward a single-sign-on, but its not easy. You have both identities and personae (based on where the user is physically, organizationally, etc.). There are reasons for compartmentalization.

A question from someone from “UK Law Enforcement.” How do you find the videos you should be looking at. What about Google’s tagging game, for example? Anything like that happening with video? Tagging only works well when there is a large group who can coalesce around a tag. What is really needed is software that can parse video, but there is enough need that it will be developed. [Aaargh. The ideal shoots and kills the workable!] Publisher tagging is the way forward on this: publishers are trying to get a message across.

Posted in General | Tagged , | Leave a comment

[OSI] Open Source on the Web

Skunk BaxterSpeakers:
* Jeff “Skunk” Baxter, Senior Fellow and Member of the Board of Regents, Potomac Institute for Policy Studies
* Richard P. O’Neill, President, the Highlands Group
* Don Cooke, Chief Scientist, TeleAtlas
* Tim Thomas, Analyst, Foreign Military Studies Office (FMSO), Fort Leavenworth

The format was very short presentations, followed by a Q&A. It’s a little strange to be at a conference where people aren’t really presenting research, but just talking about “what’s going on” and “things to think about” sorts of issues.

Jeff Baxter

Baxter started out with a plea for open source: “human beings are open source,” we gather information from around us from the outset. Despite the section topic, he spoke more broadly about the work of the LAPD anti-gang efforts and how that translates to anti-terrorism. One of the themes of the section was the degree to which those who have been concerned with strategic intelligence are now looking at urban policing (in Iraq as well) to gather information about threats. A second theme was the use of unobtrusive measures and surrogates.

Baxter noted that one of the best sources of information about gangs came from the letters section of Lowrider Magazine. (Apparently I was one of a pair of people in the audience who had ever actually cracked the cover of Lowrider.) Many of their letters came from prison, and from gang members who didn’t expect the cops to be reading it. If the audience here was any indication, that was probably not a misplaced belief. Reminds me a lot of the surprise when people find their boss is reading their blog–something that happens all the time. People write with a model audience in mind, and despite their better judgment, seem to forget that publishing something in a low friction media world means that it is very much out there.

He went on to suggest that there was a desperate need in the intelligence community for cultural anthropologists, and especially, virtual ethnographers. He noted that one of the ways that advertisers have used to come to understand an unfamiliar culture was to watch their soap operas and understand what the culture values and what motivates them. (I won’t touch that but to note that the literature is clear on that front: reading soap operas from outside the culture may not result in better understandings!) He also plugged the Second Life hype (is there anywhere that doesn’t turn into an SL discussion lately?), suggesting that it would be a good place to gather information, as would World of Warcraft. He noted that Chinese dissidents had used WoW to stage demonstrations, and not opening a dialog with them was a missed opportunity.

Richard O’Neill

O’Neill pushed for what he called “strategic listening”, and suggested that the US does a lot of broadcasting, but not nearly as much listening. A radical idea that: not just hearing, but “co-creating a dialog” with those whom we want to learn about. He drew on Margaret Meade’s idea of “smelling the dirt,” and suggesting that we need to not just collect information, but understand the cultural context in order to have good intelligence. “Paying attention is what strategic listening is all about.”

He ended by talking a bit about the long tail, describing what it is, and suggesting that the intelligence community needed to pay much more attention to it, as it promised “novelty, surrogacy [i.e., sources of surrogate measures], and diversity.”

Later, in the Q&A, O’Neill joined Thomas to suggest the kinds of transparency that I had hoped (but not expected) would be a possible direction for open source intelligence efforts moving forward.

Don Cooke

Cooke spoke a bit about the open availability of geospatial data. He talked a little bit about his work, and showed a picture of one of their sensor encrusted mapping van, and moved on to laud the open availability of the TIGER data, and noted that because of US law, even private data tends to make its way out into the public domain. He then went on to discuss the rise of the “neogeographer” and showed some ways in which people are making use of APIs for mashups–something readers of this blog are intimately familiar with.

Tim Thomas

Thomas, an area expert on Russia and China, gave an interesting talk on the role of tracking “extremist militant networking,” and the cybermolilization effort. He suggested that in a couple of cases (discussions over helicopters and UAVs) the information was out there, being discussed, that predicted future strategies. He argues that victory occurs before the first battle, and watching this planning is vital.

He claimed that there are “no experts in this area.” Although there may not be, I got the distinct impression that they are not particularly keyed into the discussions going on among social internet researchers recently. Surely some of the papers that show up at HICSS would be of interest.

Q&A

The Q&As went a little quick, but a couple were particularly interesting.

One of the questions was about terrorists’ use steganography on the web. Thomas (I believe) noted that a lot of the stuff is more nudge-nudge than actual encryption. He noted, as an example, that a photograph of a white stallion (the symbol of al Qaeda) might be shown with a foot raised to signal an attack. Indeed, especially for signaling (rather than more complex communication) that can be very effective.

Another question, from a New York Times reporter (didn’t catch her name) was about the release of analysis to an open audience, and dialog with the analysts in an open way. They noted that they didn’t directly engage discussion of their results, but they did think it was worth getting stuff out there. As one person noted “we all benefit from keeping things as open as possible.” There was a complaint (strange to hear at an intelligence conference) about over-classification. Particularly if data is collected in an open way, it should not be classified. It just makes people re-collect the same data at expense to the American people.

Someone noted and the possibility of new kinds of open discussion and expertise on the web. It didn’t get very far, but this kind of harnessing of expertise is precisely what the open web is good with.

There was also a question about training analysts. One of the panelists argued for more interdisciplinary social science education that draws in multi-methods and perspectives, and draws on a technology component. Hell, yeah.

In all, I hope that the panelists represent a significant portion of the intelligence community. There were some interesting ideas here. Given the breadth of the people attending this conference (when the steganography question was asked, people on either side of me were puzzled as to what “steganography” was), it had to be a very broad presentation. I wish, however, that they could have gotten down to real brass tacks. I hope there is some depth there.

Posted in General | Tagged | Leave a comment

[OSI] Hidden in plain view

I am down in Washington for a couple of days for a conference on open source intelligence organized by the Office of the Director of National Intelligence. I suppose I should mention why I’m here, before moving on to some conference blogging.

I’ve been interested in intelligence, and particularly open source intelligence, since I was an undergraduate political science major. “Open source” was a phrase I considered to be related to intelligence well before I heard it applied to computer programming, and I considered a dissertation on open source intelligence on the web, before turning to my project on Slashdot (which, as it may be clear, was a bit related).

I think that good policy happens only in information rich environments. Reasonable people can disagree over particular policy decisions, but there can be little doubt that good decisions are impossible when there is not a clear view of what the current situation is, and how that compares the environment historically. The intelligence community focuses on strategic intelligence for the defense of the U.S., but the principles are not that different from the application of information gathering and analysis in other policy contexts, or indeed, competitive intelligence in the business setting. Covert intelligence is important, but one of the greatest challenges in both clandestine and open source intelligence is that there is simply too much of it to make sense of in a timely fashion. My interest in open source intelligence fits quite neatly into my overall interest in transparency, self-governance, and learning. It also relates more specifically to my ongoing interest in grokking the global conversation that occurs on the web.

The current Director of National Intelligence, Mike McConnell, has written an overview of his vision for US Intelligence for Foreign Affairs, in which he argues, among other things, that new transparency among agencies, along with a focus on how information technology can help to make sense of the tsunami of open and closed source intelligence. This conference engages that transparency even further, inviting open collaboration with the academic world. That’s part of my interest in the conference, and so I’m here because–despite the fact (or maybe because of the fact) that I don’t know any of the other hundreds of attendees–I’m hoping that there is a space for building bridges and sharing perspectives.

As an aside, it’s an open conference. I haven’t seen anything that says no liveblogging, and there is a concentration of journalists here, so, time permitting, I’ll blog some of the sessions. The standard caveat applies: I don’t make any claims as a stenographer, so you shouldn’t expect a great deal of coordination between what I write and what is actually being said. It’s more a game of “that makes me think of…”

Posted in Uncategorized | Tagged | 3 Comments