Homeland Security Data Management – A Disaster Waiting to Happen? UPDATE 1/8/2009

Posted on December 29, 2009

The reports of what occurred to prevent DHS authorities from catching the Detroit Diaper Bomber before he tried detonate his crotch supports theories of data management that I had discussed December 2009.  The primary cause of not bringing up correct data:

  • Errors such as typos in input
  • Search strategies that are deficient

The above are key to the fact that our system – anyone’s system – will NEVER be 100% able to protect us.  Here is what I consider the heart of the reports proffered yesterday:

A timeline provided by the State Department officials, who spoke on condition of anonymity, showed that an initial check of the suspect based on his father’s information failed to disclose he had a multiple-entry U.S. visa. The reason was that AbdulMutallab’s name was misspelled.

“That search did not come back positive,” said one official, who called it a quick search without using multiple variants of spelling. [CNN.com]

Below is my original assessment of what might have been wrong with data management at DHS


In 2006, the Department of Homeland Security was cited as follows:

Congressional appropriators have directed the Homeland Security Department’s inspector general to investigate one of the department’s data-mining projects, saying it appears to lack clear guidelines and oversight.

In the fiscal 2007 Homeland Security spending bill — expected to be signed by President Bush Wednesday — lawmakers cite concerns over the department’s Analysis, Dissemination, Visualization, Insight and Semantic Enhancement (ADVISE) program. [Government Executive]

It is probably unnecessary to do a followup to the above since the events of last week illustrate difficulties both with data and procedures.

There are of course multiple databases in the departments that are now overseen by Homeland Security (the CIA, the FBI, etc.).  Additionally, there is data they gather from private databases including corporate. Within the Department of Homeland security, applications are built on top of database and file systems.

Here are some of the difficulties with dealing with data drawn from databases that may or may not be integrated with Homeland Security databases:

First, data that should be in a single database is usually scattered about in many different databases.

Second, databases contain dirty data. Some data are wrong, missing, outdated or inconsistent with other data, and so forth.

Third, lots of data are lost, although they are stored somewhere.

Because of differing hardware and software configurations, data migration to Homeland Security’s databases is often time-consuming and the accuracy of the migration often dependent on whoever writes the migration document (instructions for migration).

Further, because the information needs of the FBI may differ from say a first responder’s, there may be a mis-match in the way various departments within Homeland Security configure and use data drawn from even the same source.

That part of HS database used to screen at airports draws their data from the FBI, CIA, etc. and some of their formats (electronically gathered fingerprints) are difficult to migrate and integrate. There are few easy data fits and the lists of agencies and Departments within HS is long.

Other than difficulties gathering, transferring and integrating data from the various sources information on terrorists, there will always be the possibility for the following problems:

  • Poor management, policies and procedures.
  • Data entry errors, including misspelling of names due to their unfamiliarity.
  • Application of an incorrect search strategy relating to specific information needed.

The above are human errors. These are not easily “controlled.”  The best we can do in these instances is to write policies and procedures that narrow the room for error.

Finally, a lighter note.  This past week, procedures were instituted on aircraft that included remaining in your seat one hour after and one hour prior to landing/take-off; removal of blankets and other objects from your lap; etc.

It occurs to this writer that these “rules” have been directed at the situation AFTER a terrorist is on board.  Is the government relying on passengers to make another “save”?  I have not heard of many new rules for passengers to follow at the gate!


Government Executive

On U.S. Homeland Security
and Database Technology
Won Kim, Cyber Database Solutions, USA



VotingFemale Speaks! – Homeland Insecurity Janet’s little problem – Enemy Identification; They are Muslim, Janet, Muslim

HotAir – Video: Now the system didn’t work

HotAir – Video: Is it time for airport security to start profiling?

Frugal Cafe – Napolitano Turns 180 Degrees, Admits DHS System “Failed Miserably” in Preventing Jihad Attack on Flight 253 (video

SisterToldjah – Napolitano backtracks, says system “failed”

FireAndreaMitchell – Of course Janet Napolitano now blames media on her famous “System worked” comment – Comment taken out of context

Moonbat Patrol – The Democratic Attack On Capitalism

LisainTx – Leftwing Corruption of America good, Rightwing Values and Principals bad?

Warrantonegirl – Anwar al-Awlaki and dirt bag Nidal Hasan

Diary of a Mad Conservative – The West need not battle with guilt over banning burqas

Creeping Sharia – CAIR and the 9/11 Mosque

HotAir – EunuchBomber report: a return to 9/10

Frugal Cafe – Airport Insecurity & Terrorism… Michelle Malkin on Team Obama’s Intel Failures, Club Gitmo, Yemen Terrorist Camp, & Body Scanners (video)