Friday, September 26, 2008

Why Doesn't Government Get Emergency IT?

Why Doesn’t Government Get Emergency IT?

David Aylward, September 26, 2008

Achieving integrated and interoperable emergency response systems requires that the participants connect at the transport layer (communications pipes connect) and at the application layer (simplistically, software and its interactions). (For purposes of clarity of discussion, I am simplifying to two layers in the stack).

For the past several years, in the emergency space there has tended to be a very strong focus on the transport layer, almost ignoring the application layer. This translates into "building interoperable emergency networks and systems" as opposed to "linking legacy systems with software." The first is very expensive, and can't be the solution anyway as all the relevant organizations are never going to all be on the same network, using the same radios. But yet we continue to pour billions of dollars into building new pipes, while starving the application side.

In industry terms (again simplistically), we have been choosing the telecommunications industry over the information technology industry.

The commercial and military worlds are way, way ahead of what I call the "virtual safety enterprise". They are well down the road towards network-centric operations, cloud computing, managed services, service oriented architectures, and the like.

This imbalance is a very, very big deal because the only way to make rapid progress on inter-domain, inter-jurisdictional, and inter-everything else safety information sharing is to focus on the application layer: convert every communication into Internet Protocol and focus on what needs to happen "in the middle" and with "interfaces to the middle" instead of the end points (what happens in and at different agencies). (That doesn't mean transport is unimportant, but we already have lots of it, in lots of different flavors. My argument is not to ignore it, but to have balance.)

I have been ruminating on why we have this imbalance. Why do so many in the Executive Branch, the FCC, and emergency response communities make this choice, focusing almost solely on transport: in talking, writing, policy making and grant making?

It struck me that the answer is in our history since around 1978.

"Communications" has been under the purview of government for a long time. Information technology mostly grew up outside of government (DARPA aside). So people in government in this area learned communications; IT has been a side show.

I grew up professionally with a Congress and FCC that focused on wired and wireless pipes. All our discussions and debates were about the pipes. The great battles of the 1970s and 1980s over competition were about competition in telephony. In the late 1970s, in the Computer II decision, the FCC explicitly declined to regulate information technology, information services. Those were "those other things". From time to time it has drawn the same line deeper and deeper.

And until recently, the IT industry was very happy to grow up in California and elsewhere and not have to go to fundraisers for Congressmen every night.

"Public safety" at the FCC for decades has meant (almost solely) spectrum for radios for first responders. Thus, we have witnessed a great debate in and around the FCC this year about developing an "interoperable wireless broadband network for public safety." (It seems to have escaped the attention of many that safety agencies don't exchange much data today, much less with the field, much less amounts of data that require broadband.) Almost every time a reporter writes on the topic or a Congressman addresses it, they note it will be a solution to the first responder interoperability problem. Little to no attention has been paid to the critical application layer issues that would allow data to start flowing between agencies on the broad band networks that already exist. Nor has the government made a top priority the application layer issues needed to link legacy radio and wired networks to each other, much less to the new P-25 digital trunked radio systems that billions of our tax dollars are being spent for, much less this new broadband network if and when it gets built. (But kudos to the small progress being made due to grassroots leadership).

The FCC isn't just regulatory. It controls large amounts of money. It recently announced it was spending over $400 million of the Universal Service Fund on rural medical networks -- every one of which was mostly new transport capacity.

When DHS was formed and initially reached out to the traditional first responder community, not surprisingly it got the same communications-focused answers. Its policies ever since have reflected this. Billions of dollars are being spent directly and through the states and localities to buy new emergency networks (mostly radio); a few tens of millions have been spent on application layer issues (and mostly on end point, or area specific, applications, which should not be a priority).

The National Telecommunications and Information Administration at the Department of Commerce is about communications and spectrum. So when Congress handed it the lead role on the special $1 billion in "interoperability grants", what did it do? It let those one time dollars be spent almost entirely to buy new radio networks. After major lobbying, software solutions were allowed and cost/benefit analysis would have required them, but NTIA had no stomach for that. This was in part because it does not have a great deal of IT expertise, and that is because it doesn't have real IT jurisdiction.

The safety market is relatively small, and so balkanized in its decison making and purchasing (120,000+ individual agencies), that it is not an easy market to crack. Nor are the individual domains (EMS, 9-1-1, fire, police, transportation, emergency management) calling for integrated emergency information services amongst all of them. Nor can I find anyone in power in government taking that overall view. After all, as discussed above, most of those in government have communications training and responsibilities. Federal budgets and programs are about communications. The government IT people are generally elsewhere.

So it has been simple for the big IT players, who now have large DC presences, to mostly ignore the safety market.

If Google, Yahoo, Microsoft or their ilk took on the safety market, treated it like a virtual enterprise, and developed standards-based managed application layer services for it, could they cause huge leaps forward in service to the public in emergencies large and small (and major overall cost savings)? Absolutely. Could they make a pile of money breaking down the wall between the public and emergency response (e.g. making sure my health records stored at Google were available to 9-1-1, EMS and the trauma center when I get hurt). You bet.

But from their perspective, sacrificing the small submarket today and a larger potential future one are cheap prices to pay to avoid the federal world the telcos have to inhabit. Plus they have a lot of other things to do.


Pete O'Dell said...


Some very good thoughts, and having spent the last 6 years in the application space, I think I have a few insights.

Applications are hard to do well, and the interoperable data structures that will allow radical breakthroughs in sharing between organizations are just getting their legs (Common Alerting Protocol, RSS, KML, NIEM, SAR, HAVE and others).

At Golden Phoenix in San Diego, we were able to integrate mulutiple application feeds into an overall Common Operating Picture by "snapping together" versus welding - with a corresponding reduction in time and complexity. The standards in the data allowed sensors, 9-1-1, people, mobile devices to all blend together in one inforamtion application.

It has long been in the interest of software applications to get "lock in" or a proprietary set of data structures - standardization leads to more competition (good for the customer, tougher for the creators).

The IP infrastructure, enhanced by some of the IPV6 capabilities, will spawn a new generation of interoperable applications working off shared, interoperable data.

Kudos to NOAA for their data interface - it lets anyone access what they need and use it to obtain their results - in a very standardized format.

There is an upcoming Yale article - Government Data and the Invisible Hand that outlines much of this in a very good way (you can google an early version).

Pete ODell
Swan Island Networks

Gary Ham said...

As David said, there is lots of physical bandwidth available. There is also equipment. There are also applications. The least addressed need is information level interoperability between diverse systems an platforms. This is primarily the result of classic vertical stovepipe system development. Folks buy a system to do X and it does X very well, but does not talk to other systems who do Y (and often not even to other systems that do X). In emergency situations ALL emergency systems need to exchange certain basic information (warnings, resource needs, situation reports, etc.) with each other. Standard communication message structures are being developed, but they do not solve the problem by themselves. What we need is a non-proprietary, level playing field middleware network with well established consistent standard interface definitions that can be used for communication of information between other wise proprietary systems and networks of systems. My thought is that this might best be provided as Federal Government Infrastructure; sort of an Interstate Highway System for Emergency Information. Such an implementation, however has to be simple and straightforward to use. If it even appears complicated, adoption will be difficult. And if bureaucracy gets in the way....... (I do not even want to think about it.)

David Billstrom said...

David, as you know I am almost always in agreement with your thoughtful point of view... and I am this time too.

And I agree with Pete's comments here as well.

The mistaken impression taken by elected officials and then amplified by a media eager for a technical solution to the tragedy of non-interoperable voice communications in horrific incidents... is understandable, but regretable. I have spent years talking with police chiefs, fire chiefs, and cops on the streets and firefighters, and I can report that they are concerned about interoperability... but they are terrified about operability.

The #1 issue is operability, which translates to 'capacity' (number of users talking at once) and 'coverage' (geographic area where the voice comms work).

Only very expensive systems in dense urban areas offer the operability the first responders need. There are 2,000 of these systems, all of which are outdated and in the process of expensive upgrades. How expensive? construction costs on the order of $200-300K per user. That's not a typo -- check out the facts. Any of the 25 major metros in the u.s. have $25-45M radio systems for 2,000-5,000 users on duty at any one time.

Moore's Law has not been applied to public safety communication systems, obviously.

Interoperability is a distant #3 priority, and an important one (in my view anyway, I've dedicated my career to it).

As you have pointed out before, if we address this challenge in software, then interoperability is a happy side effect. Practically for free, if software best practices are followed.

In my view, NTIA utterly failed to deploy the PSIC grant money effectively not only in the way you describe, but also because of pressure from the always-politicized grant process. A pity that they didn't take advantage for business-not-as-usual. I have yet to see a PSIC grant for the Congressionally intended (and in the law) purpose of new technology. So far, all are for buying more radios of the usual type.

The truth of the matter in the $2.5B of DHS grants (prior to the $1B PSIC grant via NTIA) is that nearly all of the funds have gone to buy more radios, at traditional pricing and with traditional solutions... and I have yet to meet a local public safety agency anywhere in the U.S. that got a new radio *system* from the grant money. Four state governments got new systems, but no locals.

Although locals provide 78% of the response in the U.S., they did not receive the $2.5B... at most they got a handful of new radios for their old system. Ask any rural cop or firefighter you meet to confirm this simple truth.

I am in full agreement, it is time for a massive policy change regarding emergency IT.

David Billstrom