Security Manager's Journal: Breached!

13.02.2006
It finally happened. We had a security breach that could have severe ramifications for a state agency.

I was packing up to leave on a Friday when the webmaster came into my office and shut the door behind him. It was unusual for him to be in the office so late, and he looked particularly nervous. So I took off my coat, set down my briefcase and sat down. He refused the chair I offered him.

"OK, what's going on?" I asked.

"Well, uh, I think we have a problem with one of our Internet Web sites, and I'm afraid to tell you about it, but I think I have to, and I've already fixed the problem, but you might need to know about this, since you are the information security officer," he rambled. I held up my hand as if to say "Stop," and he collapsed into a chair with tears in his eyes.

I have dealt with plenty of security incidents in my time, and I couldn't imagine what could be so horrible that he was afraid to tell me. I smiled and told him to take a deep breath and start from the beginning. Here's what he told me: An employee was doing a Google search on the name of a client of the agency, when up came the URL for an agency directory. She clicked on the link and, lo and behold, the supposedly password-protected page appeared with the client's Social Security number on it, even though the employee hadn't been asked to log in or use a password. Social Security numbers are "personally identifiable" information, as defined by the Health Insurance Portability and Accountability Act (HIPAA), and we're subject to its security and privacy rules.

The employee immediately called the webmaster, who started reviewing the file structure, moving files and changing permissions, all the while generally panicking.

This was huge. It was my turn to take a deep breath. Why, I wanted to know, are we storing client data on an external Web server? That flies in the face of everything having to do with security! The answer: It had always been done that way. I took another deep breath and pondered some realities. Our inexperienced webmaster is responsible only for content, while our Web site management is outsourced to the state-level webmasters. Our Web sites are hosted by the state in its data center. With so many cooks, it's not surprising that a disconnect of this sort could happen.

Before he left, I told the webmaster, "This weekend, you cannot allude to this even in casual conversation unless you want to see our agency on the front page of Monday's paper -- understood?"

There was nothing that could be done over the weekend, and the immediate error in configuration had been fixed. I needed to think about what steps to take. I knew that the law states that an "unauthorized disclosure" has to be reported in a timely manner and that all persons whose personal information is compromised must be notified. And I had developed the incident response policies and procedures, so those didn't worry me. But a political misstep would be painful for our agency.

On my way out the door, I dialed my boss's cell phone number but got no response. That was OK; I wasn't ready to talk to him yet. The weekend was a sleepless one. I tried to distract myself with family duties, but I thought about the incident every minute.

Monday Morning Blues

On Monday, I fought the traffic to get into the office early. I hadn't been able to get in touch with my boss over the weekend, but I hadn't tried very hard either. Now, I stepped into his office and gently tapped on the door.

Then I really got his attention by telling him that a security incident had come to my attention on Friday evening and I needed to get him up to speed before he heard about it from someone else. I told him what details I had at that time and explained that I would at that point be following policy and procedure in handling the investigation. But he needed to understand that under HIPAA and state law, if the results of my investigation turned out unfavorably, we would be required to inform all our clients that their personal information had possibly been compromised.

After he absorbed this disturbing news, I asked him what he knew about the site's design. The answer: very little. (My boss has a background in software development and programming, but I wasn't surprised that he didn't know much about the architecture of this site.) I said the site would have to be redesigned, with several layers of security added. The least of that would be making sure that personally identifiable information resides in a database behind a firewall, not on a public Web server. Then we spent a few moments commiserating, since we are both relatively new to the public sector and are still prone to making assumptions about the way things are done. For example, we both assumed that a state-run Web site would be constructed properly.

Looking to the Future

I'm in the midst of the investigation now. The key will be determining whether any "unauthorized" disclosures were made. The employee who stumbled across the problem is authorized to access the data in the directory, so there's a chance that no unauthorized disclosure took place. Right now, I'm searching through a year's worth of Web site logs and identifying the source IP addresses from which the various URLs were accessed. I have also imported the Web site to one of our local servers in order to perform a security review. I'm hoping I find nothing worth reporting.

What Do You Think?

This week's journal is written by a real security manager, "C.J. Kelly," whose name and employer have been disguised for obvious reasons. Contact her at mscjkelly@yahoo.com, or join the discussions in our security blogs: computerworld.com/blogs/security

To find a complete archive of our Security Manager's Journals, go online to computerworld.com/secjournal.