Kevin Finisterre admits that he likes to hew close to the ethical line separating the "white hat" hackers from the bad guys, but little did he know that his company's actions would draw threats of a lawsuit from Hewlett-Packard.

This summer, the consultant with security firm Secure Network Operations had let HP know of nearly 20 holes in its Tru64 operating system. But in late July, when HP was finishing work to patch the flaws, another employee of Finisterre's company publicly disclosed one of the vulnerabilities and showed how to exploit it--prompting the technology giant to threaten litigation under the Digital Millennium Copyright Act.

Finisterre, who was not hired by HP, now says he'll think twice before voluntarily informing another company of any security holes he finds.

"As more laws come out, you are going to have to make a decision on which side of the fine line you want to be--black hat or white hat," the 22-year-old consultant said.

In recent months, hackers of all backgrounds have been forced to rethink their practices while facing a roundhouse combination of the DMCA, heightened law enforcement activity and deeper scrutiny by employers.

The issue pits two extremes against one another. At one end are the corporate-security experts who wear their metaphorical white hats because they adhere strictly to regulations and tend to believe that software vulnerabilities should be disclosed only to the software maker or a trusted third party. At the other are the black hats who are generally interested only in gaining access and breaking security.

Who is a hacker? In the most general sense, a "hacker" is someone who enjoys modifying and subverting systems, whether technological, bureaucratic or sociological. Most often the term is used to describe someone who has learned about technology by picking apart systems. In the past decade, however, "hacker" has come to describe those people with a hands-on interest in computer security and circumventing such security.

In the middle are the gray hats, who are finding their once-acceptable acts, such as informing the public of company security holes, could now land them in jail.

Even the White House has weighed in on the controversy. While acknowledging the need for third-party discovery of flaws, President Bush's cybersecurity team believes that more stringent ethics need to be the rule, rather than the exception.

"We are reaching a crossroad where decisions have to be made as to which way people are going to go: Are they going to continue to function as a security consultant or go to the dark side?" said Howard Schmidt, vice chairman of the White House's Critical Infrastructure Protection Board.

That sentiment is echoing across the once-vast gray area where the majority of today's serious hackers toil. With law enforcement and corporate legal departments increasingly on the attack, many security experts are worrying that the next bug they discover or tool they create could get them sued or prosecuted.

"You can't do anything these days," complained H.D. Moore, a security expert and hacker for network protection firm Digital Defense. "It used to be that you could hack a box and people would say, 'Ah, it's just a stupid kid.' Now it's a mission-critical server you just hit, and that's terrorism."

Making the situation more difficult is the amorphous definition of ethical hacking. Although the subject has been addressed extensively in law and ethics philosophy, rarely a month goes by without a debate over whether a particular vulnerability had been disclosed responsibly.

The term "gray hat" was originally coined by the L0pht--one of the best-known old-school hacking groups, pronounced "the loft"--for those who wanted to stand apart from corporate security testers but also distance themselves from the notorious black hats. The category defined by this phrase has come to encompass most independent security experts and consultants, as well as many corporate security researchers.

"We chose the term 'gray hat' to represent the independent researcher who didn't have a vested interest in any particular company or product," said Chris Wysopal, director of research and development for security firm @Stake, a company that had been formed out of the core group of L0pht hackers. Wysopal himself went by "Weld Pond" when he was part of the L0pht.

But others don't believe that a gray area should exist, even for hackers who break into a company's servers only to inform its network administrators about the vulnerabilities--a technique made famous by itinerant hacker Adrian Lamo. He has found his way into the networks of WorldCom, the New York Times, America Online and Excite@Home before breaking the news to the company or, more often, to the press.

To those like Peter Lindstrom, director of security strategies for the Hurwitz Group consultancy, Lamos and others of his ilk are criminal hackers.

"If you are gray, you are black," Lindstrom said. "It's not that I don't understand what they are trying to do, but it comes down to what you are actually doing."

When hackers attack a network, an administrator has few ways to judge their intent. Every incident must be treated as an emergency, Lindstrom maintains, so every trespasser should be treated as a criminal.

That point of view may be in the minority today, but it's rapidly gaining support. The trend is lending new strength to such laws as the Digital Millennium Copyright Act

Cracking down on grays
Last year, the FBI arrested Russian programmer-cum-hacker Dmitri Sklyarov for violating the criminal provisions of the DMCA by producing a program that could circumvent the copy protections surrounding Adobe Systems' e-book format. Adobe forced the issue with the FBI and then backed off amid wide criticism. Now the Justice Department is pursuing the case against Sklyarov's company, Elcomsoft.

The arrest has worried those who find holes in software. At this year's Defcon hacking conference, some international researchers doubted they would attend in 2003, given the turn in the U.S. legal environment.

"The DMCA is so vague and complex and confusing," said Jennifer Granick, a defense lawyer and clinical director at Stanford University's Center for Internet and Society. "This is the most serious problem."

The DMCA has become a favorite legal weapon of the software and media industries to silence critics and security experts, despite exemptions written by the Library of Congress for security research. Princeton University professor Edward Felton delayed presenting his findings regarding the security of several music standards when the Recording Industry Association of America threatened him with a lawsuit.

In addition to the case against ElcomSoft, the FBI is reportedly investigating Lamo for his hacking of a database that contained contact information for New York Times columnists.

Internal affairs
Many security companies, such as Digital Defense, Internet Security Systems and @Stake, trumpet the fact that they hire hackers as part of their cachet. Oracle even maintains a staff of its own homegrown hackers, bringing in outsiders only on occasion, said Chief Security Officer Mary Ann Davidson.

"I use the term 'hacker' mostly in a term of professional respect," she said. "I don't believe in blaming the research community for our own failings, but we should let light in on the situation."

Others, however, operate on a don't-ask, don't-tell policy.

"Companies say, 'We don't hire hackers.' But you go there and they have a room full of them," said "md5," a member of the GhettoHackers, a Seattle-area group of white hats.

Today's security-conscious climate means that programmers and hackers have to pay more attention to politics and laws, a new sensitivity that some believe has discouraged them from notifying companies of vulnerabilities.

"There are a lot of (flaws) still being discovered, but no one is releasing them," Moore said. While lists such as Bugtraq continue to post flaws, he added, "interesting" vulnerabilities aren't being disclosed as often.

The recent experience of Secure Network Operations is a case in point. Finisterre--who also goes by "dotslash"--has not changed his philosophy, but his company has become far more wary of publicizing security flaws. "We are more treading on water when we approach a vendor now, because what HP did scared the crap out of us," he said.

Hats of the future
The debate has given rise to some new possible guidelines for defining hacker ethics. For some time, a hacker known as Rain Forest Puppy has adhered to a policy that spells out how a security researcher and a software maker should communicate. At its core, the so-called RFPolicy guidelines recommend that a software company give updates to the researcher every five days.

@Stake's Wysopal co-authored a more formal set of rules for researchers that advocates more leniency for software makers. Rather than five days, the report asked researchers to give a company seven days to respond and 30 days to make a good-faith attempt to fix the problem.

Oracle's Davidson said such guidelines begin an important dialogue. "Not to excuse ourselves for sitting on our keisters, if that's what we are doing, but to say, 'Step into our shoes,'" she said. "Hackers only have to find one hole to make a name for themselves, but we have to find all of them."

And as companies and law enforcement agencies focus increasingly on the vulnerabilities of critical networks and systems, those considering themselves gray hats may not have much longer to play in the middle of the road.

"I think that we have seen a shift in people and their focus to do the right thing," said Schmidt of the White House cybersecurity team. "No matter what color your hat, you need to realize that there is a greater dependency on networks today."