[ISN] Inside Symantec's security bunker

Tue Nov 29 01:46:40 EST 2005

http://news.com.com/Inside+Symantecs+security+bunker/2100-7355_3-5973864.html

By Tom Espiner 
Special to CNET News.com
November 28, 2005

In one of the rolling hills above Winchester, England, is a
decommissioned nuclear bunker that houses Symantec's U.K. Security
Operations Center.

The facility, built at enormous cost to British taxpayers at the end
of the Cold War in the early 1990s, is now owned by the security
company. The popular image of a bunker is a dank, rat-infested hole in
the ground, but luckily for Symantec's team, the interior looks
surprisingly like any other office.

The facility is home to Symantec's U.K. Managed Security Services
team, whose main task is to filter and monitor data fed back from
customers' intrusion prevention systems, firewalls and intrusion
detection systems.

The Winchester team analyzes some 1.5 billion lines of code per day,
said Jeff Ogden, Symantec's director of managed security services for
Europe, the Middle East and Africa. "We spend our lives gathering and
analyzing information and intelligence," he said. "This is an enormous
amount of information, and we're trying to pull it into a coherent
state."

The managed security services team is located in a room glassed off
from the main bunker, which has 15 workstations ranged in three rows
of five. Four large flat-screen monitors, mounted on the wall, face
the workstations. Sky News plays constantly in the background to help
the team monitor the geopolitical situations that may affect the
info-threat landscape.

Tight security

Access to the bunker is closed--even other Symantec personnel cannot
enter the building without prior clearance. Any visits must be
announced at least 24 hours in advance. Symantec customers must sign
nondisclosure agreements before visiting.

Once inside, all employees must log in at a special workstation and
must log out when leaving. Three external cameras have a 360-degree
view of the building. A digital recorder keeps 30 days of backup. The
bunker runs round the clock, staffed by a minimum of four and a
maximum of 15 analysts.

Even the atmosphere inside is highly managed. It is pressurized to 1.5
pounds per square inch greater than outside air pressure, so air is
constantly being forced out--handy if someone decides to drop an
atomic bomb in the vicinity. In the event of a nuclear attack, the air
can be filtered through charcoal, and there are still safeguards in
place against a gas attack.

The bunker has features like a security alarm--two strips of black
plastic with glowing red insides--that's activated if any unauthorized
visitor steps inside the glassed-off internal perimeter, where the
analysts work away. Get too close to the alarm and it bleeps and
registers an intruder.

If anyone gets past that, there's one last line of defense to deal
with. "That's when I appear with a baseball bat," said Gordon May,
Symantec's facilities manager.

Globally, there are 120 million desktops and servers using Symantec's
products, which all feed back samples of malicious code. The company
uses basic agent technology to collect the information, or customers
can choose to send in the information manually.

"We deploy a small agent onto the customer collection point--the
firewall, or the syslog server. The agent is a small piece of software
that collects, compresses, signs and encrypts the data before
forwarding it to us," Ogden said.

The data process

Once the data has been collected, it is sent to Symantec where it is
analyzed and, if there is any danger of attack, a report is speedily
sent to the client. "If the situation is critical or an emergency, we
pick the phone up and say to the customer 'You could be under
attack,'" Ogden said.

All customer information is stored centrally and run through two
filters: a "progressive threat model," which decides whether the code
is a threat, and an "expert query engine." The expert query engine
decides what the threat is targeting, where it's coming from and what
the threat is. This code is then analyzed by a Symantec engineer and
the incident classified according to its threat level:

* Informational: The client has been scanned by hackers, but no more
  action is required

* Warning: The client has been scanned and a vulnerability has been
  detected by hackers

* Critical: The client has been scanned, and vulnerable machines are
  being targeted

* Emergency: There is a possibility of code being deposited on
  vulnerable machines

During ZDNet UK's visit to the facility, an attempted distributed
denial-of-service attack, launched using a botnet in Romania, was
detected.

"We profile the threat by finding out where it's being launched from,
who it's being aimed at and what it's trying to achieve," Ogden said.

On a wider network

The Security Operations Center's Winchester facility is part of
Symantec's global network of information monitoring stations. Customer
data is monitored in five centers. The other four are located in
Sydney, Australia; Munich, Germany; Alexandria, Va.; and San Antonio.

The security operation centers work closely with Symantec's seven
security response centers, located around the globe, in locations
including the U.S., Canada, Ireland, Japan and Australia. Where the
primary role of the operations center is to identify attacks against
customers, the response centers work on a higher level and collate
information from a wider variety of sources.

Along with monitoring viruses directly detected by customers, Symantec
scans 25 percent of global e-mail traffic for malicious code. It has a
number of "honeypot" e-mail boxes, which are accounts provided by
ISPs. They are not used, so anything that ends up there is usually
spam, Trojan horses, viruses or other forms of malicious software.

An attack quarantine system linked to the honeypot network captures
such malicious code. "It is a virtual network that simulates servers,
and so looks like a real network," said Art Wong, vice president of
security response and managed security services at Symantec.

Symantec maintains a list of all the vulnerabilities found across its
network, called Bugtraq. Wong said that it's both a clearing house and
a database of vulnerabilities. This list is shared with other security
vendors to speed up the process of issuing patches.

The threat of botnets

As a leading security vendor, Symantec is well-positioned to identify
future threats. Some of the biggest offenders on the radar at the
moment are botnets, which are extensive networks of compromised
computers controlled by hackers. These botnets are usually used to
launch distributed denial-of-service attacks, which effectively flood
Web servers or e-mail boxes with traffic.

The growth of botnets is a major problem, with a 100 percent increase
in the U.K. since 2004, according to Symantec. The company believes
that right now, the U.K. contains the highest number of botnets in the
world.

"Just over a third of the botnets we've seen are in the U.K.," said
Wong, quoting figures from Symantec's Internet Security Report VIII,
published in September 2005. This is higher than the U.S., which has
traditionally had more botnets.

The high incidence of botnets in the U.K. probably has to do with the
recent explosion in broadband usage and the fact that most U.K. home
users wouldn't know if their computer was compromised, Wong suggested.  
"Maybe there's a slightly lower awareness level in Britain of
botnets," he said. "The IP addresses could come from legitimate
machines that have been compromised by hackers. Maybe the machines
don't have patches, or are not running up-to-date anti-malware
products. Plus, if you have 10,000 machines in a botnet, it's
difficult to track back to each IP address."

Taking control

On average, it takes eight minutes for a new machine to be compromised
when hooked up to the Web for the first time, according to Symantec
tests on a Microsoft Windows PC not running XP Service Pack 2 or
antivirus software.

There is a particular danger for businesses using the same network as
a compromised machine, because once one machine has been infected
behind the firewall, hackers can use it to infect others. "If
attackers manage to infect a machine within an organization, they can
profile additional machines within that subnet. Executable code can be
injected onto other machines to profile the users," Ogden said.

Symantec does not tell those people with compromised IP addresses that
their computers are being controlled by hackers, due to the sheer
scale of the problem. "A botnet can consist of thousands of machines,
and we just don't have the time to contact everyone. Our first
priority is our customers," Ogden said.

However, when it comes to serious incidents, Symantec does support the
police. But the company is keen to point out that it doesn't supply
any direct details on customers. "The information we supply to our
customers belongs to them, and it's up to them to provide information
to law enforcement agencies regarding any suspect activity. When
companies are targeted, it's the customer who initiates giving
information about the offending individuals," Ogden said.

It also supports the police in its efforts to counter botnets. "In the
U.K., the National Hi-Tech Crime Unit has been proactive in trying to
close down botnet activity. We welcome any initiative which closes
down botnets," Ogden said. "We have had some contact with the
authorities in the past, and it works quite successfully."

If a company is the subject of an attack, Symantec recommends it goes
to the police. Symantec will only go so far with chasing potential
criminals. If an attack has been unsuccessful, they are unlikely to be
hunted down, Ogden said.

"If we have controlled and closed down a particular threat to a
customer, there's not a great deal of benefit in tracking down the
individuals who mounted the attack," he said.

Tom Espiner of ZDNet UK reported from London.