Security Best Practices

Changing Times

We'd like to revisit a couple of themes briefly.  As we've noted, people's thinking about security is changing.

One theme is what might be called "Classic Firewall Design" (and changes to it). The idea to Classic Firewall Design is roughly that you have some sort of perimeter which is guarded. Some people have done this with the true classic approach, a couple of firewall entry points, well-guarded. Others have scattered firewalls about all their external entry points, which begs the question, is it an architected design or a quick fix? We question the wisdom of having many points of entry to the network, especially managed by different groups. It's better than nothing. It's reasonable as an intermediate state. But it may not be where you should wind up for the long term.

Having said that (and perhaps offended a few), the Classic Firewall Design is the hard shell around the soft chewy middle. This design model can unravel in various ways.

One situation that causes issues with this design model is if different groups wish to share links but remain isolated from each other. There's a tension there: networks are about connectivity, and security is to some extent about denying connectivity. You'd like to share a link, but if you do, you share security and access (or have to maintain lots of access lists).

One solution is to use MPLS for isolation of routing even when shared links and routers are used.  Large organizations with parallel links and routers owned by different Business Units but in common locations might want to consider moving to an MPLS architecture to cut WAN and support costs, and simplify routing.

The second change to the Classic Firewall Design stems from recent infestations of viruses and worms. The recent worm storms were often caused by laptops importing the problem into the corporate environment. The Classic Design is static, assuming the enemy is outside. You can think of it as a castle wall and moat. But if somebody inside opens a door to the enemy, or if there is a turncoat within, the enemy gets free run of the interior.

Consider that classic castles had separate compartments to them, so that even if the enemy breached parts of the castle, the remainder might be defended (probably with a diminished number of combatants).  Does your network have internal barriers so that the enemy within only gains partial access to the interior?

We also now see an emphasis on what might be thought of as security and id checks at the gate: virus scanners, personal firewall enforcement, and so on. Some related links are in the following table.

Cisco CSA has been receiving a lot of customer interest lately. The Network Admission Control program shows Cisco was listening to customers saying they needed control of what connects to their network (and seeing that it may work better if it is not purely an end-system software solution).

Cisco Security Agent	http://www.cisco.com/en/US/products/sw/secursw/ps5057/index.html
Cisco Network Admission Control Program	http://www.enterpriseitplanet.com/security/news/article.php/3111341
Cisco Press Release about Network Admission Control	http://newsroom.cisco.com/dlls/prod_111803d.html

Perhaps security designs should also include some common sense access lists and controls, recognizing that employees in most cases do not need to send all types of traffic to any other location in the network.  (Exception: IP telephony and other peer-peer networking tools).

Up Front

Ok, so what are some up front best practices that you should implement? The following are some of the things you should be considering:

Tripwire. After you've been hacked, it may be hard to confirm what, if anything, has been compromised. Products like Tripwire (commercial or freeware version) snapshot the clean state of the system using file checksums. You can then re-scan after compromise and see what's changed. Otherwise, the only way you can be really sure the system is clean is to rebuild it from scratch or from backup, both of which are time-consuming.

Pre-scan servers. If  you use "netstat  -a" and/or nmap to scan your servers or systems for services and open ports, then you have a baseline. You may also notice any unnecessary services that you can permanently disable. Trojan horse programs, keystroke loggers, and other malware may want to make themselves available for the hacker to access from outside, or they may connect to an external system, perhaps to send email. Scanning for open ports and comparing with the baseline shows any permanently open connections. Unfortunately, intermittently open ports may not be caught by this. Still, doing this catches all but the more competent malware programs.

Shut down unnecessary services. Services you don't use and don't secure are a compromise waiting to happen. Shut them down!

Understand applications/data flows. If you understand the use of applications and the data flows within your organization, and manage to that, you'll be in much better shape. This is useful for other things as well: capacity planning, and QoS policy determination and implementation. If you don't know how your network is being used, you don't know what to expect, how to secure the internal network, nor how to spot traffic that does not belong. Take some time and start getting to know your network. If multicast is enabled, find out what that mysterious multicast traffic is (most networks have some).

HIPS and Personal Firewalls. This is not about hip-ness, but about Host Intrustion Protection (see CSA, above). Consider some form of protection for desktops and especially laptops, so they don't get hacked, infected, or compromised. Strongly consider extra protection for your servers, because if they get infected, you're not able to do business.

Policy. Do you have a security policy? Is it readable? Have you put it out where your users can read it? Do you remind them to read it periodically? Is an educational effort being made to get employees to understand the policy, buy into it, and be security-conscious? Or does your staff use common words or names as passwords, write them down, give them to unidentified persons on the phone, etc.? Does your staff understand the consequences for the company if security is violated? Can they readily describe actions they might in good faith take that might compromise security? Are the personal consequences for those who get sloppy? If you answered "no" to a few of these, you can throw all the technology you want out there, and you still won't be secure. People and policy and education are important for security!

Risk Analysis. Have you thought about security priorities: risk analysis? What's a hacker most likely to target? What does the most harm if compromised, altered, deleted, published to the outside, or copied to the Internet? What systems produce the most value to the organization, or would cost the most to restore or replicate? If you haven't at least thought a little bit about this, you're shooting in the dark. It's better to look at your budget, allocate it across gaping holes and high risk items, and work with upper management on what you can't cover. Either you get the budget or you're covered if something in the "B list" gets hacked.

Incident Planning. Have you thought about what you'd do if hacked? Pulling the plug on the Internet or server farm may well not be the answer: it is a self-inflicted Denial of Service. What's the information and approval chain for the various kinds of security events? How and when do you protect evidence? What are recovery priorities? How is the decision about involving legal authorities made? Who would you contact if you do wish to pursue civil or criminal legal recourse?

Disaster Recovery and Business Continuity. How thorough is your DR and BC planning? It doesn't do any good if the backup servers are up and humming at Sungard or wherever, if there's no place for the staff, phones, desktop computers, and so on. If you cannot connect users to DR site and/or the rest of the corporate network with adequate bandwidth, there's also a problem. DR planning is a team effort, and each fiefdom can't just say "our piece is done" without working with the others.

Network and Application Architecture. There is still a lot of accomodation of the existing infrastructure, both as far as network/firewall/DMZ, and as far as just opening ports in the firewall for whatever applications are deemed needed on the outside, by business partners, etc. This is often due to junior network or server security staff being over-ruled for the sake of revenue. Revenue is pretty important, it's what pays the rent. But if you just keep opening holes in the firewall, you end up with a sieve, not a firewall. Really, each application should be planned for how it fits into the network, into the security architecture. And long-term planning should be considering minimizing the applications and exposure to the Internet and business partners. For example, the crudest approach is to allow business partners right through the firewall and to the same servers as the rest of the business uses. A more secure approach might be partners to use an HTTPS server in a DMZ, with some form of authentication (vary depending on what level of security is appropriate). The partner would then run an application or use web forms on the HTTPS server, and only that server would be allowed through the inner firewall to the actual database servers or mainframe. Costly, yes. So maybe it doesn't happen overnight. But the alternative is trusting the security of all your partners. If you know your firm has cut some corners, what are the odds a business partner has cut even more corners?

Use the DMZ. This one may sound a bit silly, or not worthy of mention. But we've seen some sites with a fine firewall and DMZ, but where the actual servers haven't yet been re-located to the DMZ. We're all short on time. This is something that needs to happen ASAP. You may feel that you don't have the time. But the alternative is having to make the time when valuable data, credit card info, or customer information goes out the door. Possible with economic or image or other damages accompanying them.

WLAN Security. This is a big topic. Suffice it for now to say that physically limiting signals is hard and doesn't work at all well. And WEP is inadequate for corporate security. You need to design for WLAN Security, see Pete's seminar and article on this topic, at http://www.netcraftsmen.net/welcher/seminars/wlan-security-02.pdf.

You also cannot refuse to play. If you don't get out in front on implementing a secure WLAN, and promulgating a WLAN security policy, others with deploy rogue WAP's. And then you almost certainly will have the equivalent of the proverbial "Ethernet jack in the parking lot" for security.

Work Together. It's becoming more clear that no one group can do it themselves. Nor can you stick your head in the sand and say "I've secured the network, and it's up to somebody else to secure the servers". Security is as good as the weakest link. And there are some aspects that are best handled on the computer side of things, and some best handled on the network side of things. If all the teams work together, you'll have tighter security and less work all the way around. Start talking to folks in the corporate security, network, server, desktop, voice, and other teams! Believe it or not, they're nice folks too, and just trying to do their job as they see it!

Host Computer Best Practices

Most companies require a virus scanner. Automated signature updates are good, but the worms and virii are now spreading too fast to count on those. So some are now considering the HIPS, see above. The point is to protect the system without relying only on signatures. A personal firewall is an alternative. WIth both, some end-user education has to be done.

One of us has been using a PC personal firewall product. It starts getting annoying every once in a while, and you have to resist the urge to turn it off, permanently. But it's also very tempting to just click on "Permit" every time it pops up a dialog box to ask about an inbound or outbound connection. Nowhere does one see anything really making one realize that you are creating permanent rules for what to allow or block, and that one ought to deliberate before doing so. We suspect the average user just clicks on "Permit" and over time their PC becomes more and more vulnerable. What do you think?

Concerning Microsoft and security patches, we think  most now realize companies need to actively push patches and updates to desktops. This is going to be very interesting the first time Microsoft goofs, with a buggy patch that disables DNS or the Ethernet NIC or some other fairly critical functionality required to get more patches from Microsoft. No, you don't want to trust any outside entity, especially Microsoft, to detemine the need and remotely update desktops. That sounds like a security gap waiting to be exploited. That's always the problem with any central patch/updater tool. The other problem is the exceptional people (like us), who maintain their own PC's, which may not be the same as the corporate desktop build.  You don't want to accidentally trash those users' PC's.

Virus and security scanners	http://housecall.trendmicro.com/ http://security.symantec.com/sscv6/ http://www.symantec.com/ http://us.mcafee.com/default.asp
PC firewalls	http://www.zonelabs.com/store/content/home.jsp http://blackice.iss.net/ http://www.tinysoftware.com/

Server Best Practices

Just a few thoughts, since our expertise lies more on the network side of things.

First, make sure you have a good and thorough backup scheme. If human thought is used often, then chances are you're not fully backed up. Tapes, etc. are cheaper than not being backed up! Also make sure you take backups to a secure off-site location regularly. And do not bring your most recent offsite backup back to the office before you take one out of the office: rotation should always involve sets of at least 3, so that the offsite copy only returns to the office after the latest backup or copy has arrived at the offsite storage.

Just like desktops, you shoudl get corporate servers under central patch and software management, preferably automated. These systems are fully under IT control, so there's no good reason this shouldn't be done (other than perhaps ability to test patches impact on key applications before deploying them).

The data center should have good physical security, backup power and A/C, etc.
Some central oversite is appropriate for firewalls, servers, etc. If different regions have separate responsibility, chances are one is going to be a lot less strict and a lot less secure than the others. It's just human nature!

Test yourself! Use Nessus, CIS tools, nmap, and other tools, from inside and from the outside. And as we noted earlier, do consider using Tripwire or some other tool to capture the state of your system, so you can detect changes.

Nessus	http://www.nessus.org/
Center for Internet Security	http://www.cisecurity.org/
Nmap port scanner	http://www.insecure.org/
Insecure.org's tools page -- lots of good links!	http://www.insecure.org/tools.html
Commercial, supported version of Tripwire	http://www.tripwire.com/
Free version of Tripwire 2.2.1 (10/2000) from Tripwire Inc.	http://www.tripwire.org/
SourceForge Tripwire Project	http://sourceforge.net/projects/tripwire

Network Best Practices

Have a clear architecture. If you don't immediately know how and where you'll hook up a new business partner, a second Internet link, etc., then it's time to re-think your network design. Everything should have a place, and then you should stick to your design unless there are extremely good reasons not to.

We're big fans of minimizing the firewall count, and putting them under common control. See above. If there are many firewalls, how do you know you've got all the external connections really covered? How consistent is the deployment of security to the firewalls? How consistent and robust and easy to troubleshoot is the routing and external connectivity?  If various application groups have been attaching their firewall sandwich designs to the network, and they're all different from each other, then you've got a problem. It's costly to support and troubleshoot all those different approaches. Was the network team involved in the design and planning?

We also like "firewall sandwiches". The idea is to have different firewalls on each side of every DMZ, so that no single switch or firewall compromise gets hackers to the inside. Yes this is costly. Smaller shops make do with fewer pieces of hardware, with lower costs and easier support. It's another one of those trade-offs, security versus $$.

Readable access lists. The PIX now supports class-oriented ACL's. CiscoWorks ACLM tool does too. You can make your long access lists much more readable and maintainable with such tools. Consider moving to them if you have not already done so!

Start thinking of security as a business practice . Security needs to be a process of continual improvement. There is too much to do to be able to just say "we've done our security for this year". Establish outside audits to validate that your are following industry best practices.

The following are some Cisco links we hope you'll find useful. Note that the ISP article has numerous good ideas for enterprises to follow too, especially larger enterprises.

Improving Security on Cisco Routers	http://www.cisco.com/en/US/tech/tk648/tk361/ technologies_tech_note09186a0080120f48.shtml
Virtual LAN Security Best Practices	http://www.cisco.com/en/US/products/hw/switches/ps708/ products_white_paper09186a008013159f.shtml
Catalyst 4500 Security Features Best Practices for Cisco IOS Software-Based Supervisors	http://www.cisco.com/en/US/products/hw/switches/ps4324/ products_white_paper09186a00801b1d13.shtml
Best Practices for Catalyst 6500/6000 Series and Catalyst 4500/4000 Series Switches Running Cisco IOS	http://www.cisco.com/en/US/products/hw/switches/ps700/ products_white_paper09186a00801b49a4.shtml
Essential IOS Features Every ISP Should Consider	http://www.cisco.com/public/cons/isp/documents/IOSEssentialsPDF.zip
Cisco Enterprise: Network Security	http://www.cisco.com/warp/customer/779/largeent/issues/security/
Cisco Enterprise: Network Security - Cisco SAFE	http://www.cisco.com/warp/customer/779/largeent/issues/security/safe.html

PIX

Even if you don't own a Cisco PIX, there's some good advice at http://www.cisco.com/univercd/cc/td/doc/product/iaabu/pix/pix_60/config/intro.htm.

Some of the sections from that document:

• Know Your Enemy
• Count the Cost
• Identify Your Assumptions
• Control Your Secrets
• Remember Human Factors
• Know Your Weaknesses
• Limit the cope of Access
• Understand Your Environment
• Limit Your Trust
• Remember Physical Security
• Make Security Pervasive

Some Interesting Security Web Sites

The following are some security web sites that you might find useful or interesting.

NSA Router Security Guide	http://nsa2.www.conxion.com/cisco/download.htm
CIS (see links for free system and router security tools)	Center for Internet Security
Nessus (ultimate free system security scanner)	http://www.nessus.org/
SANS Institute (security training, etc.)	http://www.sans.org/
CERT Coordination Center	http://www.cert.org/
CERT® Guide to System and Network Security Practices (book link and online HTML documents)	http://www.cert.org/security-improvement/
Counterpane	http://www.counterpane.com/
Counterpane: Crypto-Gram (Free Newsletter)	http://www.counterpane.com/crypto-gram.html
NFR Security Inc.	http://www.nfr.com/
TrustWave	http://www.trustwave.com/
Al Berg.Com	http://www.al-berg.com/
InfoSysSec (portal, articles, magazine list, etc.)	http://www.infosyssec.org/

Conclusion

To wrap things up, let's talk priorities, what to tackle first. The list is mostly network-related, but the application folks have to work with the network staff to determine how to best use the network to secure external access to applications and data.

Here's a high-level sequence of steps to take.

• Get firewall in place, permitting only what's really necessary. Specify only the necessary ports on a per-server basis.
• Start using the DMZ.
• Get other servers into the Data Center.
• Start building an application architecture plan. Minimize risks by allowing only a trusted web server or other very controlled server to be externally accessible.
• Work to cut "what's necessary" to a minimum, then watch for vulnerabilities in those applications.
• Start setting up some internal firewalls.
• Consider using IDS or HIPS.
• Consider performance and traffic (NetFlow) reporting.

If you're in a hurry, you might consider the new Cisco feature, autosecure. This secures your Cisco devices in a hurry. The potential issue is that it also turns off features needed for scalable management. Still, looking at what auto-secure does is a good way to examine your options as far as "locking down" your routers. Just make sure you and your management understand that beyond a point, such lockdown comes at a high cost increase due to more difficulty managing the equipment. For more about autosecure, see

http://www.cisco.com/en/US/products/sw/iosswrel/ps5187/products_feature_guide09186a008017d101.html

Even if you don't plan to use  the "auto secure" command, some of the commands and the anti-spoofing ACL's in that document may be quite useful to you. I'd consider some field testing, then using CiscoWorks RME and ACLM to push the parts you like out to your devices.

Please contact Chesapeake Netcraftsmen (http://www.netcraftsmen.net) if you'd like us to come do a security risk assessment, help with virus effects mitigation, help with network management or security management software (CiscoWorks!), etc.