Why should I protect my dev and staging servers?

Most of the discussion about securing systems focuses on production systems; these are the critical machines we need for businesses to operate.  These are the machines where we run live applications on live data with live orders and real money changing hands.  But what about development, QA, and staging servers?  They’re generally private in that we don’t advertise their location; should we spend time securing these as well?

Why lock down the development machines?

These secondary boxes might seem to be optional at first glance.  If one of them went down, it wouldn’t be anything more than a momentary annoyance.  But there’s a problem with that viewpoint; system security is far more than just Availability.  We need to consider Confidentiality and Integrity too.

An attacker that gets access to a secondary system gets both information that might be used to attack primary systems and potentially the ability to take action that wouldn’t have been possible outside your network.

Secondary machines provide information

When an attacker gets access to a secondary system, no matter how insignificant that system might seem, they can quickly pull information off the box:

  • User account names
  • OS and application level password hashes
  • Login times and habits
  • Source IP addresses of most recent logins
  • Trust models between systems
  • System authentication type
  • Running programs and their open ports
  • Firewall and security settings
  • Operating system details including package versions
  • How and when you patch – or fail to patch – systems

Most of the above don’t even need an administrator-level login; just the ability to run commands.

Once the attacker has these, they can use this information to attack more critical machines.  If they see a vulnerable version of Adobe Acrobat reader on a secondary machine, it’s a good bet that the same version is installed on production systems.

Vendor-supplied packages aren’t the only source of information.  The attacker can view the code running your internal applications as well.  By either examining the source code (if placed on the system) or watching the actions taken by running programs, the attacker can get details about the program’s security model and sources of data.  Even in an environment where the secondary machines only access a dummy database filled with junk values, these details tell the attacker how the program will work on a production machine filled with real data.

The sad truth is that many secondary machines are not limited to dummy databases filled with junk data.  In a recent break-in, an attacker was able to pull customer information out of the database from a development machine; their development servers had direct access to production data.  It seems like an obvious mistake, but it’s more common than we’d like to admit.

Secondary machines allow attackers to take action

Worse yet, secondary machines are useful for more than just information gathering.

Once an attacker has acquired access to a secondary machine, they can use this system as a jumping off point to access other systems.  In addition to the database example above, final targets might include authentication services, log servers, internal mail relays, and private web services.  These might be reached over a network socket or over API calls – access methods that are likely blocked for the public Internet but available to these systems.

In a scenario where internal applications are developed on one or more development machines and the code is stored locally, an attacker that gets access to one of those machines may also get the ability to modify that code.  Ideally their changes would give them some special access without anyone knowing.  In an order-taking system that emails the final order, how about sending a BCC (blind carbon copy) to a mail account controlled by the attacker?  Perhaps we take customer’s entered passwords and ship them off to a remote system before they’re stored?  If the attacker is a competitor to your business, why not intercept 1 out of every 10 orders and direct them over to their fulfillment center instead?  The customer still gets their product, and everyone’s happy except the accounting crew that can’t figure out why their sales figures are down 10%.

“So what?  That code is only on a development machine,” you say.  Well, true – but where is that code going to end up?  On production machines.  With production orders.  And actual money changing hands.  Nobody would read the entire source code to an enterprise application before deploying it, so our only real hope of catching trojan code is to review the patches that are applied to make a new version before the new version goes out.  This ability to review changes and follow a formal approval process would allow us to identify the injected code.

Even if the attackers get no direct access to other systems or information that helps them plan future attacks, they still have access to a running system.  They can use your machine to attack others, send spam, or even just use your computer as a hiding space for file downloads.

And you pay the bandwidth bill.

Related Posts