I was working with a client implementing a vulnerability scanning program. We were analyzing some results when I noticed a few systems vulnerable to Heartbleed. This was a surprise, since it is a Windows shop, although the scan showed a lot of Tomcat around (presumably vendor systems) OpenSSL is not used by Java either. We ran it down and it turned out to be the server management GUI for a couple of machines. This reminded me that there was a fair amount of embedded code, management GUIs for servers, router firmware, etc., that could be vulnerable.
What to do? My friend Steve Bellovin would say the most important thing is "Don't Panic." I concur. Also, don't aggressively scan for it if you have older servers on your network. HP's note on Heartbleed and embedded code notes:
IMPORTANT: Reports have been received that scanners used to identify the Heartbleed vulnerability cause first-generation Integrated Lights-Out (iLO) and Integrated Lights-Out 2 (iLO 2) to lockup and become unresponsive. Although the server's operating system will continue to function normally, first-generation iLO and iLO 2 will no longer be responsive over the management network. To recover, power must be PHYSICALLY removed from the server. HP recommends not using vulnerability scanners to test first-generation iLO and iLO 2 devices, as these products are not vulnerable to the Heartbleed vulnerability.
This means that you should aim your Heartbleed tests and known URLs, not every thing a port scan turns up as running HTTPS.
Instead, you should assess your vulnerability by checking your inventory of servers, routers, switches, firewalls, etc for firmware version numbers and determine which have vulnerable versions. Check with the vendors, and figure out the highest risk stuff to be upgraded. This is a long-term campaign to be planned out and executed with deliberation not haste. Not every asset will be a high priority for upgrade.
On the other hand, exposed services are a high priority. If you have something that's exposed to the Internet, protect the vulnerable services with firewall rules. If the vulnerable thing is the firewall, make triple-sure that the vulnerable service is only accessible from internal networks. Hopefully you don't expose the management GUI on any device to the Internet, but check to make sure.
This is just one vulnerability. There are probably many more lurking in embedded code, since firmware is hard to patch and the patches may be hard to schedule.
But I'll still stress Steve's advice: don't panic. Just plan and execute your campaign.
About Andy Sherman
Andy Sherman, Eden Technologies’ security practice lead has a PhD in physics from Rensselaer Polytechnic Institute and started his career in the academic world. He then went to AT&T Bell Laboratories where he discovered the power – and hazards – of large distributed computer networks. It was also at Bell Labs, during the early days of the Internet, that Andy became interested in the security problems associated with public networks. From Bell Labs Andy moved to the financial services industry. There he worked on a large range of infrastructure design, deployment, and management projects, but is best known for his 15+ years in information and technology security.