What is the 'Heartbleed' bug?
The Heartbleed Bug is a serious vulnerability in the popular OpenSSL cryptographic software library. This weakness allows stealing the information protected, under normal conditions, by the SSL/TLS encryption used to secure the Internet. SSL/TLS provides communication security and privacy over the Internet for applications such as web, email, instant messaging (IM) and some virtual private networks (VPNs).
The Heartbleed bug allows anyone on the Internet to read the memory of the systems protected by the vulnerable versions of the OpenSSL software. This compromises the secret keys used to identify the service providers and to encrypt the traffic, the names and passwords of the users and the actual content. This allows attackers to eavesdrop on communications, steal data directly from the services and users and to impersonate services and users.
What does this mean for web sites in general?
The nature of the Heartbleed Bug and the fact that is has been afflicting web servers since 2011 means that in the absolute worst case scenario, all data on afflicted sites could have been accessed by unauthorized persons. This would be accomplished by using the Heartbleed Bug to retrieve the private key from an afflicted web server, capturing the HTTPS traffic, using the captured private key to decrypt the captured HTTPS traffic and then either using any contained passwords to gain unauthorized access to the site or using the data itself elsewhere (ie. credit card data). These properties of the Heartbleed Bug mean that passwords and other data could have been captured without touching affected servers. As a result, data could have been stolen without leaving a trace. In theory at least, every bit of data sent or received since the Heartbleed Bug first appeared could have been captured.
In practice, we have to consider that outside of a few government agencies it would be impractical to capture data on such a scale. The data then needs to be analyzed to separate the valuable bits of information from the junk. What this means is that the easily identifiable, high value data has been and will be exploited as soon as it is identified. It also means that there may be more captured data that will take longer to analyze and that exploitation of that data may continue for some time to come.
For those of us operating servers, all we can do is patch any affected servers, force users to reset passwords and to continue keeping an eye on things for suspicious activity. The bigger problem is for all of us as users. Once vulnerable services are updated to eliminate the Heartbleed Bug, we as users need to assume our passwords were exposed and change them. If you're unsure of whether a service is affected by the Heartbleed Bug, or curious about their overall SSL security, you can generate an SSL Report at Qualys' SSL Labs.
How was ScreenLight affected?
ScreenLight is hosted on Amazon's cloud and uses their Elastic Load Balancer (ELB) service to terminate HTTPS connections from our users' browsers. Since ELB was using a vulnerable version of OpenSSL, we were vulnerable to the Heartbleed Bug.
We also use Amazon's Cloudfront CDN to deliver uploaded assets over HTTPS to reviewers. Cloudfront was also vulnerable to the Heartbleed Bug.
Once Amazon patched their servers, we still had some work to do to protect our users data.
We patched our servers
While ScreenLight is only accessible via Amazon's load balancers, we still encrypt the traffic between the load balancers and our application servers. While this traffic is not exposed to the public internet, we encrypt it to keep it secure. The web server front ends of our application servers were using an affected version of OpenSSL and were patched.
We replaced SSL/TLS certificates and revoke old ones
With all our web servers patched against the Heartbleed Bug, we could replace our possible exposed SSL/TLS certificates. While we did this, we also increased the size of our private keys from 2048-bits to 4096-bits.
We forced all users to reset their passwords
User passwords are the most important data to be kept secret as they are long lived. Compound this by the fact that some user accounts may be dormant and the certain incomplete compliance with a mere request for users to reset their password and it's clear that an honest effort at protecting user data requires forcing users to set a new password before they can log in again. This approach also prevents continued access to compromised dormant accounts.
We removed all password protected links
Similar to user passwords, but on a smaller scale are password protected sharing links. Eliminating the risk of unauthorized access using passwords captured using the Heartbleed Bug meant removing these links. Our thinking being that if these shared assets are important enough to be protected by a password, then we need to do whatever we can to prevent unauthorized access.
We reviewed our SSL/TLS implementation
We review the details of our SSL/TLS implementation each year when we renew our certificates. While our security practices prior to the Heartbleed Bug being revealed have followed best practices, we are taking a much more critical, some would say paranoid, approach to our security practices.
Where we are today
The results of the SSL Report at Qualys' SSL Labs for ScreenLight are great.
Network traffic between modern browsers and our servers will use 256-bit TLS 1.0+ with Perfect Forward Secrecy. The last bit is really important for stopping bugs like the Heartbleed Bug from being so significant. Briefly, without Perfect Forward Secrecy, getting a web server's private key via a bug like the Heartbleed Bug allows one to decrypt any HTTPS traffic that was encrypted using that key. By using Perfect Forward Secrecy, it's no longer the web server's private key that is used to secure the HTTPS traffic, so capturing the private key doesn't give an attacker access to all HTTPS traffic in and out of the affected server.
There is however still one blemish on this nearly perfect picture, and that is Internet Explorer 8 when running on Windows XP. The good news is that as of April 8, 2014, Microsoft has finally dropped support for Windows XP. The bad news is that some large corporations are still running Windows XP and only allow their employees to use Internet Explorer 8. What makes this a problem is that Internet Explorer 8 when running on Windows XP only supports 128-bit TLS 1.0 and it doesn't support Perfect Forward Secrecy. What makes this peculiar is that Internet Explorer 8 on Windows 7 supports 256-bit TLS 1.0 with Perfect Forward Secrecy, as do Chrome and Firefox when running on Windows XP. Unfortunately in some corporate environments, it is just not possible to be upgraded to one of the more secure configurations.
We're hoping that by the end of the year Internet Explorer 8 on Windows XP usage will be low enough that we can remove support for 128-bit encryption allowing us to encrypt all of our HTTPS traffic with 256-bit encryption with Perfect Forward Secrecy.