Axel's root Blog

for nerds only - little stories from the everyday sysadmin life with problems and their hard-to-find solutions

fighting against slowloris attack

2010-07-22 by Axel Reinhold, tagged as apache, network, security
Last week my webserver was hit by a "slowloris" attack. slowloris is a simple and effective low-bandwidth Deny-Of-Service-attack which brings almost every apache to its knees.

This DoS-attack can easily bring down any typical webserver by consuming all threads/processes waiting just for more data. Smaller timeout settings can help but cannot really make the system immune by this or similar attacks. Firewalling tricks to limit the connections from one IP are also helpful, but also lock out customers sitting behind natting routers like big companies or from mobile devices.

What really helps in this situation is to setup a reverse-proxy before the webserver which can consume all the slow connects. The proxy hits the webserver only with valid complete requests. In this way the proxy is taking the load from slowloris - the webserver stays healthy serving valid requests.

We found Willy Tarreau's HAProxy ideal for this approach, because it's small and has only the feature we need, others like nginx or lighttpd are fullblown webservers. This small config is enough to set up haproxy as reverse proxy:

global
  daemon
  maxconn 1000

defaults
  mode http
  maxconn 960
  timeout client 120000ms
  timeout server 120000ms
  timeout queue 60000ms
  timeout connect 8000ms
  timeout http-request 6000ms
  option httpclose
  option abortonclose
  option forwardfor
  retries 2

listen http-in
  bind *:80
  server server1 127.0.0.1:8000 maxconn 400

The only thing that has to be changed on the apache side is the Port 8000 directive to make apache listen to port 8000 instead of 80 because now the reverse-proxy listens to port 80 and connects to apache on port 8000.

This setup breaks all source-ip-based functions in apache, because the source-ip of all requests is now the reverse-proxy! So expecially logging and htaccess are candidates to break. The small apache module mod_rpaf helps. It just takes the ip from the X-Forwarded-For-header written by the proxy and replaces the source-ip within apache. All is then back to as it was before with these in httpd.conf:

LoadModule rpaf_module libexec/apache2/mod_rpaf-2.0.so

RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1
RPAFheader X-Forwarded-For

Be sure to load this module last because then it is processed before all other modules and makes them work with the correct source-ip.

Fortunately we could not measure any performance degration using the HAProxy reverse-proxy.