Limit requests per IP on Nginx using HttpLimitZoneModule and HttpLimitReqModule except whitelist

– Make sure to check Nginx, PHP posts for information on Nginx and PHP setup and configuration.

Nginx offers two modules, HttpLimitReqModule and HttpLimitZoneModule, to simultaneous connections for the assigned session and the number of requests for a given session from one IP address. Basically these modules are built to protect the web server from possible attacks; For example, this configuration limits remote clients to no more than 20 concurrently “open” connections per remote ip address:

    limit_conn_  $binary_remote_addr =concurrent:10m;
    limit_conn_log_level warn;
    limit_conn  concurrent  20;

Additionally, this configuration zone “allips” allocating 5MB for this session then limits queries for remote ip address to 200 request per second

    limit_req_zone $binary_remote_addr zone=allips:5m rate=200r/s;
    limit_req zone=allips burst=200;

As a result, additional requests or connections from the same remote ip address to Nginx will be dropped with 503 (Service Temporarily Unavailable) error. But what if we want to create a whitelist of IPs; for instance, Search bots should not obey these limits. That can be accomplished by HttpGeoModule.

Firstly, a list of search bots CIDR is added to $limited variable, so that any IP that is not on list will return “1” else it returns “0”, as following:
– The example contains real CIDR list of search bot.

     geo $limited {
        default 1;
        #google 0; 0; 0; 0; 0; 0; 0; 0;
        #M$ 0; 0; 0; 0; 0; 0; 0;
        #yahoo 0; 0; 0; 0; 0; 0; 0; 0;
        #My IPs 0;
        123.456.0.0/28 0; #example for your server CIDR

    map $limited $limit {
        1        $binary_remote_addr;
        0        "";

Secondly, $limit variable will be added to limit_req instead of $binary_remote_addr to limit requests from other IPs that are not on the whilelist:

    limit_req_zone $limit zone=notabot:5m rate=200r/s;
    limit_req zone=notabot burst=200;

Finally, test these configuration with ab against PHP-FPM:
– example :

for( $i=0; $i < 1000; $i++)
	echo 'Hello World';

– It is important to check the IP you are connecting with to the server before you .

#benchmark  ab with compression enabled
ab -n2000 -c400  -H 'Accept-Encoding: gzip,deflate'

# with server CIDR added to whitelist
Concurrency Level:      400
Time taken for tests:   0.426980 seconds
Complete requests:      2000
Failed requests:        275             <-------------------------
   (Connect: 0, Length: 275, Exceptions: 0)
Requests per second:    4684.06 [#/sec] (mean)
Time per request:       85.396 [ms] (mean)
Time per request:       0.213 [ms] (mean, across all concurrent requests)

# after commenting server CIDR from the whitelist
Concurrency Level:      400
Time taken for tests:   0.665429 seconds
Complete requests:      2000
Failed requests:        1807             <-------------------------
   (Connect: 0, Length: 1807, Exceptions: 0)
Write errors:           0
Requests per second:    3005.58 [#/sec] (mean)
Time per request:       133.086 [ms] (mean)
Time per request:       0.333 [ms] (mean, across all concurrent requests)

In conclusion, requests or connections from IPs on the whitelist will have no limits, whereas IPs outside the whitelist will be limited to a number of connections and requests, consequently, they will be dropped or fail as showen in the above ab example.

  • Jonathan Matthews

    When you say “zone ‘notabot’ is added to limit_req”, the next config example doesn’t mention “notabot”.
    Is this a mistake?

  • Mb

    Failed requests:  275  even thought the IP is on the whitelist ? How is that correct ?

  • certabiz

    Hi, this doesn’t work for me. The limit_req_zone indiscriminately limits requests for everything, even though I set the zone to the name of the geo.

    • try to include your IP range to notabot list as mentioned and test again. Also make sure your have Nginx 1.0.12 or later version.

  • Anony

    The CIDR number you give for Google include 0 which is a microsoft range and not google

  • Afdah

    instead of limit requests, i tried to limit connection using the above method. doesn’t work. any ideas?

  • SplitIce

    Do you know if this works with other methods of setting variables or just geo? I wasnt able to find the C code that implements this feature in the source.

  • Pingback: 用HttpLimitReqModule建立白名单限制Nginx访问频率 | sw32技术博客()