Dynamic Content Caching using Lighty + mod_magnet + lua

*Updated 11 Jan 2007 to include luazlib, fix lighty config file and add zlib decoding to cache.lua for older browsers
Since I read the documentations for mod_cml, I was very excited to use this module since caching using PHP running as FastCGI is not helping much during server peak load. However, mod_cml was replaced by mod_magnet which is more flexible and gives more control over request handling in Lighttpd. This Article will focus on caching your PHP scripts using Lua and mod_magnet under Lighttpd, You should read “Compressing your HTML, CSS and Javascript using simple PHP Code” as I will use some functions from there

Installing Lua 5.1.2 + md5 lib

tar xfz lua-5.1.2.tar.gz
cd lua-5.1.2
yum install readline-devel
make linux install
cd ..
tar xfz md5-1.0.2.tar.gz
cd md5-1.0.2
make install
cd ..
#if only you have to unrar on linux
#get your compatible unrar binary from
wget wget
gunzip unrar-3.7.7-centos.gz
chmod +x unrar-3.7.7-centos

./unrar-3.7.7-centos x luazlib-0.0.1.rar
cd luazlib-0.0.1
make install

Installing Lighttpd 1.4.18 with mod_magnet support

tar xfj lighttpd-1.4.18.tar.bz2
cd lighttpd-1.4.18
export LUA_CFLAGS="-I/usr/local/include" 
export LUA_LIBS="-L/usr/local/lib -llua"
./configure --with-lua
make install

Enable mod_magnet on your lighttpd.conf

server.modules += ( "mod_magnet" )
#use cache.lua script only on php script from domain
$HTTP["host"] == "" {
	$HTTP["url"] =~ "(^/(\?.*)?$|\.php)" {
		magnet.attract-physical-path-to = ( "/myserver/cache.lua" )

Now every time a request to .php file will be handled by cache.lua script to see if we already have a cached copy of that script.
Here is cache.lua


--is request empty
if ( lighty.env["request.uri"] == nil ) then 
	lighty.env["request.uri"] = "/"

local file = "/server/cache/" .. md5.sumhexa( lighty.env["request.uri"] )

if lighty.stat(file) then
	--is encoding empty
	if ( lighty.request["accept-encoding"] == nil ) then 
		 lighty.request["accept-encoding"] = ""
	if( string.find( lighty.request["accept-encoding"] , "gzip") ) then
		lighty.content = { { filename = file } }
		lighty.header["Content-Encoding"] = "gzip"
		local ht =, "r")
		local t = ht:read("*a")
		lighty.content = { zlib.gzuncompress(t) .. ""}
	lighty.header["Content-Type"] = "text/html"
	return 200

Now a simple “hello world” page with cache functions

function html_compress($html){
#for compression code read previous post 

		if($GLOBALS['cachethis'] == 1){
				$filename = '/myserver/cache/'.md5($_SERVER['REQUEST_URI']);
				$gz = gzopen($filename, "w9");
				gzwrite($gz, $html);
				#_echo('No Cache');
		}else #you can add special cookie or keyword that you generate during the script and store in memory as alternative cache
	return $html;

#enable caching for this script
$cachethis = 1;

#alternative cache in memory using APC
if($cachethis == 2 && $html=apc_fetch($_SERVER['REQUEST_URI'].'2')){
	echo $html;

#add page content to $html just dont echo anything
$html = "";
	$html .= "Hello World<br>";

#displaying page content and caching it
echo html_compress($html);


Server info :

~: cat /proc/cpuinfo
vendor_id       : GenuineIntel
model name      : Intel(R) Celeron(R) CPU 2.66GHz
cpu MHz         : 2668.432
cache size      : 256 KB

~: cat /proc/meminfo
MemTotal:       239792 kB
MemFree:         12336 kB

let’s check the difference with ab

#benchmark using ab with compression enabled, as most of today browsers do have gzip
ab -n2000 -c400  -H 'Accept-Encoding: gzip,deflate'

#with $cachethis = 0; -- No Caching (no mod_magnet)
Requests per second:    99.55 [#/sec] (mean)
Time per request:       4018.271 [ms] (mean)
Time per request:       10.046 [ms] (mean, across all concurrent requests)

#with $cachethis = 2; -- Caching with APC (no mod_magnet)
Requests per second:    131.93 [#/sec] (mean)
Time per request:       3031.818 [ms] (mean)
Time per request:       7.580 [ms] (mean, across all concurrent requests)

#with $cachethis = 1; -- Caching with mod_magnet
Requests per second:    899.37 [#/sec] (mean)                                      <----------------------------|-
Time per request:       444.758 [ms] (mean)                                        <----------------------------|-
Time per request:       1.112 [ms] (mean, across all concurrent requests)          <----------------------------|-

#what about old browsers, will lua still be faster after decompressing data
ab -n2000 -c400

#with $cachethis = 0; -- No Caching (no mod_magnet)
Requests per second:    110.72 [#/sec] (mean)
Time per request:       5419.094 [ms] (mean)
Time per request:       9.032 [ms] (mean, across all concurrent requests)

#with $cachethis = 2; -- Caching with APC (no mod_magnet)
Requests per second:    156.24 [#/sec] (mean)
Time per request:       2560.163 [ms] (mean)
Time per request:       6.400 [ms] (mean, across all concurrent requests)

#with $cachethis = 1; -- Caching with mod_magnet
Requests per second:    112.84 [#/sec] (mean)
Time per request:       3544.811 [ms] (mean)
Time per request:       8.862 [ms] (mean, across all concurrent requests)

make sure to clear your cache folder before running ab.

I have to say I was amazed by the results myself, you're giving users static cached pages while you are enjoying your dynamic code, if your system is built with that on mind you should have no problem caching all your pages. Additionally, you can add cookies or browser language to the requested URI using Lua before searching for its Id in the cache. I would recommend using Javascript to read cookies instead or disable cache while users are logged-in.

Clearing cache
Since we did not check the mtime of cached files we should create a cron job to clear old files, let's say over 10 days old

0  7  */3  *  *  find /myserver/cache -type f -mtime +10 | xargs -r rm;

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.