*Updated 11 Jan 2007 to include luazlib, fix lighty config file and add zlib decoding to cache.lua for older browsers
Since I read the documentations for mod_cml, I was very excited to use this module since caching using PHP running as FastCGI is not helping much during server peak load. However, mod_cml was replaced by mod_magnet which is more flexible and gives more control over request handling in Lighttpd. This Article will focus on caching your PHP scripts using Lua and mod_magnet under Lighttpd, You should read “Compressing your HTML, CSS and Javascript using simple PHP Code” as I will use some functions from there
Installing Lua 5.1.2 + md5 lib
wget http://www.lua.org/ftp/lua-5.1.2.tar.gz
tar xfz lua-5.1.2.tar.gz
cd lua-5.1.2
yum install readline-devel
make linux install
cd ..
wget http://luaforge.net/frs/download.php/2384/md5-1.0.2.tar.gz
tar xfz md5-1.0.2.tar.gz
cd md5-1.0.2
make
make install
cd ..
wget http://luaforge.net/frs/download.php/1678/luazlib-0.0.1.rar
#if only you have to unrar on linux
#get your compatible unrar binary from
#http://www.rarlab.com/rar_add.htm
wget wget http://www.rarlab.com/rar/unrar-3.7.7-centos.gz
gunzip unrar-3.7.7-centos.gz
chmod +x unrar-3.7.7-centos
./unrar-3.7.7-centos x luazlib-0.0.1.rar
cd luazlib-0.0.1
make
make install
Installing Lighttpd 1.4.18 with mod_magnet support
wget http://www.lighttpd.net/download/lighttpd-1.4.18.tar.bz2
tar xfj lighttpd-1.4.18.tar.bz2
cd lighttpd-1.4.18
export LUA_CFLAGS="-I/usr/local/include"
export LUA_LIBS="-L/usr/local/lib -llua"
./configure --with-lua
make
make install
Enable mod_magnet on your lighttpd.conf
server.modules += ( "mod_magnet" )
#use cache.lua script only on php script from domain gadelkareem.com
$HTTP["host"] == "gadelkareem.com" {
$HTTP["url"] =~ "(^/(\?.*)?$|\.php)" {
magnet.attract-physical-path-to = ( "/myserver/cache.lua" )
}
}
Now every time a request to .php file will be handled by cache.lua script to see if we already have a cached copy of that script.
Here is cache.lua
require"md5"
--is request empty
if ( lighty.env["request.uri"] == nil ) then
lighty.env["request.uri"] = "/"
end
local file = "/server/cache/" .. md5.sumhexa( lighty.env["request.uri"] )
if lighty.stat(file) then
--is encoding empty
if ( lighty.request["accept-encoding"] == nil ) then
lighty.request["accept-encoding"] = ""
end
if( string.find( lighty.request["accept-encoding"] , "gzip") ) then
lighty.content = { { filename = file } }
lighty.header["Content-Encoding"] = "gzip"
else
require"zlib"
local ht = io.open(file, "r")
local t = ht:read("*a")
ht:close()
lighty.content = { zlib.gzuncompress(t) .. ""}
end
lighty.header["Content-Type"] = "text/html"
return 200
end
Now a simple “hello world” page with cache functions
<?php
function html_compress($html){
#for compression code read previous post
#http://gadelkareem.com/2007/06/23/compressing-your-html-css-and-javascript-using-simple-php-code/
#caching
if(!empty($GLOBALS['cachethis'])){
if($GLOBALS['cachethis'] == 1){
$filename = '/myserver/cache/'.md5($_SERVER['REQUEST_URI']);
$gz = gzopen($filename, "w9");
gzwrite($gz, $html);
gzclose($gz);
#_echo('No Cache');
}else #you can add special cookie or keyword that you generate during the script and store in memory as alternative cache
apc_store($_SERVER['REQUEST_URI'].$GLOBALS['cachethis'],$html,86400);
}
return $html;
}
#enable caching for this script
$cachethis = 1;
#alternative cache in memory using APC
if($cachethis == 2 && $html=apc_fetch($_SERVER['REQUEST_URI'].'2')){
echo $html;
exit;
}
#add page content to $html just dont echo anything
$html = "";
for($i=0;$i<10000;$i++)
$html .= "Hello World<br>";
#displaying page content and caching it
echo html_compress($html);
?>
Server info :
~: cat /proc/cpuinfo
vendor_id : GenuineIntel
model name : Intel(R) Celeron(R) CPU 2.66GHz
cpu MHz : 2668.432
cache size : 256 KB
~: cat /proc/meminfo
MemTotal: 239792 kB
MemFree: 12336 kB
let’s check the difference with ab
#benchmark using ab with compression enabled, as most of today browsers do have gzip
ab -n2000 -c400 -H 'Accept-Encoding: gzip,deflate' http://gadelkareem.com/helloworld.php
#with $cachethis = 0; -- No Caching (no mod_magnet)
Requests per second: 99.55 [#/sec] (mean)
Time per request: 4018.271 [ms] (mean)
Time per request: 10.046 [ms] (mean, across all concurrent requests)
#with $cachethis = 2; -- Caching with APC (no mod_magnet)
Requests per second: 131.93 [#/sec] (mean)
Time per request: 3031.818 [ms] (mean)
Time per request: 7.580 [ms] (mean, across all concurrent requests)
#with $cachethis = 1; -- Caching with mod_magnet
Requests per second: 899.37 [#/sec] (mean) <----------------------------|-
Time per request: 444.758 [ms] (mean) <----------------------------|-
Time per request: 1.112 [ms] (mean, across all concurrent requests) <----------------------------|-
#what about old browsers, will lua still be faster after decompressing data
ab -n2000 -c400 http://gadelkareem.com/helloworld.php
#with $cachethis = 0; -- No Caching (no mod_magnet)
Requests per second: 110.72 [#/sec] (mean)
Time per request: 5419.094 [ms] (mean)
Time per request: 9.032 [ms] (mean, across all concurrent requests)
#with $cachethis = 2; -- Caching with APC (no mod_magnet)
Requests per second: 156.24 [#/sec] (mean)
Time per request: 2560.163 [ms] (mean)
Time per request: 6.400 [ms] (mean, across all concurrent requests)
#with $cachethis = 1; -- Caching with mod_magnet
Requests per second: 112.84 [#/sec] (mean)
Time per request: 3544.811 [ms] (mean)
Time per request: 8.862 [ms] (mean, across all concurrent requests)
make sure to clear your cache folder before running ab.
I have to say I was amazed by the results myself, you're giving users static cached pages while you are enjoying your dynamic code, if your system is built with that on mind you should have no problem caching all your pages. Additionally, you can add cookies or browser language to the requested URI using Lua before searching for its Id in the cache. I would recommend using Javascript to read cookies instead or disable cache while users are logged-in.
Clearing cache
Since we did not check the mtime of cached files we should create a cron job to clear old files, let's say over 10 days old
0 7 */3 * * find /myserver/cache -type f -mtime +10 | xargs -r rm;