Topics

cache

Varnish with secure AWS S3 bucket as backend

Serving static contents from S3 is common, but using Varnish in front is a bit tricky. Especially if you want to keep the bucket secure and only serve from Varnish, here is a simple Varnish file to solve this problem.

First secure your bucket via IP policy:

{
  "Version": "2012-10-17",
  "Id": "S3PolicyId1",
  "Statement": [
    {
      "Sid": "IPAllow",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::example.bucket/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": [
            "5.6.7.8/32"  //varnish ip
          ]
        }
      }
    },
    {
      "Sid": "Explicit deny to ensure requests are allowed only from specific referer.",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::example.bucket/*",
      "Condition": {
        "StringNotLike": {
          "aws:Referer": [
            "https://example.com/*"
          ]
        }
      }
    }
  ]
}

Setup postgreSQL RDS using Ansible

Setting up PostgreSQL on RDS using ansible is a bit tricky because the main user on RDS is not a SUPERUSER and roles membership is not automatically granted for ex: “ERROR: must be member of role ..” is quite common. Here is a working solution:

Cachita is a golang file and memory cache library

Cachita

Cachita is a golang file and memory cache library

Build Status
GoDoc

  • Simple caching with auto type assertion included.
  • In memory file cache index to avoid unneeded I/O.
  • Msgpack based binary serialization using msgpack library for file caching.

API docs: https://godoc.org/github.com/gadelkareem/cachita.

Examples: https://godoc.org/github.com/gadelkareem/cachita#pkg-examples.

Installation

Install:

go get -u github.com/gadelkareem/cachita

Fix W3 Total Cache W3_Plugin_TotalCache::ob_callback() expected to be a reference

Around a year ago I was playing with W3 Total Cache plugin on HHVM while I got an annoying warning

Warning: Parameter 1 to W3_Plugin_TotalCache::ob_callback() expected to be a reference, value given in /wp-includes/functions.php on line 3269

The funny part is that one of the functions was passing by reference which I still could not find the reason for it, maybe it was a limitation in PHP 4.3. Removing one character “&” fixed the issue so I submitted a pull request to the author although the repository currently does not accept pull request. Later on as PHP 7.0 was released, the issue started to show on PHP as well which brought more attention to this small fix. The users comments included anger such as user @kmob2 who said

this is becoming a running gag

Even more user @pratham2003 proposed a one line bash command to solve the problem

sed -i.bak 's/function ob_callback(&/function ob_callback(/g' /path/to/public_html/wp-content/plugins/w3-total-cache/lib/W3/Plugin/TotalCache.php

I hope that the developers will finally listen to the users and fix it soon!

Warming up WordPress cache, HHVM and testing blog pages

If you are minifying scripts and css files using a caching plugin or using FastCGI cache then you might need to warmup your blog after purging your cache. This is a simple warm up cli script for WordPress to initiate cache or HHVM HHBC and making sure all pages/posts do not have errors. Additionally the script creates a urllist.txt file that you can use with siege to test load your server.

#!/usr/bin/env php
< ?php

if (PHP_SAPI != 'cli') {
    header('Status: 403 Forbidden');
    header('HTTP/1.1 403 Forbidden', true, 403);
    exit();
}


$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
define('BASE_PATH', "");
define('WP_USE_THEMES', false);
global $wp, $wp_query, $wp_the_query, $wp_rewrite, $wp_did_header;
require(BASE_PATH . 'wp-load.php');

//Disable cache
ob_end_clean();

$homeUrl = home_url() . "/";
$urlList = [$homeUrl];


$the_query = new WP_Query(['post_type' => 'any', 'post_status' => 'publish']);
//add homepage pages
warmup_pages_links($the_query->max_num_pages, $homeUrl);

//get all posts
$the_query = new WP_Query(['post_type' => 'any', 'posts_per_page' => '-1', 'post_status' => 'publish']);
$the_query->the_post();
while ($the_query->have_posts()) {
    $urlList[] = get_permalink();
    $the_query->the_post();
}

$categories = get_categories();
foreach ($categories as $category) {
    $catUrl = get_category_link($category->term_id);
    $urlList[] = $catUrl;
    $the_query = new WP_Query(['cat' => $category->term_id]);
    warmup_pages_links($the_query->max_num_pages, $catUrl);
}

$tags = get_tags();
foreach ($tags as $tag) {
    $tagUrl = get_tag_link($tag->term_id);
    $urlList[] = $tagUrl;
    $the_query = new WP_Query(['tag_id' => $tag->term_id]);
    warmup_pages_links($the_query->max_num_pages, $tagUrl);
}

$authors = get_users(array('who' => 'authors'));
foreach ($authors as $author) {
    $authorUrl = get_author_posts_url(false, $author->user_nicename);
    $urlList[] = $authorUrl;
    $the_query = new WP_Query(['author_id' => $author->term_id]);
    warmup_pages_links($the_query->max_num_pages, $authorUrl);
}

//test search page
$urlList[] = get_search_link('bala la lam');

file_put_contents('urllist.txt', implode("\n", $urlList));

echo "Found " . count($urlList) . " URLS and saved to urlList.txt \n";
echo "Testing URLs \n";
foreach ($urlList as $url) {
    warmup_getUrl($url);
}
echo "\nWarmup done \n";

function warmup_getUrl($url)
{
    if ($contents = wp_remote_get($url, ['timeout' => 15])) {
        if (is_wp_error($contents) || (isset($contents['response']['code']) && $contents['response']['code'] != 200)) {
            $subject = "[WP " . get_bloginfo('name') . "] Error on {$url} ";
            $error = is_wp_error($contents) ? $contents->get_error_message() : print_r($contents, true);
            wp_mail(get_bloginfo('admin_email'), $subject, $error);
            die("\n{$subject} :" . $error);
        }
        echo ".";
    }
    return $contents;
}

function warmup_pages_links($total, $url)
{
    for ($i = 2; $i < = $total; $i++) {
        $GLOBALS['urlList'][] = "{$url}page/{$i}/";
    }
}

// siege -c200 -d1 -r30 -f urllist.txt