Docs For Class PHPCrawlerSQLiteURLCache

Description

Class for caching/storing URLs/links in a SQLite-database-file.

Located in /libs/PHPCrawler/UrlCache/PHPCrawlerSQLiteURLCache.class.php (line 8)

PHPCrawlerURLCacheBase
   |
   --PHPCrawlerSQLiteURLCache

Variable Summary

mixed $db_analyzed

PDO $PDO

PDOStatement $PreparedInsertStatement

mixed $sqlite_db_file

Method Summary

PHPCrawlerSQLiteURLCache __construct (string $file, [bool $create_tables = false])

void addURL (PHPCrawlerURLDescriptor $UrlDescriptor)

void addURLs (array $urls)

void cleanup ()

void clear ()

bool containsURLs ()

void createPreparedInsertStatement ()

void getAllURLs ()

PhpCrawlerURLDescriptor getNextUrl ()

void getUrlCount ()

void markUrlAsFollowed (PHPCrawlerURLDescriptor $UrlDescriptor)

void openConnection ([bool $create_tables = false])

void purgeCache ()

Variables

mixed $db_analyzed = false (line 26)

access: protected

PDO $PDO (line 15)

PDO-object for querying SQLite-file.

access: protected

PDOStatement $PreparedInsertStatement (line 22)

Prepared statement for inserting URLS into the db-file as PDOStatement-object.

access: protected

mixed $sqlite_db_file (line 24)

access: protected

Inherited Variables

Inherited from PHPCrawlerURLCacheBase

PHPCrawlerURLCacheBase::$url_distinct_property
PHPCrawlerURLCacheBase::$url_priorities

Methods

Constructor __construct (line 34)

Initiates an SQLite-URL-cache.

access: public

PHPCrawlerSQLiteURLCache __construct (string $file, [bool $create_tables = false])

string $file: The SQLite-fiel to use.
bool $create_tables: Defines whether all necessary tables should be created

addURL (line 106)

Adds an URL to the url-cache

access: public

void addURL (PHPCrawlerURLDescriptor $UrlDescriptor)

PHPCrawlerURLDescriptor $UrlDescriptor

Redefinition of:

PHPCrawlerURLCacheBase::addURL(): Adds an URL to the url-cache

addURLs (line 134)

Adds an bunch of URLs to the url-cache

access: public

void addURLs (array $urls)

array $urls: A numeric array containing the URLs as PHPCrawlerURLDescriptor-objects

Redefinition of:

PHPCrawlerURLCacheBase::addURLs(): Adds an bunch of URLs to the url-cache

cleanup (line 281)

Cleans up the cache after is it not needed anymore.

access: public

void cleanup ()

Redefinition of:

PHPCrawlerURLCacheBase::cleanup(): Do cleanups after the cache is not needed anymore

clear (line 95)

Removes all URLs and all priority-rules from the URL-cache.

access: public

void clear ()

Redefinition of:

PHPCrawlerURLCacheBase::clear(): Removes all URLs and all priority-rules from the URL-cache.

containsURLs (line 179)

Checks whether there are URLs left in the cache that should be processed or not.

access: public

bool containsURLs ()

Redefinition of:

PHPCrawlerURLCacheBase::containsURLs(): Checks whether there are URLs left in the cache or not.

createPreparedInsertStatement (line 261)

Creates the prepared statement for insterting URLs into database (if not done yet)

access: protected

void createPreparedInsertStatement ()

getAllURLs (line 88)

Has no function in this class

access: public

void getAllURLs ()

Redefinition of:

PHPCrawlerURLCacheBase::getAllURLs(): Returns all URLs currently cached in the URL-cache.

getNextUrl (line 53)

Returns the next URL from the cache that should be crawled.

return: An PhpCrawlerURLDescriptor or NULL if currently no URL to process.
access: public

PhpCrawlerURLDescriptor getNextUrl ()

Redefinition of:

PHPCrawlerURLCacheBase::getNextUrl(): Returns the next URL from the cache that should be crawled.

getUrlCount (line 40)

access: public

void getUrlCount ()

markUrlAsFollowed (line 166)

Marks the given URL in the cache as "followed"

access: public

void markUrlAsFollowed (PHPCrawlerURLDescriptor $UrlDescriptor)

PHPCrawlerURLDescriptor $UrlDescriptor

Redefinition of:

PHPCrawlerURLCacheBase::markUrlAsFollowed(): Marks the given URL in the cache as "followed"

openConnection (line 212)

Creates the sqlite-db-file and opens connection to it.

access: protected

void openConnection ([bool $create_tables = false])

bool $create_tables: Defines whether all necessary tables should be created

purgeCache (line 201)

Cleans/purges the URL-cache from inconsistent entries.

access: public

void purgeCache ()

Redefinition of:

PHPCrawlerURLCacheBase::purgeCache(): Cleans/purges the URL-cache from inconsistent entries.

Inherited Methods

Inherited From PHPCrawlerURLCacheBase

PHPCrawlerURLCacheBase::addLinkPriorities()
PHPCrawlerURLCacheBase::addLinkPriority()
PHPCrawlerURLCacheBase::addURL()
PHPCrawlerURLCacheBase::addURLs()
PHPCrawlerURLCacheBase::cleanup()
PHPCrawlerURLCacheBase::clear()
PHPCrawlerURLCacheBase::containsURLs()
PHPCrawlerURLCacheBase::getAllURLs()
PHPCrawlerURLCacheBase::getDistinctURLHash()
PHPCrawlerURLCacheBase::getNextUrl()
PHPCrawlerURLCacheBase::getUrlPriority()
PHPCrawlerURLCacheBase::markUrlAsFollowed()
PHPCrawlerURLCacheBase::purgeCache()

Class Constants

Inherited Constants

Inherited from PHPCrawlerURLCacheBase

PHPCrawlerURLCacheBase::URLHASH_NONE
PHPCrawlerURLCacheBase::URLHASH_RAWLINK
PHPCrawlerURLCacheBase::URLHASH_URL