Before I continue, I should mention that, not only is it impolite, but quite possibly illegal to take the content of a third party site without the owners permission or knowledge.

To accomplish our task we need some open-source tools:

• Zend Framework (Required)
• Quix (Recommended)
• Dojo Toolkit (Recommended)

Our basic strategy is this:

1. Find a target page with some cool content
2. Find the CSS Selector (or X-path) of the element we want to scrape
3. Verify the selector using Dojo
4. Build a Zend_Service object to fetch the cool content to PHP

Step 1: Finding the cool page

Say I am a developer and I really want to get the outward facing IP of the machine that my PHP script is running on. If there is a router or proxy in between my machine and the Internet, this can be non trivial. As a solution I decide to use the service http://www.whatismyip.com/

Step 2: Finding the CSS3 Selector for the element

Its pretty obvious which element we want to get the selector for. If you are feeling clever you can simply read the code or use a DOM inspector to figure out what the query is on your own. However, selectorgaget.com provides a cool tool that will allow you to point and click to determine the appropraite selector. You can get a bookmarklet directly from their site, or you can get the much more powerful Quix bookmarklet which includes the selectorgaget and a bunch of other cool tools.

If you decided on Quix (I know I did), click on the bookmarklet and enter “sg” in the command box. The selectorgaget should load up and as you move your mouse around the screen it should highlight different DOM nodes. We want the top one, so click on the text “Your IP…” and, checking out the path, we see that it is a rather boring h1.

Step 3: Validation via Dojo

I like to validate the path before taking it over to Zend. I use a Dojo and firebug lite bookmarklet which injects a debugging version of dojo into any page via the AOL CDN. To add this bookmarket drag Inject Dojo into your bookmarks toolbar. In the debugging console that pops up, enter dojo.query(‘h1′); and you should see the h1 DOM element being returned.

Step 4: Moving it all to PHP

Now that we have successfully found our CSS3 selector path, we can move over to PHP and come up with a new Zend_Service component. Our approach will be to extend Zend_Service_Abstract, and implement some custom methods to preform the screen scrape.

The finished class looks like this:

class WhatsMyIpService extends Zend_Service_Abstract
{
	/**
	 * The service endpoint.
	 * This is where Zend_Http_Client will navigate
	 * to fill service requests
	 * @var string
	 */
	protected $_endpoint = 'http://whatismyip.com';
 
	/**
	 * handle to the client
	 * @var Zend_Http_Client
	 */
	protected $_client;
 
	public function __construct()
	{
		$this->_client = self::getHttpClient();
	}
 
	/**
	 * Method to get the external IP of the computer / server
	 * script is running on
	 * @return string
	 */
	public function getMyIp()
	{
		//reset the client parameters, set the URL to whatismyip.com
		//and actually preform the request
		$result = $this->_client
			->resetParameters()
			->setUri($this->_endpoint)
			->request(Zend_Http_Client::GET);
 
		//check to make sure that the result isnt a HTTP error
		if($result->isError()){
			throw new Exception('Client returned error: ' . $result->getMessage());
		}
 
		try{
			//setup the query object with the result body (HTML page)
			$query = new Zend_Dom_Query($result->getBody());
 
			$domCollection = $query->query('h1');
		}catch(Zend_Dom_Exception $e){
			throw new Exception('Error Loading Document: ' . $e);
		}
 
		//check to make sure the query return a result
		if($domCollection->count() == 0){
			throw new Exception('Cannot find DOM Element');
		}
 
		//get the titlestring from the nodevalue
		$titleString = (string) $domCollection->current()->nodeValue;
 
		//now we should have the content of h1 stored in the titleString
		//it should read something like "Your IP Address Is: xxx.xxx.xxx.xxx"
 
		//Now we will parse out the IP address using regular expressions
		if (preg_match('/([\d]{1,3}\.[\d]{1,3}\.[\d]{1,3}\.[\d]{1,3})/', $titleString, $matches)) {
			return $matches[1];
		}else{
			throw new Exception('Unable to parse IP from page');
		}
	}
}

I hope that the script is sufficiently commented, but to summarize:

In the constructor, we get an local instance of a Zend_Http_Client object. Clone() is used to prevent any other Zend_Service objects from polluting our client with their own requests.

In the getMyIp() method, we first setup and preform the request (using the fluid interface provided by Zend_Http_Client). Notice how we reset the parameters: In this case its not actually necessary as we aren’t passing any parameters, but it is good habit to get into in case in the future we expand this class to include GET or POST parameter passing.

Next, we examine what the HTTP client has passed back to us. Hopefully, if nothing has gone wrong, it is a HTML string representation of the page at whatismyip.com. Some basic checks are preformed to ensure that no HTTP errors have occurred, and then we instantiate a Zend_Dom_Query object which provides both CSS and Xpath selectors.

Finally, after running the CSS selector query, we check to ensure we got a DOM element back, get its value and parse out the IP using Regular expressions. Its pretty impressive what can be done in <70 lines of code with the Zend Framework.

To run this class, we could create a directory structure as follows:
library <- Includes Zend framework
WhatsMyIpService.php <- The above class
getIp.php <- php file including the following:

set_include_path(implode(PATH_SEPARATOR, array(
    realpath(dirname(__FILE__) . '/library'),
    get_include_path(),
)));
 
/** Zend_Loader_Autoload **/
require_once 'Zend/Loader/Autoloader.php';
Zend_Loader_Autoloader::getInstance()->setFallbackAutoloader(true);
 
$service = new WhatsMyIpService();
echo $service->getMyIp();

The first couple of lines gets the library on the include path, the second block sets up autoloading so we don’t have to manually include files, and the final lines instantiates the class and calls the method.

The Zend Framework based API that I have written duplicates the functionality described on Micheal’s page and exposes the API functionality to PHP applications.

Please keep in mind that as this is an unofficial API, Translink may not want their data being used without their permission. Please contact Translink before making use of this API on a production site.

Example Usage

$translink = new Zext_Service_Translink();
 
//Get the route directions for the 135
$directions = $translink->getRouteDirections('135');
Zend_Debug::dump($directions);
 
//Get the east-bound stops for the 135
$stops = $translink->getRouteStops('135', 'east');
Zend_Debug::dump($stops);
 
//Get the details for an arbitrary stop
Zend_Debug::dump($translink->getStopDetails(51845));
 
//get the latitude and longitude for a stop
Zend_Debug::dump($translink->getStopLocation(51845));
 
//get the stops near a given latitude and longitude
Zend_Debug::dump($translink->getLocationStops(49.25344,-123.167895));

Download & Install

Download the following file:
Zext_Service_Translink
Add it to your library path (same folder that Zend of the Zend framework resides in) if you have autoloading enabled.

I will try to respond to any problems in the comments.

Example Usage

$bclc = new Zext_Service_BCLC();
 
//Get the results of the latest 649 results
$bclc->get649Results();
 
//get the estimated jackpot size for the latest 649 results
$bclc->get649Estimate();
 
//Get the 649 Extra numbers for the latest draw
$bclc->get649ExtraResults()

There are many other functions that the API exposes. Take a look through the source to view them all.

Download & Install

Download the following file:
Zext_Service_BCLC
Add it to your library path (same folder that Zend of the Zend framework resides in) if you have autoloading enabled.

I will try to respond to any problems in the comments.

Epguides.com is a listing of TV show episode listings. This can be useful, for example, if you are downloading episodes from the internet and want to automatically name the files. Alternatively, if you are hosting a website this API could expose current episode listings for shows of your choice.

The API consists of three parts:

1. Zext_Service_Epguides – The basic service class. Has one public method: getShowObject($showName)
2. Zext_Service_Epguides_Show – The objet that is returned by getShowObject($showName)
3. Zext_Service_Epguides_Episode – Individual episode objects that reside in the show object

Zext_Service_Epguides

Fortunately, Epguides.com has a very regular naming scheme for their pages. The basic pattern is http://epguides.com/{show_name}/ where show_name is the name of the show, in lowercase, with whitespace and special characters removed. This can be accomplished using the following regex:

$showName = strtolower(preg_replace('/[^\w]/', '', $showName));

Rather unfortunately, epguides.com doesn’t follow this naming scheme for all shows. For example, “The Office” (North American Version) is called “office_us”. Therefore, when querying epguides.com using this API you want to be using the string that identifies the proper URL of the desired show. The API will generate the correct URL for shows that conform to the regular naming scheme, but you must use the epguides.com naming conventions for show names that are exceptions to the rules.

After the API generates the URL for a given show, it fetches the corresponding page from Epguides.com and parses it. The parsing is done by a combination of domnode selection using the DOMDocument object and regular expressions. Fell free to read the source for more details.

The parser returns an array which is then passed to the Zext_Service_Epguides_Show() object.

The Show and episode classes are fairly straight forward and mostly act as containers to provide some convenience functions. I hope you can get a fairly good idea of how to work with them by viewing the below usage example.

Example Usage

$epguidesAPI = new Zext_Service_Epguides();
 
//Gets a show object for The Office
$show = $epguidesAPI->getShowObject('office_us');
 
//Print out all the episodes as strings.
foreach($show->getAllEpisodes() as $episode){
	echo $episode . "\n";
}
 
//Get the 10th episode of the 2nd season
$episode = $show->getEpisode(2,10);

Download and Install

Download the following file:
Zext_Service_Epguides
Add it to your library path (same folder that Zend of the Zend framework resides in) if you have autoloading enabled.

I will try to respond to any problems in the comments.