Sometimes when you are building a web application, you want to use different layouts for different parts of the site. For example, in a content management system, you may want one layout for normal users and another, completely different layout for site administrators. Read the rest of this entry »
Today I was optimizing a site that uses heavy PHP and Ajax. I wanted to reduce the amount of data that was being sent from the server. To put this in perspective, if there were no cache hits in a page load there would be a total of 755 KB pulled down over 123 requests.
Read the rest of this entry »
I have several servers which run an assortment of http, svn, ssh, and ftp services. One of the largest annoyances are automated breaking scripts pounding my services. Recently, I have been looking into blacklist.py: a handy python script written by Reto Glauser, which monitors syslog-ng logs looking for possible break-in attempts. The script uses iptables to block future traffic from suspicious IP’s for a specified amount of time.
After I got the script setup and running I wanted a Gentoo init script that would automatically start the script on boot. After reading through some examples in my /etc/init.d/ directory I seem to have managed to cook up something that works: Read the rest of this entry »
I have been using the Dojo toolkit as my Javascript library of choice since back in early 2006 when it was still around version 0.4. Since then, the project has made tremendous strides including the release of version 1.0 and 1.1 with 1.2 on the way. At the beginning of 2008 I started using the Zend Framework to build MVC PHP applications and, with the release of 1.5, it has become my PHP framework of choice.
To my delight, the Zend Framework and Dojo have recently announced a partnership which will lead to tighter integration of these two great open source frameworks.
One of the most exciting additions to the Zend Framework is the Zend_Json_Server support for JSON-RPC. I have been using JSON-RPC with Dojo for quite some time and, up until now, it has been challenging to find a design pattern that plays nice with the Model View Controller implementation in the Zend Framework. However, with the addition of the Zend_Json_Server this is no longer the case. Read the rest of this entry »
I love Adblock Plus - I refuse to use the internet without it. I also quite enjoy Greasemonkey. Unfortunately, neither is yet compatible with the new Firefox 3b5. I got tired of waiting so I took measures into my own hands.
The XPI file format is basically a glorified zip file, so you can unzip it with a compression utility and modify the maximum version number in install.rdf. Basically I replaced the value in the em:maxVersion field with 3.* so to make it compatible with all versions of Firefox.
Of course, these maximum version numbers are there for a reason: newer versions of Firefox may break the extension functionality, but no problems so far.
For the lazy, I have uploaded the patched install files. Of course I give zero guarantees or warranty for the correct operation of these patched extensions. I encourage you to upgrade to the official compatible version as soon as possible.
To install:
Adblock Plus For Firefox 3 Beta 5 The Adblock Plus Development build now supports Beta 5. Get it here: http://adblockplus.org/devbuilds/
Recently I have been playing with the dojox.dtl: the javascript port of the Django templating engine. So far I am quite impressed, not only is it fast and full featured, but by writing a wrapper class it is easy to make it behave like server side templating systems: you specify a template and pass it an object and it will render that object according to the template rules.
The only trouble I ran into was when I wanted to used Dijit Widgets in my templates. Since on my main page I didn’t know what template I would be calling, I didn’t know which Widget classes to include with dojo.require(). To fix this, I have come up with a little hack that does the job, although not too elegantly.
Bascially my approach includes hooking into dojo.parser and changing its behaviour. Now when it is parsing widgets from the page it checks to see if they exist and, if not, it does the appropriate dojo.require() to try to pull them in.
var fn = dojo.parser.instantiate;dojo.parser.instantiate = function(nodes){dojo.forEach(nodes, function(node){var className = node.getAttribute(dojo._scopeName + "Type")if(!dojo.isFunction(dojo.getObject(className))){//It is not an object... yetdojo.require(className);}});return fn(nodes);}
The above code-block shows the implimentation. Simply place this in between <script> tags in your header. Now you should be able to do dojotypes anywhere on your page and have the appropriate classes automatically included.
There are several disadvantages to my approach. For example, if you have a typo in one of your dojoType, it will try to pull down a file that doesn’t exist. We also have the impact of running through the array of nodes an additional time, although since the dojo.parser.instantiate function is already bounded by O(n) this shouldn’t make a noticeable impact.
There are several Firefox plugins which allow the user to manipulate their browser cookies. However, most of these plugins force the user to manipulate cookies individually. This can become tedious if the user is simply “importing” cookies from, say, a wireshark dump.
The CookieInjector userscript simplifies this process, by allowing the user to copy-paste the cookie portion of the dump and have the cookies from the dump automatically created on the currently viewed web page.
Fire up Wireshark, formally Ethereal, if you don’t have Wireshark you can grab it from: http://www.wireshark.org/. Start listening for traffic on the same interface you use to access the internet. To cut down on extra packets, enter tcp as a capture filter. TCP is a transport layer protocol featuring reliable transport, congestion control and connection oriented transfers. Since HTTP uses connections between client and server and therefore the TCP protocol, is is safe to filter out all non-TCP packets. To further filter the packets that Wireshark is displaying enter http.cookie in the filter field. This will filter out all packets which are not using the HTTP application layer protocol and all HTTP packets which do not contain cookies.
Next go to a website that uses cookies. Most websites which support user logins or shopping carts use cookies for these purposes. Make sure that the website that you visit does not encrypt the entire session (such as a banking website), otherwise the packets will be encrypted and not viewable in wireshark. After capturing a couple packets which contain cookies scroll down to the Hypertext Transfer Protocol portion of the packet preview, expand it, and scroll down to the cookie line. Right click on the line, and select copy->Bytes (Printable Text Only). This will copy the human-readable portion of the packet which represents the Cookies associated with this website.
If you haven’t already, install Greasemonkey, and the CookieInjector userscript. Clear your private data, ensuring that the Cookies and Authenticated Sessions options are selected. This will delete all your cookies, so we can see the script in action. Press alt-c to view the CookieInjector dialogue, paste the cookie string from wireshark into the text box and click OK.
Congratulations! Your cookies have now been restored!
After the page has loaded the CookieInjector class is initialized. This involves setting up the dialogue and binding a function to the onkeydown event. When the user presses the ALT-C key combination, the CookieInjector keyPress function is called, which checks to see if the correct key combination has been triggered. If it is valid, the dialogue’s display style is changed, making it visible in the middle of the page.
After the user enters the cookie that was copied from Wireshark, the script does a quick cleanup of the string, and then adds the cookies to the browsing session.
Note that the cookie’s host will be the domain that is loaded in the browser when the cookie is injected. The root path will be used for the root of the cookie to ensure that the cookie is persistent across the entire domain. Finally, the cookie is a session cookie, which means that the cookie will expire when the browser is closed.
The use of cookies for identification and authentication presents a dangerous security risk for un-encrypted connections. Most websites (such as Hotmail, Facebook and Gmail), only encrypt the username and password when initially authenticating the user and all traffic following the initial handshake is un-encrypted. As a result, the cookie information is readable by anyone who is listening with appropriate software, and malicious users can steal the cookies of other users on the network, possibly gaining access to their accounts. Un-encrypted or weakly encrypted wireless connections (those which do not use WPA or stronger encryption schemes) are especially susceptible to cookie stealing. This is because anyone with a wireless card can simply listen to all network traffic as it is broadcast through the air, intercepting cookies, images, web pages and any other traffic which may or not be intended for them. Intercepting traffic on a switched network (most LANs) is more complex, but can be accomplished using ARP Poisoning or software such as Ettercap
The take-home lesson is to use encrypted connections, like https, whenever privacy is important. Always remember that if the connection is not encrypted anyone could be listening in.
Download CookieInjection Userscript
Greasemonkey: https://addons.mozilla.org/en-US/firefox/addon/748
HTTP Protocol: http://en.wikipedia.org/wiki/HTTP
TCP Protocol: http://en.wikipedia.org/wiki/TCP
Cookies: http://en.wikipedia.org/wiki/HTTP_cookie
Wireshark: http://www.wireshark.org/
Ettercap: http://ettercap.sourceforge.net/
ARP Poisoning: http://en.wikipedia.org/wiki/ARP_spoofing
Ethereal: http://www.ethereal.com/
For those of you who read about the re-launching of the SFU Bookswap website (http://sfubookswap.com), you may remember that one of the new features was the integration of the Google Books database.
In this entry I am going to talk about how I integrated the Google Book Search functionality into my existing Dojo framework.
First things first: you need to sign up for a Google AJAX Search API key. You can get your key, as well as a description of the API and documentation from Google here: http://code.google.com/apis/ajaxsearch/signup.html.
Google offers a Search container, which allows the integration of several search sources, including the web, images, blogs, news items and books. I only wanted the Book source so I chose to take the simple route and not use the container class. Instead I used the GbookSearch object directly, which allows me to access the data more easily as well as utilize only the desired functionality.
The very first step in integrating the Google Books search results is including the Google class on your page:
<script type="text/javascript" src="http://www.google.com/jsapi?key=<yourapikey>"></script>
Next, we need to ensure that the Google Search namespace is loaded so we have access to the searching functions. This is as simple as including script tags with the following
google.load("search","1");
Because objects are swell, and will allow us to instantiate multiple searches on the same page, we are going to encapsulate the Google Book Search object in a wrapper object which exposes only the functionality we desire.
dojo.declare("GoogleBookSearch", null, {
constructor: function(){
this._bookSearch = new GbookSearch();
this._bookSearch.setNoHtmlGeneration();
dojo.connect(GbookSearch,"RawCompletion",this,"resultCallback");
},
query: function(q){
this._bookSearch.execute(q);
},
resultCallback: function(){
console.debug("Got Data Back, Lets Take A look:");
console.dir(this._bookSearch.results);
for(var x=0; x<this._bookSearch.results.length; x++){
document.body.appendChild(this._bookSearch.results[x].html);
}
},
goToPage: function(p){
this._bookSearch.gotoPage(p);
},
getNextPage: function(){
this._bookSearch.gotoPage(this._bookSearch.cursor.currentPageIndex+1);
}
});
The first line of this class (the dojo.declare) creates a new GoogleBookSearch object, which does not inherit from any object (the null parameter), which has several functions and attributes.
The constructor function sets up the search object by creating a new GbookSearch instance. To gain a slight speed improvement, we disable HTML generation, which will prevent the GbookSearch class from downloading images and generating Google’s HTML representation of the search results. Finally we connect the “RawCompletion” event that is fired by the GbookSearch object to a callback function in the local object. After every search query is completed, this event is fired by the global object.
The rest of the functions are fairly self explanatory: the query function accepts one argument, which is the query from the user. The query can be an ISBN, author, title or any other combination of book properties that Google stores in their book database. The resultCallback function processes the data after the GbookSearch class has received search results from the server. Notice that they do not come in the form as an argument to the function but, rather, are accessed by iterating through the _bookSearch.results array. The gotoPage and getNextPage move to the next page of search results (as only 4 are returned by default). No other code is needed, as after getting the results for the next page, the “RawCompletion” event is fired once again.
To use your shiny new class, simply instantiate it after the page has loaded. So, for example:
dojo.addOnLoad(function(){
myBookSearch = new GoogleBookSearch();
});
Make sure you don’t put a var in front of bookSearch, otherwise the object will have only local visibility, and be removed after the function has executed.
In my case, I wanted to have a default callback function that is executed when the search results are returned. However, I wanted separate instances of the GoogleBookSearch object to handle search results differently. To do this, we want to “overload” the resultCallback function using dojo’s hitch functionality.
dojo.addOnLoad(function(){
myBookSearch = new GoogleBookSearch();
myBookSearch.resultCallback = dojo.hitch(this,function(){
if( myBookSearch._bookSearch.results.length>0){
alert('Showing '+myBookSearch ._bookSearch.results.length+' of '+ myBookSearch._bookSearch.cursor.estimatedResultCount +' Results');
}
});
});
The dojo.hitch function accepts two arguments, a function or method name as the second argument, and the scope in which the method executes as the first argument. It is crucial that this scope is set correctly as the default behaviour is to execute in the scope of the GoogleBookSearch object which may be ok if only global or Google BookSearch attributes and functions are used, but is insufficient if we are wrapping the GoogleBookSearch object inside another object and want the callback to be defined as a function in the outer object.
When returning the results, if any of the attributes of the book were a keyword in the search, for example, the author field contained a keyword, then the keyword in the author field would be enclosed in bold tags. In my application I am importing data into a database so I want it to be as clean as possible. My solution is using a regular expression to remove all HTML tags. For example
myBookSearch._bookSearch.results[0].authors.replace(/(<([^>]+)>)/ig,"")
You may notice that the book covers are dynamically generated for each query and expire after a short time. My workaround for this is to use regular expressions to parse out the book ID and then use the same (permanent) address that Google does when displaying the book covers on the book details page.
This can be accomplished as follows:
var imgUrl = "http://books.google.com/books?id="+/.*id=([^&]*)/g.exec(myBookSearch._bookSearch.results[0].tbUrl)[1]+"&printsec=frontcover&img=1&zoom=1"
The regular expression extracts the book ID from the thumbnail URL and combines it with the persistent book cover URL
There are a bunch of other attributes stored by the GbookSearch object. Look in the Google API to see a bunch of them or, alternatively, use Firebug to snoop the object after results have been returned.
Happy Coding!
External Links
Google AJAX Search: http://code.google.com/apis/ajaxsearch/
Google AJAX Search API: http://code.google.com/apis/ajaxsearch/documentation/reference.html
Firebug: www.getfirebug.com/
