Posts filed under: Coding
Introducing prankmail.org (and the Drupal way VS roll’yer own)
A few rainy weekends ago, I had the urge to roll up my sleeves and build something. Every web application I’ve built in the last few years has been built on Drupal, and I wanted to see (remind myself) what it would be like to develop an application from scratch (where “scratch” = a random collection of open source components combined with bits from Drupal). I also had given myself the timeframe of a weekend, so building “Basecamp right” was out. I wanted a simple coding project.
Since I’m extremely childish, immature, and a fan of the practical joke, I decided to build a web app that let you send an email to someone, but make it appear as if it comes from someone else. I decided to call it Prankmail. Frivolous, slightly dangerous, and perfect for a rainy weekend indoors.
Skipping to the good part, you can check it out at:
http://prankmail.org
And if you would like to read about how I built it, and how the
“build from scratch” process compares to building a Drupal site, continue on:
Before I really started with application specific code, I wanted to grab a few Open Source pieces and get a simple and efficient framework to build on (We’re not using Drupal, after all).
I wanted to have clean URL’s (none of that goofy query string nonsense) so I took a look at how Drupal uses MOD_REWRITE and .htaccess files and cobbled together a simple solution. I’ve also been looking at the new Zend framework, and I like the way it maps URL’s to controllers. I am also a fan of Drupal menu system, so I hacked up the arg() function from Drupal, and made it route URL’s to controllers. I like Ruby’s “convention over configuration” and so by default, I set it up so a URL of ‘/message/123′ gets passed to a function ‘controller_message’ with an arg of ‘123′, if that function exists.
I really like Drupal’s very light weight DB abstraction layer, and so I grabbed that as well (mostly because the syntax is so natural to me at this point), and stuck it in my web app.
For templating, I decided to give Smarty one last try. Every time I struggled with the syntax, or found myself fighting along, I could hear Rasmus saying “PHP *is* a templating language, stupid!” Next time, I think I will just stick with a ‘pure’ PHP templating solution.
Setting up this simple framework let me write the rest of the application with speed and ease. Included libraries aside, the actual application itself is quite small.
For help on the UI, I used JQuery and specifically thickbox. I also found the visual JQuery site to be a great resource when I got tripped up on syntax. JQuery is truly fun to code with, and it’s a great feeling to have so much control over the DOM, with such simple and beautiful syntax.
I mean, how is the following snippet not poetry in it’s simplicity:
//add a token to the form that we will check to
//make sure the form processor only looks at forms
//submitted from this page...
$.get("/token",function(txt){
$("#mailform").append('
‘);
});
The above snippet leads into how I was concerned with spammers and bots submitting mail, so I borrowed a technique from Jack Born and used JQuery to append a token (MD5′d salt + timestamp) to the submission forms DOM, and at the same time inserting the same token into the Session. When the form is submitted, I check to make sure that the timestamp is “fresh” (within 20 minutes), and that the two tokens match. This should prevent, or at least make it more difficult for a bot to make a post directly to my form processor (and I don’t have to use a silly CAPTCHA).
So how did the whole process work, compared to using my usual framework of choice, Drupal?
Setting up the whole framework from scratch was definitely the fun part. I really enjoyed having a chance to “do things my way”, rather then hunt through documentation or lines of code in search of an answer. I think many people code because it feels good to build something, and while Drupal gives you many things “for free”, it was very enjoyable and satisfying to dive in and do it myself.
When it got to the point of adding comments and RSS feeds to the site, I have to admit that it was proving to get a bit tedious, and it would have been nice just to “turn comments on”, in Drupal. But all that being said, the whole exercise reminds me that there are many ways to get a job done. Drupal is a fantastic tool for a specific set of problems, but it’s just that.
Moral of the story is?
There are no silver bullets, and building silly websites is easy and fun.
Now go send some mail!
Back from Brussels
So I’m back in San Francisco after returning from Brussels and one insane week packed with conferences. (EuroOSCON, GovCamp, DrupalCon, and BarCampBrussels). I’m exhausted, but buzzing with excitement. There are just too many people doing interesting things in this world.
Next up: San Fran Drupaler’s should unite and throw something serious down at the upcoming Yahoo! Hack Day. I’d like to go, so if there are other Drupal heads in attendance, I’d love to work on any kind of hack involving Drupal, Yahoo! maps, upcoming.org concert data, and last.fm. Send me a shout if you’re interested!
Speeding up XML-RPC calls in Drupal
Through Bryght, I’ve been working on an interesting project that really exercises the “toolkit” and “web services platform” like nature of Drupal. The final architecture of our project ended up having many distributed processes, all sending data to Drupal through XML-RPC, and we were using Drupal mainly to aggregate and display the results.
Performance was very important, and we were having a huge bottle neck with Drupal’s XML-RPC library. XML-RPC calls were talking a long fricken time, and we were making tons of them.
Fortunately I had help from Walkah and Moshe, and I thought I would pass on what we did to drastically increase XML-RPC performance. This worked for us, so maybe it might help someone else out.
The problem is of course that Drupal does a full bootstrap for every XML-RPC call.
Our solution:
The first, simple (and huge) win was to utilize a PHP opcode cacher - this drastically reduces this bootstrapping time. We used eAccelerator, but there are many others.
Secondly, since we were working in a controlled environment (we knew what modules were going to be called via XML-RPC) we were able to hack the xmlrpc.php file to avoid a full bootstrap. The downside is we had to hardcode a few path things into the file, but again, this is for a particular app/site, and the performance gains were worth it.
Courtesy of Moshe, here is our revised xmlrpc.php file (you might want to rename it to xmlrpcs.php or something):
< ?php
// $Id$
/**
* @file
* Optimized php page for handling xml-rpc requests. Your module must use declare the hook_init() or hook_exit()
* so that it is loaded during the boostrap. Otherise, use hook_xmlrpc as usual. Note that only required modules are loaded at this point.
*
* @author
* Moshe Weitzman
*/
// CONFIGURE
$path = ‘./sites/mwpb-9.local/modules/’;
$module = ‘my-xmlrpc-module’; //the name of the module where your xml-rpc calls are
// END CONFIGURE
include_once ‘./includes/bootstrap.inc’;
drupal_bootstrap(DRUPAL_BOOTSTRAP_DATABASE);
include_once ‘./includes/xmlrpc.inc’;
include_once ‘./includes/xmlrpcs.inc’;
require_once $path. $module. ‘.module’;
$function = $module. ‘_xmlrpc’;
$callbacks = $function();
// print_r($callbacks);
xmlrpc_server($callbacks);
// put any common.inc or other core functions here that your module relies upon.
function t($string) {
return $string;
}
?>
This was enough to get a huge increase in performance. (Yeah!)
Further thoughts:
One could use “mysql_query()” and avoid bootstrap all together, although I don’t think this is very much overhead, and the benefits of using Drupal’s “Database abstraction layer” probably out way any performance overhead here.
I wonder if there is any way to get Drupal to intelligently load necessary code instead of doing a full bootstrap. The “hardcoding” method above certainly works, but I am craving a more elegant and universal solution. Any thoughts?
Playlist module released
Over the last little while I’ve been working on a playlist module for Drupal. Farsheed was also working on a playlist module. So we have combined forces and now present to you the fruit of our labours: The new, improved, ass kicking playlist module.
Features include:
- support for mulitple playlist formats, including xspf, m3u, pls
- podcastable playlists
- AJAX ordering of audio tracks
- multiple ways to add an audio track to a playlist
- For the geeks in the crowd, playlists are implemented as a new node type.
Farsheed has set up a nice demo site where you can go and kick tires a bit, but you have to sign up first to create your own playlists. Enjoy!
Micro Formats and Audio = Peanut Butter and Chocolate
Boris has an interesting post summarizing a discussion we had on micro formats and audio files. The problem is that audio found on the internet has its meta information (title, artists, album, etc..) obfuscated by the binary nature of the file. Jonathan left a comment on Boris’ post saying that it’s pretty straight forward to grab this information from the file itself. But how much easier (not to mention computationally less expensive) would it be for crawlers and bots to find audio if meta information is embedded right in the glorious page itself; no need to parse another large audio file again! I think a micro format that is a one-to-one representation of ID3 meta information will be quite useful.
I think it’s also important to point out that I’m not saying this is the only way this information should be stored and retrieved. The audio module (shameless plug) I have just released for Drupal stores all of it’s meta information in a database and will soon be query-able via a web service, but I also think it would be great if this rich meta information was available right in the XHTML.
There is some good preliminary information on the micro format wiki, and in related news Lucas Gonze has also recently been talking about a playlist micro format based on XSPF.
I think audio meta information is one of the most compelling use cases for micro formats yet, and I look forward to more discussion on this!
AJAX side effects
I’ve been busy adding some AJAX code to a new album/playlist module for Drupal. Since this is the first time I have really worked with the XMLHttpRequest stuff, I was surprised at how adding these calls actually made my code a bit cleaner. I felt like I was really able to base my code on the actions my module was going to perform, and as a bonus, I basically have a RESTful interface to my module as well! I don’t know why I never really thought about it, but someone would have to work actively to not let AJAX-ing some code open up a web service. You’ve got to like it when technology that is chosen (primarily) for user interface reasons, comes bundled with a few nice architectural bonuses.
It seems like as AJAX adoption continues, we should see a corresponding increase in RESTful web services being offered.
Java gives way…
Java guru Jason Hunter has a good post on how Java is loosing mind share to “good enough” solutions like Ruby, and although he doesn’t mention it, I think PHP could be on the short list as well.
“It’s been 10 years since Java started beating C++ in the mindshare game. Sure, C++ is still widely used, but it’s mindshare is small compared to Java. JavaOne this year they celebrated Java’s 10 year anniversary on stage with a big cake. It reminded me that C++ itself was 10 years old when Java came on the scene. Remember back: Java wasn’t as “good” as C++. It did less; it ran slower. Its sole advantage was simplicity — a language easier to write and deploy — and over time it got more featureful and faster. Ruby on Rails today looks poised to eat Java’s mindshare on the web tier. If not Rails, then something else. Empirically 10 years seems like the right point.”
Podcasting and the post-album world
Fellow Vancouverite Tim Bray has an interesting post about podcasting and the economic impact it will have on musicians. I agree with just about everything he says. Which is good, since the same ideas he discusses form the impetus for Bryght’s new musician focused web platform.
Tim suggests a subscription based podcast service, and this is one type of revenue generating solution Bryght plans to offer to musicians. I’ve said this before, but based on my own informal surveys, I think there are many musicians out there who might want to be free from the constraints of the album. As an artist, why not just release and sell a few tracks a month, delivered straight to your fans MP3 player through Podcasts?
One could argue that the very concept of the “album” is arbitrary and is simply an artifact of our current (and physical) distribution channels. In the past, an album has been a convenient way to get music to the people. Stores stocked high selling albums, due to limited shelf space. People came and bought them. But now, we are at an interesting time. Digital distribution channels have matured to the point where they will start to influence the art that people produce and consume. Music distribution technology influencing music? That’s right, it’s a two way street!
For example, the phonograph and radio drastically influenced the music that was produced, simply by creating new markets. Would 80’s rock bands have worn as much makeup if their faces weren’t being beamed via MTV to millions of homes?
Looking ahead, I think a post-album future filled with regular releases of individual tracks will be more then simply a sea of singles, and I can’t wait to find out what takes shape.
– Update –
Richard chimes in with some good thoughts
Mambo developers stick a fork in it
It is being reported in a few different places that the core developers for Mambo have split from Miro, the commercial entity that holds the Mambo copyright and trademark. The problems stem from how Miro set up the Mambo Foundation, particularily how they failed to consult many of the main developers.
In response, it seems that the Mambo development team is going to fork the open source project and go forth on their own, which certainly puts many members of the Mambo community in a tight spot.
Most important though, is what will the new forked project be called? My vote is for Lambada, all though Meringue also has a nice ring to it.
Lessons for the Drupal community? Greed sucks, be open and transparent (especially when setting up a foundation), and play nice with others!