xmpp

Part of the reason why pjs pjabberd only works on a single processing unit (core) right now is because of the GIL in Python. The architecture uses the asyncore module (asynchronous, like twisted). Looks like it was a good choice, though. In Concurrency with Python, Twisted, and Flex Bruce Eckel demonstrates how he got extreme parallelism from Python+Twisted by spawning n python interpreters, where n is the number of processing units on a machine, and communicating between them via sockets. The Flex part can be mostly ignored. This will come in handy for pjabberd when we add support for multiprocess operation and clustering.

This is my lame-o jabber video.

Video editing on Linux sucks. I used Cinerella to make this. Issues encountered:

• The slides had to be individually exported to PNG from OpenOffice.
• Each slide then had to be individually resized in gimp to match the size in the video. Cinerella just couldn’t do that for me.
• Cinerella crashes once in a while, so I learned to save my work every 5 seconds.
• Cinerella’s UI is awful, but you get used to it.
• The biggest problem is its importing and rendering. I had to try out 10-15 different combinations of audio/video formats to make it render the video properly. It couldn’t import any avis or movs that I generated from recordmydesktop’s ogvs (via mencoder). The only format it liked was mp4.
• Then it couldn’t get the sound right. It was recorded with Audacity in mono with 16kbps bitrate. Cinerella kept producing either no sound or sound in only one channel. I had to manually drop the bitrate in the rendered result to 16kbps and tinker with output settings (neither mp3 nor mpeg audio worked). I had to manually duplicate the contents of the audio track into another, which resulted in 2-channel output (even though the project was already set to 2-chan).

To give it credit though, the docs are pretty decent.

I spent way too much time making it all work. Next time, I’ll just find a Mac that I can use to do this.

These slides by Peter Saint-Andre cover a lot of interesting potential applications of XMPP (and similar technologies). The vastness of applications is astounding.

Done:

• Routing protocol. It’s very simple. Just a dictionary of handlers with sequential routing rules. Will post more about it when I have time.
• In-process and threaded handlers. The threaded handlers are messy, but seem to work. They’ll be required for all I/O operations.
• XMPP <stream>s.
• SASL PLAIN and DIGEST-MD5 auth (no iq-auth yet, which means most clients can’t connect yet). Gaim does connect properly, though.

TODO:

• S2S (ie. dialback)
• <message>, <presence> from RFC 3921.
• rosters and subscriptions from RFC 3921. Probably won’t have time for privacy lists this term.
• TLS using tlslite. Have been trying to make it work with my own asyncore module (which watches functions as well as sockets!).
• docs + c2s/s2s tests.

When you have an application that’s event-driven (like twisted), you’ve gotta eliminate all potentially blocking operations, or you risk putting your entire app to sleep while you wait for the blocking operation to complete. The XMPP server I’m working on has to send out DNS queries all the time, and these could take a while. I had to figure out how to make these queries asynchronous, so that they wouldn’t block the entire server. There are adns bindings, but the GNU adns library is licensed under the GPL (I’m trying to stick to more liberal licenses) and not straightforward to install on Windows. There’s a DNS toolkit library for python, but it doesn’t support asynchronous lookups natively. Enter the asyncore module. The following should allow you to query DNS and then get callbacks to handle_read() when the data is available.

import asyncore, socket
from dns import resolver, rdatatype, rdataclass, message
 
class AsyncDNS(asyncore.dispatcher):
    def __init__(self):
        asyncore.dispatcher.__init__(self)
        self.r = resolver.Resolver()
        self.query = ''
 
        self.create_socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.connect((self.r.nameservers[0], self.r.port))
 
    def makeQuery(self, server):
        self.query = message.make_query('_xmpp-server._tcp.%s' % server,
                                   rdatatype.SRV, rdataclass.IN).to_wire()
 
    def writable(self):
        return (len(self.query) > 0)
 
    def handle_read(self):
        data = self.recv(4096)
        answer = message.from_wire(data)
        print answer.answer[0].to_text()
 
    def handle_write(self):
        self.send(self.query)
        self.query = ''
 
    def handle_accept(self): pass
    def handle_connect(self): pass
    def handle_bind(self): pass
    def handle_close(self): self.close()
 
if __name__ == "__main__":
    d = AsyncDNS()
    d.makeQuery('livejournal.com')
    asyncore.loop()

There’s quite a bit of commotion around XMPP. Bill de hÓra collects a bunch of links in a single post. I stumbled recently upon ISS, which uses JIDs for identities. RSSPing looks like a hack compared to the XMPP standard (with PubSub).

This is all exciting as my CSC490 project involves creating a Python-based XMPP framework (disguised as a server).