distributed

Python concurrency

Part of the reason why pjs pjabberd only works on a single processing unit (core) right now is because of the GIL in Python. The architecture uses the asyncore module (asynchronous, like twisted). Looks like it was a good choice, though. In Concurrency with Python, Twisted, and Flex Bruce Eckel demonstrates how he got extreme parallelism from Python+Twisted by spawning n python interpreters, where n is the number of processing units on a machine, and communicating between them via sockets. The Flex part can be mostly ignored. This will come in handy for pjabberd when we add support for multiprocess operation and clustering.

Posts feed Posts feed