Friday, June 20, 2008

Psyco - massively speed up your Python programs without modifying the source code

http://psyco.sourceforge.net/index.html

"Think of Psyco as a kind of just-in-time (JIT) compiler, a little bit like what exists for other languages, that emit machine code on the fly instead of interpreting your Python program step by step. The difference with the traditional approach to JIT compilers is that Psyco writes several version of the same blocks (a block is a bit of a function), which are optimized by being specialized to some kinds of variables (a "kind" can mean a type, but it is more general). The result is that your unmodified Python programs run faster."
Parallel Python - an awesome distributed computing library for Python

http://www.parallelpython.com/

Parallel Python helps you execute your Python code on multiple cores on the same machine, or on a whole cluster of network machines, all transparently, load balanced and with fault tolerance.

"Features:
  • Parallel execution of python code on SMP and clusters
  • Easy to understand and implement job-based parallelization technique (easy to convert serial application in parallel)
  • Automatic detection of the optimal configuration (by default the number of worker processes is set to the number of effective processors)
  • Dynamic processors allocation (number of worker processes can be changed at runtime)
  • Low overhead for subsequent jobs with the same function (transparent caching is implemented to decrease the overhead)
  • Dynamic load balancing (jobs are distributed between processors at runtime)
  • Fault-tolerance (if one of the nodes fails tasks are rescheduled on others)
  • Auto-discovery of computational resources
  • Dynamic allocation of computational resources (consequence of auto-discovery and fault-tolerance)
  • SHA based authentication for network connections
  • Cross-platform portability and interoperability (Windows, Linux, Unix, Mac OS X)
  • Cross-architecture portability and interoperability (x86, x86-64, etc.)
  • Open source"
I tried it, it's very easy to set up.

A review: http://www.devchix.com/2007/08/03/parallel-python-review-in-a-nutshell-wow/

Tuesday, June 10, 2008

Keep alive SSH session

SSH sessions time out normally if you leave them open for a few minutes. You can fix this by configuring SSH to send keep alive packets from time to time. To do this, add the following line:

ServerAliveInterval 5
in

/etc/ssh/ssh_config