February 24, 2010

PyCon 2010 Recap

The videos from PyCon 2010 appear to be coming online now and I highly recommend checking them out if you weren't able to attend or missed some of the sessions you wanted to see while there at other sessions. There was a ton of great stuff to soak in through osmosis. In addition, it was great to meet others in the community that I know through IRC / Twitter / Email / Projects.

Here is my recap of things that impressed me and that I saw as directly applicable to what I am working on.

redis / pyres / celery

  • redis, pyres, celery
  • I need to use Redis with pyres or celery.
  • Could replace a lot of my crontabs
  • Could replace local queuing (versus distributed queuing running in the cloud).
  • Could replace adhoc email queue infrastructure
  • Simplifies stuff done asynchronously with crontabs - queue up the fact that something needs to be done at the moment instead of running queries repeatedly looking for conditions to trigger action.
  • Becomes super simple to mark things as something to execute asynchronously so more functionality can be added without slowing down website (e.g. logging, near real-time data warehousing, etc.)

picloud

  • picloud
  • Looks promising, however, cannot execute non-python code (e.g. ffmpeg)
  • Client is LGPL but it is pure Python so should be able to at least glean ideas from it especially how bytecode is compiled and shipped for remote execution.

Haystack / solr

  • Haystack, solr
  • solr seems to be what everyone prefers. not much talked about this, but rather Haystack
  • Haystack is genius.
  • It's designed for Django. It's modular.
  • Interface/API is simple and clean.
  • Makes adding new search indexes easy and useable.

Beautiful Architecture

  • Greg Wilson
  • Beautiful Code author
  • 20 pages describing architecture of great software or, comparing architectures of two similar pieces of software from community for chapters.
  • Looking forward to reading this when complete.

Unit Testing

  • Ned Batchelder
  • coverage.py, lots of improvements over the past year
  • HTML reporting
  • Branch coverage
  • Cobertura-compatible for XML output for CI reporting
  • Can set settings in .coveragerc to ignore certain lines
  • Coming: will work with nose and test.py
  • unittest in Python 2.7 gets major update
  • new unittest stuff is backported and available as module as unittest2
  • provides nose-like test discovery

Creating/Reading Word Documents in Pure Python

virtualenv

  • Need to be doing things in virtualenv with virtualenvwrapper
  • creates virtual python environments
  • easy segregation of third party library dependancies
  • good for managing upgrade efforts to things like Django or SimpleJson, etc.

pip

  • pip blows easy_install out of the water
  • can install directly from svn, hg, or git
  • can cleanly uninstall
  • recovers from failed installs (doesn't leave site-packages in strange state like easy_install can)
  • easy requirements management for a package
  • this could be part of our internal deployment strategy
  • easy upgrading of existing packages
  • could be very useful in bootstrapping different versions of software in cloud environment
  • especially powerful when used with virtualenv