February 24, 2010
PyCon 2010 Recap
Here is my recap of things that impressed me and that I saw as directly applicable to what I am working on.
redis / pyres / celery
- redis, pyres, celery
- I need to use Redis with pyres or celery.
- Could replace a lot of my crontabs
- Could replace local queuing (versus distributed queuing running in the cloud).
- Could replace adhoc email queue infrastructure
- Simplifies stuff done asynchronously with crontabs - queue up the fact that something needs to be done at the moment instead of running queries repeatedly looking for conditions to trigger action.
- Becomes super simple to mark things as something to execute asynchronously so more functionality can be added without slowing down website (e.g. logging, near real-time data warehousing, etc.)
- Looks promising, however, cannot execute non-python code (e.g. ffmpeg)
- Client is LGPL but it is pure Python so should be able to at least glean ideas from it especially how bytecode is compiled and shipped for remote execution.
Haystack / solr
- Haystack, solr
- solr seems to be what everyone prefers. not much talked about this, but rather Haystack
- Haystack is genius.
- It's designed for Django. It's modular.
- Interface/API is simple and clean.
- Makes adding new search indexes easy and useable.
- Greg Wilson
- Beautiful Code author
- 20 pages describing architecture of great software or, comparing architectures of two similar pieces of software from community for chapters.
- Looking forward to reading this when complete.
- Ned Batchelder
- coverage.py, lots of improvements over the past year
- HTML reporting
- Branch coverage
- Cobertura-compatible for XML output for CI reporting
- Can set settings in .coveragerc to ignore certain lines
- Coming: will work with nose and test.py
- unittest in Python 2.7 gets major update
- new unittest stuff is backported and available as module as unittest2
- provides nose-like test discovery
Creating/Reading Word Documents in Pure Python
- Mike Maccana / @mikemaccana
- creates word documents
- extracts plain text of documents (parse documents)
- search + replace
- Need to be doing things in virtualenv with virtualenvwrapper
- creates virtual python environments
- easy segregation of third party library dependancies
- good for managing upgrade efforts to things like Django or SimpleJson, etc.
- pip blows easy_install out of the water
- can install directly from svn, hg, or git
- can cleanly uninstall
- recovers from failed installs (doesn't leave site-packages in strange state like easy_install can)
- easy requirements management for a package
- this could be part of our internal deployment strategy
- easy upgrading of existing packages
- could be very useful in bootstrapping different versions of software in cloud environment
- especially powerful when used with virtualenv