lost-theory

I started using DreamHost PS yesterday and after a few hours had a Zine blog running. That's the quickest turnaround I've ever experienced for getting a Python web app up and running and exposed to the web.

It was better than starting from scratch on a new VPS, but there was some weirdness. DreamHost PS is like a VPS where you can run what you want, but without root. It's like using your friend's server and he doesn't trust you very much. :)

Here are my steps for getting Zine running on DreamHost PS.

Administrivia

• First, I cranked the resources down to the minimum in the DreamHost PS admin panel. They start you off at the highest setting so you can monitor how much resources your server actually needs, but if you're going to have little traffic to start I don't see the point.
• Next, I created a new shell user on my DreamHost PS instance.
• Finally, I made a new subdomain (blog.lost-theory.org) belonging to that user.

virtualenv & pip

DreamHost is nice enough to give you a good version of Python (2.4.4) and easy_install, so you can dive right in. I started by first setting up my virtualenv:

$ cd ~
$ mkdir -p zine/lib
$ easy_install --install-dir=~/zine/lib virtualenv
$ easy_install --install-dir=~/zine/lib pip
$ cd zine
$ virtualenv .

Install Zine and its packages

You can then start installing Zine and the packages it depends on into that virtualenv.

$ cd ~/zine
$ wget http://zine.pocoo.org/releases/Zine-0.1.2.zip
$ unzip Zine-0.1.2.zip
$ pip -E . install Werkzeug
$ pip -E . install Jinja2
$ pip -E . install MySQL-python
$ pip -E . install SQLAlchemy
$ pip -E . install simplejson
$ pip -E . install pytz
$ pip -E . install Babel
$ pip -E . install html5lib
$ (try to install lxml...)
(over 9000 compilation errors)

Now here you will run into a problem since lxml requires the libxml and libxslt packages. On DreamHost PS we don't have root, so we can't install these packages with apt-get install. I took a peek at how Zine uses lxml and it seemed like I might be able to get away without installing it:

./importers/wordpress.py:15:    from lxml import etree
./importers/feed.py:13:    from lxml import etree
./zxa.py:21:    from lxml import etree

I tried wrapping those lines with try/except like so:

try:
    from lxml import etree
except:
    print "Skipping lxml import... will die later"

And sure enough that will work when you start serving up your Zine instance. I haven't had a problem so far with skipping the lxml import (because I haven't used those features yet that require it). It might be possible to use elementtree instead, but it's working fine for now.

Install and quickstart

Install the Zine package:

$ cd ~/zine/Zine-0.1.2/
$ ./configure --prefix=~/zine
$ make install

After that you can create and start an instance:

$ cd ~/zine
$ mkdir instance
$ ./Zine-0.1.2/scripts/server -I instance

This will start the install wizard on port 4000. Go check it out!

Database

You won't get very far without a database though. One important thing to remember is that you do not need to use the DreamHost PS MySQL service. You can use your existing DH MySQL. All I had to do was set up a new user and database on the existing MySQL from my regular shared hosting service.

After that's set up put the DB URI in the install wizard and you're pretty much done.

At this point you'll have Zine fully functional on port 4000. You can start writing entries and checking out the themes and all that. But we can do better than running on port 4000 with a development server, can't we!

Serve Zine using cherrypy and lighttpd

I want to run Zine on port 80 and serve it with something a little more powerful, so I checked out what the DreamHost admin panel offered. There are settings for proxying and Mongrel and FCGI, but those don't really apply. I checked out Private Servers > Configure Server and saw that you can switch between Apache serving on port 80 or port 81. I switched it over to 81 thinking I could run whatever server I want on port 80.

Unfortunately, you'll get a socket.error: (13, 'Permission denied') error when firing up your own server on port 80. I first thought it was because Apache was still bound to 80, but it's actually because unprivileged users can't bind to ports under 1024. Not having root bites us again. :(

DH gives you two choices for serving on port 80: Apache (the default) or lighttpd. You can run your own long-running processes, but they have to serve through Apache/lighty (using CGI, FCGI, and I think Phusion Passenger and maybe a few other options). You can't run your own server on 80.

I chose lighty for the smaller footprint and because I find its configuration a bit easier. To proxy all requests at the root of the domain from port 80 to port 4000 you can use the following:

$HTTP["host"] == "blog.lost-theory.org"
{
  proxy.server = (
    "" => (
      "blog" => (
        "host" => "127.0.0.1",
        "port" => 4000
      )
    )
  )
}

The configs are stored per-host here: /usr/local/dh/lighttpd/servers/your-user/

We can't mess with the main lighty config file, again, because we don't have root. You only have write access to the individual config files for each host.

Once that was set up I decided to swap out the Werkzeug development server for cherrypy's WSGI server. You can keep most of the Zine-0.1.2/scripts/server script the same, just pip install cherrypy and make this change:

#change this...
run_simple(options.hostname, options.port, app, threaded=options.threaded,
           use_reloader=options.reloader, use_debugger=options.debugger)

#to this:
from cherrypy import wsgiserver
import threading
s = wsgiserver.CherryPyWSGIServer((options.hostname, options.port), app)
threading.Thread(target=s.start).start()

Note: This is one of the reasons I <3 WSGI.

One more important change: Zine will probably still think that the address of the blog is http://example.com:4000/. This will make all the links point to that site, which is ugly. To fix this just drop the port number off of the blog_url setting in zine.ini.

Conclusion

That's all it took! I am pretty happy with how easy it was.

Python web apps are not as easy as something like PHP to get up-and-running, but this process was pretty ideal in my opinion. I hated using a bare bones VPS because you end up spending more time on sysadmin and thinking about security holes than on building and deploying cool apps. DreamHost PS seems like a good middle ground between bare bones VPS / dedicated servers and shared hosting. It has a lot of good defaults, polished administration (via the panel), decent price, and the ability to scale upwards.

Google App Engine is a similar service, but I haven't tried it yet. I am a little worried that existing Python apps / code aren't portable to GAE.

As far as resources go, the 150MB memory has been smooth so far. I'll monitor if I need to up my resources. If I do need to increase my resources I'll probably do it since they make it so easy and the price is reasonable. :)

One interesting thing is that after switching to lighttpd+cherrypy (after the spike in the graph) my memory usage went down. I've heard that DreamHost PS has about a 100MB footprint when idle. After switching to lighty+cherrypy my memory usage is ~50MB when idle.

Hope this was helpful if you're interested in running Python apps on DreamHost PS. Happy zine-ing to you!

Update:Dreamhost PS now gives you root access by setting up a new account under "Manage Admin Users" in your control panel. This should make things much easier!