My best Python HTTP test server so far

I’ve implemented a bunch of test HTTP servers in Python and now I think I’ve got the implementation right:

• Test client code doesn’t have to care about the specific HTTP port. Any free port can be used by the test without interference with other running processes.
• The HTTP server is up and running when the test is ready.
• Resource handling uses the the with statement. The HTTP server is shut-down at the end of the test.
• The concrete request urls (host and port) are transparent for the test.
• Test can be run fully in memory. The only resource allocated is the HTTP socket.

The actual test is brief. We call the http_server with a HTTP handler and the function url is returned. The test can use the url function to create the request url. This is handy as the allocated port of the HTTP server is not fixed. In the test below we check that the returned content matches.

with http_server(Handler) as url:
    assert list(urllib2.urlopen(url("/resource"))) == [content]

To run this test we need a HTTP handler implementation. You could use the SimpleHTTPServer.SimpleHTTPRequestHandler that comes with python and work with files served from a directory. This is any good point to start but setting up a test folder with the necessary content is cumbersome and inflexible.

This handler runs in memory without any additional setup. It will always returns with the 200 response code writes the content into the request.

code, content = 200, "Ok"
class Handler(BaseHTTPServer.BaseHTTPRequestHandler):
  def do_GET(self):
      self.send_response(code)
      self.wfile.write("\n" + content )

The http_server implementation starts a thread, opens a socket and yields the url function. The HTTP request handler runs in the spawned thread.

@contextlib.contextmanager
def http_server(handler):
  def url(port, path):
      return 'http://%s:%s%s' % (socket.gethostname(), port, path)
  httpd = SocketServer.TCPServer(("", 0), handler)
  t = threading.Thread(target=httpd.serve_forever)
  t.setDaemon(True)
  t.start()
  port = httpd.server_address[1]
  yield functools.partial(url, port)
  httpd.shutdown()

I leave it as an exercise to you to write an implementation that reuses the HTTP server in multiple tests. This could be necessary if the overhead of allocating ports dominates the test running time.

Convert videos to wav and mp3 audio files

On the one hand, I listen almost daily to a fluctuating list of podcasts - mostly while commuting and jogging. I find it very relaxing. On the other hand, I have an ever increasing list of presentation videos I would like to watch, but I don’t actually do it. For some talks and most interviews the video is nice but not necessary. The audio is sufficient to get the information and it’s handy to be able to listen to interesting stuff on the run.

The following bash script downloads the video from youtube and encodes the audio into an low quality mp3 file.

#!/bin/bash -e
cd $(mktemp -d youtubedownXXXX)
youtube-dl -f worst -t $1
name=$(echo *)
ffmpeg -i $name -vn -acodec pcm_s16le -ar 44100 -ac 2 $name.wav
lame -m m -q 0 --vbr-new -B 64 $name.wav $name.mp3

The script depends on youtube-dl, ffmpeg and lame. It creates a temporary folder where it puts the downloaded video, wav and mp3 files.