Thoughts from Mads Sülau Jørgensen » Work

Figuring out largest/smallest/median filesizes

Mads Sülau Jørgensen — Thu, 04 Feb 2010 14:00:52 +0000

I had to get some statistics about file sizes today, but couldn’t really find a tool for the job, so naturally, I wrote one.

import os, sys, re
from os.path import join, getsize, exists
 
def median(numbers):
    s = sorted(numbers)
    l = len(numbers)
    if l % 2 == 0:
        a, b = s[l / 2 - 1 : l / 2 + 1]
        if a != b:
            return a + b / 2.0
        else:
            return a
    else:
        return s[l / 2]
 
sizes = []
req_re = None
target = '.'
 
if len(sys.argv) > 1:
    target = sys.argv[1]
 
if len(sys.argv) == 3:
    req_re = re.compile(sys.argv[2])
 
for root, dirs, files in os.walk(target):
    for name in files:
        absp = join(root, name)
        if exists(absp):
            if not req_re or req_re.search(absp):
                sizes.append(getsize(absp))
 
num = len(sizes)
total = sum(sizes)
 
print "Num files: %d" % num
print "Average  : %0.2f KB" % ((total / num) / 1024.0)
print "Median   : %0.2f KB" % (median(sizes) / 1024.0)
print "Min      : %0.2f KB" % (min(sizes) / 1024.0)
print "Max      : %0.2f KB" % (max(sizes) / 1024.0)

Usage should be self-explanatory.

Keeping ssh connections alive

Mads Sülau Jørgensen — Fri, 12 Jun 2009 17:24:16 +0000

I’ve got nothing more to say than:

mads@workmads ~ % cat .ssh/config
ServerAliveInterval 60

Happy ssh’ing.

Python http_head method

Mads Sülau Jørgensen — Fri, 15 May 2009 10:30:28 +0000

Seeing as there is no really easy way to do a HTTP HEAD request from python, I wrote up the following small method:

In advance I’d like to apologize for the method that assemblies the request path.

Update: Added handling of redirects.

def http_head(url):
    import httplib
    import urlparse
 
    redirects = 0
 
    while redirects < 10:
        scheme, netloc, path, query, fragment = urlparse.urlsplit(url)
 
        if scheme == 'https':
            conn = httplib.HTTPSConnection(netloc)
        else:
            conn = httplib.HTTPConnection(netloc)
 
        conn.request("HEAD", "%s%s%s%s%s" % (path, query and "?" or "", query,
                                             fragment and "#" or "", fragment))
 
        res = conn.getresponse()
 
        if res.status in (301, 302) and res.getheader('location'):
            url = res.getheader('location')
            redirects += 1
        else:
            break
 
    return res.status, res.reason

Formatting xml with xmllint

Mads Sülau Jørgensen — Wed, 13 May 2009 13:11:48 +0000

I keep forgetting how to format and indent xml from the command line. The tool xmllint does a fine job of doing just that, which has saved me numerous times whilst working with sports results. So. Much. Data.

Running

xmllint --format <file>

will re-format and re-indent the xml in the input file, and check it for various errors while doing it.

Very simple email sending method in python

Mads Sülau Jørgensen — Mon, 11 May 2009 17:30:24 +0000

I needed to send an email, so I came up with this:

 
def send_plain_mail(subject, body, from_mail, to):
    import smtplib
    from email.MIMEText import MIMEText
    from email.Encoders import encode_quopri
 
    msg = MIMEText(body, 'plain', 'iso-8859-1')
 
    msg['Subject'] = subject
    msg['From'] = from_mail
    msg['To'] = to
 
    s = smtplib.SMTP()
    s.connect()
    s.sendmail(from_mail, [to], msg.as_string())
    s.close()

Not rocket science, but it gets the job done.

Using mercurial with eclipse to edit actionscript files on os x

Mads Sülau Jørgensen — Wed, 11 Mar 2009 09:38:33 +0000

We have recently switched to use mercurial as our DVCS. We’re hosting our many repositories on bitbucket.

I love it.

When it comes to actionscript files, flash has always had odd newlines. For some reason, it has always used r as it’s newline. Mercurial, beeing a unix tool, likes it’s newlines to be n. So what better to do, than to make mercurial encode/decode the files for us.

To do that, we can simply add the following to our .hgrc file:

[encode]
*.as = perl -pe 's/r/n/g'
[decode]
*.as = perl -pe 's/r/n/g'

I don’t really know wether the decode part is necessary, but I like to keep it around (if someone should commit poison).