During last night's meeting a question was raised about benchmarks for the python language, or how fast it is relative to other computer languages. Since I am a computer language buff, I will attempt to give a more informative answer to this question, (to augment and correct my utterance of last evening).

Basically a benchmark is an implementation of a timing procedure, that determines how fast a specified program runs in a given computer language, or a software manufacturer's implementation of a given language. Various accounts have been written as to what should be the components of the ideal benchmark, (using the sieve of Eratosthenes, etc.). Of course, these, can give an idea as to how fast various languages perform. However, a benchmark should be a reflection of the type of programs, you are most interested in using.

If your applications will mostly be using disk file i/o, or large memory transfers, then a number crunching algorithm, as a benchmark, may not give you a relevant answer. Even, if you are focused on number crunching, the numerical algorithms used in calculating say, perspective views in a graphics program may be quite different from a program performing finite element analysis, or number theory conjectures.

(Urban Legend Gossip Note: a software manufacturer may fine tune or optimize their product to run more quickly with the well known benchmark algorithms.)

The above then becomes an argument for designing your own benchmark. By carefully noting what you want to perform with your programs, you can determine what kind of routines will be used. These are the routines you should use in your benchmark.

Having said the above, I will give a benchmark with results, as an example --knowing full well that that I can not please everyone-- the results will only be an indication of run times that may or may not be relevant to your application needs.

In my benchmark, I decided to test run times for: function (or subroutine) calls, and basic arithmetic. I decided in using a not too generally well known algorithm, but one having documented results.

The benchmark I chose, consists of an algorithm that calculates the length maxima, and the maxima of height maxima, of consecutive hailstone numbers from 1 to 100 000. For the definition and properties of hailstone numbers, as used here, refer to the Collatz conjecture. One of the earliest references to this with tabulated results is by:

Brian Hayes SCIENTIFIC AMERICAN (Computer Recreations) Jan. 1984, page 10.

No matter what language is used, all of my benchmark programs are written in the same format. This consists of a main program that writes an output table to a data file, the table entries are calculated by calling a single function or a subroutine.

The benchmark test suite consists of programs written in: C, Java, Python, and a hybrid (consisting of a main routine in Python with a C subroutine). Here are the results...

Running table.c (written in C), elapsed time is:

real 0m0.500s user 0m0.500s sys 0m0.000s

Running table.java (written in Java), elapsed time is:

real 0m15.413s user 0m14.930s sys 0m0.070s

Running table.py (written in Python), elapsed time is:.

real 1m57.656s user 1m57.620s sys 0m0.030s

Running combo.py (written in Python, with a subroutine in C), elapsed time is:

real 0m5.161s user 0m5.090s sys 0m0.070s

Here is the source code...

****************************** table.c *************************************** #include <stdio.h> #include <stdlib.h> #include <sys/types.h>

/* prototypes */ void hsTry(long int, long int *); void hsDisp(void);

/* globals */ static int maxTry =100000; static char fname1[] = "Hsc.data";

main() { hsDisp(); }

void hsDisp(void) { /* prints the longest path and largest height, (as encountered so far) */ typedef struct { char hst; long int hs; long int hsMax; } rcdHM;

int i =0, ecMPH =1; long int hs = 0, hsMaxPath =0, hsMaxHeight =0, hsPH[2] ={0, 0}; rcdHM *hsMPH; rcdHM *idxMPH ;

FILE *fil1;

hsMPH = (rcdHM *) calloc(ecMPH,sizeof(rcdHM)); if (!hsMPH) { printf("\n calloc error \n"); exit(1);} idxMPH =hsMPH;

while (hs < maxTry) { hs++; hsTry(hs, hsPH); if (hsPH[0] > hsMaxPath) { hsMaxPath =hsPH[0]; ecMPH++; hsMPH = (rcdHM *) realloc(hsMPH, ecMPH * sizeof(rcdHM)); if (hsMPH == NULL) { printf("realloc error"); exit(1);} else { idxMPH++; idxMPH->hst ='l'; idxMPH->hs =hs; idxMPH->hsMax =hsMaxPath; } } if (hsPH[1] > hsMaxHeight) { hsMaxHeight =hsPH[1]; ecMPH++; hsMPH = (rcdHM *) realloc(hsMPH, ecMPH * sizeof(rcdHM)); if (hsMPH == NULL) { printf("realloc error"); exit(1);} else { idxMPH++; idxMPH->hst ='h'; idxMPH->hs =hs; idxMPH->hsMax =hsMaxHeight; } } }

if ( !(fil1=fopen(fname1,"w")) ) exit(1); fprintf(fil1, "calculates hailstone numbers \n"); fprintf(fil1, " hs path maxima \n"); idxMPH =hsMPH; for(i=1;i <= ecMPH; i++,idxMPH++) if (idxMPH->hst=='l') {fprintf(fil1, " %9i %9i \n", idxMPH->hs, idxMPH->hsMax);}

idxMPH =hsMPH; fprintf(fil1, "\n hs height maxima \n"); for(i=1;i <= ecMPH; i++,idxMPH++) if (idxMPH->hst=='h') {fprintf(fil1, " %9i %9i \n", idxMPH->hs, idxMPH->hsMax);}

free(hsMPH); fclose(fil1); /* clean up */ }

void hsTry(long int hs, long int hsPH[]) { /* input positive integer, returns path length and maximum height */ /* PathLength = hsPH[0], Height = hsPH[1] */ hsPH[0] =0; if (hs <= 0) printf("out of bounds \n"); while(1) { if (hs == 1) break; else if (hs%2 == 0) hs =hs/2; else if (hs%2 == 1) { hs =(hs*3)+1; if (hs > hsPH[1]) hsPH[1] =hs; } hsPH[0]++; } }

****************************** table.java *************************************** import java.io.*; import java.lang.*; import java.text.*; import java.util.*; import java.lang.reflect.Array; import java.lang.Math.*;

public class Hsj { static int maxTry = 100000; static PrintWriter out = new PrintWriter(System.out, true);

public static long [] hsTry(long hs, long [] hsPH) { // input positive integer, returns path length and maximum height /* PathLength = hsPH[0], Height = hsPH[1] */ hsPH[0] =0; if (hs >= 0) { while (true) { if (hs == 1) break; else if (hs%2 == 0) hs/=2; else if (hs%2 == 1) { hs =(hs+hs+hs)+1; if (hs > hsPH[1]) hsPH[1] =hs; } hsPH[0]++; } } return hsPH; }

public static void hsDisp() throws IOException { // Prints the longest path and largest height, (as encountered so far) File outputFile = new File("Hsj.data"); FileWriter output = new FileWriter(outputFile);

Vector hsMP = new Vector(); Vector hsMH = new Vector();

long hs =0, hsMaxPath =0, hsMaxHeight =0;

long hsPH[] = new long [2];

while (hs < maxTry) { hs+=1; hsTry(hs, hsPH); if (hsPH[0] > hsMaxPath) { hsMaxPath =hsPH[0]; hsMP.add(new Long(hs)); hsMP.add(new Long(hsMaxPath)); } if (hsPH[1] > hsMaxHeight) { hsMaxHeight =hsPH[1]; hsMH.add(new Long(hs)); hsMH.add(new Long(hsMaxHeight)); } } output.write(" calculate hailstone numbers \n");

DecimalFormat hfmt = new DecimalFormat(" 000000000000"); output.write(" hs length maxima \n");

for (int i =0; i < hsMP.size()-1; i+=2) { output.write(hfmt.format(hsMP.elementAt(i)) +hfmt.format(hsMP.elementAt(i+1))+"\n"); }

output.write("\n hs height maxima \n"); for (int i=0; i < hsMH.size()-1; i+=2) { output.write(hfmt.format(hsMH.elementAt(i)) +hfmt.format(hsMH.elementAt(i+1))+"\n"); }

output.close(); }

public static void main (String args[]) { try{ hsDisp(); }catch(IOException event) { out.println("io error");} }

}

****************************** table.py *************************************** #! $(env_path)/env $(py_path)/python # above parameters need to be configured to your own system

# Basic Hailstone number properties, (refer to Collatz conjecture)

def hsTry(hs): 'Find path length and maximum height for a hailstone number.' length = 0 ; height = hs

while (hs>1): if hs%2 == 0: hs/=2 elif hs%2 == 1: hs = hs+hs+hs+1 if hs > height: height =hs length+=1 return [length, height]

def hsDisp(maxTry): '''Write to file, length maxima and maxima of height maxima, of a sequence of hailstone numbers up to maxTry.'''

hs =0; hsMaxLength =0; hsMaxHeight =0 hsPH =[0,0]; hsML =[]; hsMH =[];

while (hs < maxTry): hs+=1 hsPH =hsTry(hs) if (hsPH[0] > hsMaxLength): hsMaxLength =hsPH[0] hsML.append((hs,hsMaxLength)) if (hsPH[1] > hsMaxHeight): hsMaxHeight =hsPH[1] hsMH.append((hs,hsMaxHeight))

fil=open('Hsp.data','w') fil.write('calculate hailstone numbers \n\n') fil.write(" hs length maxima \n") for x,y in hsML: fil.write(' %9d %9d\n' % (x,y)) fil.write("\n hs height maxima \n") for x,y in hsMH: fil.write(' %9d %9d\n' % (x,y)) fil.close

hsDisp(100000)

****************************** combo.py *************************************** #! $(env_path)/env $(py_path)/python # above parameters need to be configured to your own system

# Basic Hailstone number properties, (refer to Collitz conjecture)

# Check for module include path, if not found include it. import sys if filter(lambda x: sys.path[x]=='/examples', range(0,len(sys.path)))==[] : (sys.path).append('/examples')

import Calc

def hsDisp(maxTry): '''write to file, length maxima and maxima of height maxima, of a sequence of hailstone numbers up to maxTry'''

hs =0; hsMaxLength =0; hsMaxHeight =0 hsPH =[0,0]; hsML =[]; hsMH =[];

while (hs < maxTry): hs+=1 hsPH =Calc.hsTry(hs) if (hsPH[0] > hsMaxLength): hsMaxLength =hsPH[0] hsML.append((hs,hsMaxLength)) if (hsPH[1] > hsMaxHeight): hsMaxHeight =hsPH[1] hsMH.append((hs,hsMaxHeight))

fil=open('table.data','w') fil.write('calculate hailstone numbers \n\n') fil.write(" hs length maxima \n") for x,y in hsML: fil.write(' %9d %9d\n' % (x,y)) fil.write("\n hs height maxima \n") for x,y in hsMH: fil.write(' %9d %9d\n' % (x,y)) fil.close

hsDisp(100000)

****************************** Calc.c (called C subroutine) ******************* #include "Python.h"

static void hextTry(long int, long int *); static PyObject *

hs_call(self, args) PyObject *self; PyObject *args; { long int hs, hsPH[2]={0,0}; if (!PyArg_ParseTuple(args, "l", &hs)) return NULL; hsPH[1]=hs; hextTry(hs, hsPH); return Py_BuildValue("[l,l]",hsPH[0],hsPH[1]); }

static PyMethodDef HextMethods[] = { {"hsTry", hs_call, METH_VARARGS, "Find path length and maximum height for a hailstone number." }, {NULL, NULL, 0, NULL} /* Sentinel (depends on python version) */ };

static void hextTry(long int hs, long int hsPH[]) { /* input positive integer, returns path length and maximum height */ /* PathLength = hsPH[0], Height = hsPH[1] */ hsPH[0] =0; if (hs <= 0) printf("out of bounds \n"); while(1) { if (hs == 1) break; else if (hs%2 == 0) hs =hs/2; else if (hs%2 == 1) { hs =(hs*3)+1; if (hs > hsPH[1]) hsPH[1] =hs; } hsPH[0]++; } }

void initHext() { (void) Py_InitModule("Hext",HextMethods); }

In conclusion, I determined that, Python is suited as a prototyping language. Not shown in the timing data, is how quickly it took me to develop complete running error free code in python. This was much less than for any of the other languages. That is, you can develop a scratch pad or prototype application using python, that can give a rough or partial estimate of results for a problem. If need be, the application can be rewritten in a faster language after some of the bugs and coding problems have been initially solved by using python. In the long run, much development time can be saved by using this approach.

-Ihor