Hello,
I must first warn you that this is a long post!
In the midst of a slightly misguided discussion at BangPypers, there was a comment made on the nature of what SciPy is all about. It occurred to me that folks outside the scientific computing community may be unaware of the nature of issues that concern the scientific community and their significant "mainstream" Python contributions. Let me add that I mean "Scientific computing" and "Python" in the larger sense, not just the "scipy" package.
So, what has the Sci brought to Py?
Scientific computing has an extremely large audience -- at least in terms of the sheer number of students. There are probably 10 times as many people who do not do IT/CSE than those who do. The problems they have to deal with are typically quite complex! Many of them have to perform serious numeric computation, symbolic computation, make plots of complex data in 2D and 3D, perform effective visualization, build user interfaces, be web aware and most importantly work with other languages in the interest of performance. They also have some of the largest computational needs anywhere.
Secondly, it is a fact that many of the mainstream scientific computing tools today that are written in Python are actually really sophisticated beasts. I will illustrate several of these with specific examples. I am detailing several packages whose internals embody very interesting tools and techniques and often these have crossed over to mainstream Python.
Probably the most popular Python interpreter used world over is IPython , which was started by a good friend of mine (Fernando Perez) who is actually a physicist by training and currently working on neuro-imaging at Berkeley today. Guess what, his motivation was to write the tool we all love today? -- the need for a powerful interactive interpreter like Matlab/IDEAL and Mathematica. Technically there is a ton of interesting stuff going on under the hood -- working with a curses-based terminal and getting that working in a cross platform fashion is a big task. IPython happens to interact really well with the underlying shell and provides several threaded environments (-wthread, -q4thread, -gthread, -pylab). IPython is also breaking interesting ground in the area of distributed/parallel computing with their newer versions. In addition to this they also broke somewhat new ground in the "Python for scientific computing" community by using DVCS very early. It is very likely that Fernando will be here with us for SciPy.in 2009.
Sage is a huge package that brings together a very large collection of other tools written in a variety of languages under one Python based platform along with a very cool web based interactive notebook. The target audience being mathematicians, scientists and engineers. Sage also has a very large developer base, and follow the principle of "release very early and very often". They have some great documentation and a really solid development policy. They have over 75% test coverage (and growing) -- for a tool that large, that is really quite something. The sage project also forked a branch from Pyrex that has led to a very important piece of the scientific computing puzzle, namely high performance Python via Cython. This is one project that will probably make best use of any type annotation additions to Python.
Keep your eyes open for a potential "Sage days" (a Sage conference) in August in India next year that FOSSEE will be organizing with the Sage lead, Prof. William Stein.
The NumPy package is the backbone of most Python based numerical computing packages and has actually motivated the need for enhancing the buffer protocol in Python3. Incidentally, Travis Oliphant, the architect of numpy and the buffer protocol for Python 3 is the keynote speaker at SciPy.in this year.
The SciPy package itself builds atop numpy and a suite of well established algorithms written both in C/C++ and FORTRAN to provide Pythonistas a mechanism for serious number crunching.
matplotlib library is the leading 2D plotting Python package in business today. Almost everyone uses it, all over the place.
The tools that I've been personally working on, for the last several years also deserve mention for their sheer technical qualities. The Enthought Tool Suite (ETS) boasts a huge array of useful packages along with very interesting technology (all of which is BSD licensed BTW). Traits forms the cornerstone of this tool stack. Traits allows us to write code with a strong emphasis on focusing on your model, strongly encourages MVC and allows for inversion of control. I like to think of traits as Python attributes on steroids (without side effects)! It makes it extremely easy to create simple and fairly complex UI's with little or no UI code (save a declarative specification of what elements you want to see). Traits also provides two UI backends for wxPython and Qt4. Gael Varoquaux has written an excellent tutorial on the topic.
ETS features Envisage which is a Python-based framework for building extensible applications. The applications can be extended by using plugins. Think of it as an Eclipse-like application framework for Python.
Sitting atop this is Mayavi which features several interesting features. It provides powerful traits based 3D visualization that is at accessible as an application, a pure library, a one-line (matlab-like) convenience wrapper usable like matplotlib's pylab from an IPython session (or pure Python scripts), envisage plugin and a pluggable widget for traits based apps. Mayavi also features some really neat features like automatic script recording.
ETS also features a powerful interactive 2D and flexible 2D plotting via Chaco.
Clearly, there is a lot of very interesting stuff in the few tools that I've listed above that have a significant impact to both folks in the scientific computing community and outside.
Incidentally, did you know that GvR was the keynote at SciPy07 and Alex Martelli was the keynote last year at SciPy08? And do you know who the keynote was at SciPy09? Peter Norvig! These conferences were held at Caltech.
From this, it should be evident to those who are in the community that really the Scientific computing community is not a fringe community in relation to the mainstream Python community.
Since Python is such a powerful programming toolkit for scientists and engineers, we wish to spread the adoption of Python in the college curriculum and thereby improve the quality of education at the college level.
