Beginnings

I wasn’t exposed to computer programming (coding) until I was in college. That first exposure was in an upper division engineering class. In that class, I completed tasks such as root finding with Visual Basic for Applications in Microsoft Excel. I found this interesting, but it didn’t knock my socks off.

When I was in graduate school, I was a research assistant and asked to solve a research problem requiring combining precipitation observations from rain gage, radar, and satellite sources. This challenging task required more sophisticated analytical techniques than I possessed at the time. With the help of my office mates and fellow researchers, I was able to learn C and Matlab during this time to help with my research assignment.

The C was used to do the heavy lifting on calculations, and I employed Matlab to process and plot data from my research. I was able to learn a considerable amount about coding in these two years. I loved the reproducibility of products with these new techniques. I also found it exciting that I could write a routine, then use or build upon the same routine easily on my next application. This feeling that I was building a collection of analytical tools was very satisfying.

Professional life

Initiation

When I started at MBK Engineers in 2001 in water resources consulting work, I started using Microsoft Excel in a more sophisticated way for simulation modeling and processing of model results. This worked very well, although there were some things I disliked about Excel for this purpose. Namely,

  • It was much more difficult to develop customized functions for calculations than I was used to with C and Matlab;
  • Some functional representations were just not possible or required unlinked (i.e., undocumented) calculations;
  • Plotting was not ideal in terms of functionality, uniformity, and reproduction.

For these reasons, I started to search for other ways to solve problems that did not suffer from these limitations. Matlab use in the professional setting was impractical due to the high licensing costs for professionals. I still occasionally used C for things, as it was free. However, the user base for C professionally in my industry is virtually non-existent, so it didn’t work as a long-term solution either.

Party of one

Around this same time, I was exposed to the Python programming language. I immediately loved its simple yet effective syntax, which was a significant departure from C. At this time (2004), however, Python was relatively immature. It didn’t have very many users, and its web and graphics ecosystem wasn’t very impressive. In other words, it wasn’t good for much of what I was doing.

I taught myself object-oriented programming with Java in 2007. This was a great learning experience, and knowing Java opened some other doors for me. Most importantly, I was able to build software for the first time. I found Java’s graphical user interface (GUI) building capabilities very intriguing, and learning these allowed me to build tools that others could also use. I built a piece of Java software for an analysis I was doing with hydrologic statistics. For clients, I built two Java applications. One had a persistent database backend to store client data and a GUI the client could use for real-time regulatory compliance monitoring. The other was a multipurpose tool that performed a specific calculation that was challenging. Since this calculation was part of the regulatory process, many clients were able to benefit from its development. It also had a nice GUI interface and several modes of operation, so our non-programming engineering staff could use this to support our clients. This tool is still in use today.

Starting the team

I kept using Java through 2011 or so. In parallel, I also kept a close eye on my friend Python. Around this time, I discovered that the two could be combined, via Jython, to leverage the best of both platforms. This allowed the simple Python syntax to be used in combination with the powerful Java libraries, such as JFreeChart, which became my go-to library for production-grade plotting.

As I started to develop a work team of my own at MBK, these were the tools we used. We developed a Jython/Java set of tools for our daily workflow. As Python started to mature and more useful libraries were developed, we started to use it more and more… without the Java part.

Python gains traction through great libraries

The gateway libraries

The existence, effectiveness, and increasing popularity of two Python libraries contributed to our team’s functional wholesale switch to Python over the last several years.

These two libraries allowed our team to develop our plots and software in Python rather than in Java-based architectures. The more simplistic syntax of Python has allowed us to be more productive and allowed new members on the team with limited prior coding experience to hit the ground running with our team and quickly build their coding confidence.

Excel make that Pandas

The addition of the library Pandas to our Python toolbox has been tremendous. Pandas uses something called a dataframe, what an old-timer like me considers as Excel-like way of representing data, as its fundamental construct but allows for sophisticated calculations to be done on that dataframe.

Let the experts write that code

The advent of great Python packages in many areas have also significantly simplified our code base. Scipy, for example, has supplanted custom developed statistical libraries. I no longer need to write code to make basic calculations like mean and standard deviation. Leveraging open source libraries with huge user bases also greatly reduced the likelihood of making errors.

Goodbye programming books!

The rise of StackOverflow has made simultaneously rendered a whole section of my bookshelf obsolete and been a great resource for both troubleshooting specific coding issues and learning coding techniques.

The future

By most measures, Python is currently both one of the fastest growing and most commonly used programming languages among programmers. Its simple syntax is probably its most attractive feature.

So, I am quite pleased that our team is currently using it. What language will dominate the landscape in five years? Ten years? How will machine learning and artificial intelligence affect the programming landscape? Stay tuned!