RyanBanerjee

Here is some Python code I wrote to parse the American Economic Association, Job Openings for Economists (JOE) XML files. This is useful if you use OpenOffice or want to add the information to an SQL database.

While uploading my Job Market Paper to a website, I was just over the maximum file size. The file sizes of my figures were not that large (22KB) but I needed to compress them.

I tried using Pdftk a useful (FOSS) command line tool for merging, splitting and compressing pdf files but this could not shrink them. In the end I used pdf2ps and ps2pdf to shrink the files. Here is a script I wrote which compressed the files by about one third

#!/bin/bash
for f in *pdf
do
    pdf2ps $f $f.large
    ps2pdf $f.large compressed/$f
    rm $f.large

done

For some reason, doing this:

for f in *pdf; do pdf2ps $f > ps2pdf compressed/$f; done

did not compress the files, any ideas?

I recently found some great open source software called View Your Mind.

Here is a Mind Map of my Job Market Paper. I think it conveys the main message of my paper in a very elegant and easy to understand way.

Here is some Python code I wrote which solves the model in Phelan and Townsend (1991), "Computing Multi-Period Information-Constrained Optima", Review of Economic Studies. multiPeriod.py

The code makes use of the PuLP solver which makes writing linear programming problems a breeze

The code uses the HPfilter and Band Pass filter code available below

Little bits of other people's code has saved me from re-inventing the wheel many times. For example the Hodrick-Prescott filter code for Python written by Alan Issac saved me having to re-write this.

This is my first contribution to the Python community, the Baxter-King Band Pass filter. Please feel free to use it.

While trying to save myself from having to re-write the Band Pass filter code for Python, I also used f2py to compile Aubhik Khan's bkfilter.f90 code. F2py is a neat program which makes Fortran code useable from Python (a bit like a MATLAB MEX file). The source code required a little alteration, but was pretty straight forward. Here is my modified f90 code bkfilterpy.f90 and a test python program to check that f2py worked properly testf2pyprog.py

It took me a bit of fiddling to work out how to access the econ department's server. SSH connections would not let me use terminal commands. I found that I could use sftp and scp to get conections. Here is how I did it using the commandline (no need to install anything like F-Secure). For the sake of security I did not write down the port number (xx), ask Veer for it.

sftp -oPort=xx username@fileserver.econ.umd.edu

You will get an sftp> prompt. Type help to find out the commands you can use. For example to list the files in your directory, type ls.

Unfortunately, sftp can't deal with folders. To upload and download folders, you can use scp

scp -P xx fileToUpload.txt username-r@fileserver.econ.umd.edu:

This will upload the fileToUpload.txt file to your root directory on the fileserver. The -P is just giving the port on the fileserver, again ask Veer for this. If you want to download the public_html folder to your local computer, use:

scp -P xx -r username@fileserver.econ.umd.edu:public_html/ .

Dynamic programming is the work horse of modern macroeconomics. The mathematical theory behind dynamic programming can be quite hard going. John Rust provides a concise overview in his Palgrave article. However, using dynamic programming on a computer to solve macroeconomic models is surprisingly easy. This mini-course was designed for graduate students in macroeconomics and international finance with some idea of dynamic programming.

I think this material can be used for advanced undergraduates interested in modern macroeconomics. I think the computer code could easily be used by an undergraduate student to write a final year disseration. More ...

In autumn 2007, I took a computational economics course taught by John Rust. From this course, I finally understood what it really means to use the power of a computer. Pointing and clicking of the mouse is all very good to do an unfamiliar task once, but if the task is repetitive such as re-formatting data from one type to make it readable by another program, pointing and clicking can be really annoying.

So how does one get started? Well you need to write your own programs. So how does one write a program? which language should I use. All this stuff was a mystery to me last year. When I took John Rust's course, I was still using Microsoft Windows and did anything vaguely computational in MATLAB or Excel. Unfortunately, the Windows environment is not very condusive to automation. I tried using the Windows Command Prompt but this is not very helpful. You can download Cygwin which adds some extra utilities to the Windows command line but in the end I found it was not worth messing around with it.

Instead I recommend installing a version of Linux on your computer. Don't worry, you don't have to get rid of Windows, instead you can partition your hard-drive (one part for Windows and the other for Linux) and then at the start-up, choose which to run.

The easiest place to start using linux is Ubuntu or one of its sisters Kubuntu or Xubuntu. I installed Xubuntu which is supposedly better for older computers. My Xubuntu partition takes up about 15Gb of my hard-drive. However, I can access the Windows side of my hard drive from Xubuntu (but I can't do it the other way round). A good guide to installing ubuntu with screenshots can be found at here. If you are not sure Google is your friend, just Google your problem and see if that helps. That is how I learnt to use xubuntu.

Now you have ubuntu installed on your computer. What next? Well there are three key elements you need to get started in programming. (Don't try and install anything quite yet! This is a big difference between Windows and Linux). (1) you will need a Text Editor. The editor is like a general wordprocessor designed to write and edit computer programs. The two main editors are VIM and Emacs. I use VIM, which is probably easier to get started with. (2) A program language compileri/interpreter. This takes computer programs (e.g written in languages such as Fortran, Python or Perl) and converts it to code which the computer can understand. (3) The third element is already in ubuntu. It is the "terminal" or commandline. You can open it up by going to: Applications, Accessories, Terminal.

The first thing to do is get familiar with the Commandline. A good tutorial is linuxcommand.org. Now you know how to use cd or ls. Now you need to install your editor and learn to use it.

The next step is to install a text editor. I use VIM, so I will explain how to get started with VIM. Assuming you installed Ubuntu, you can install the full version of VIM by opening the terminal and typing:

sudo apt-get install vim

apt-get is a very useful package which installs packages on Ubuntu. Now you have VIM installed, you need to learn how to use it. Initially it is not very intuitive at all, but it comes with a very good tutorial. To start the tutorial, at the command prompt type:

vimtutor

Make sure when you finish the vimtutor to follow the instruction and copy the vimrc_example.vim file to your .vimrc file. This file will make VIM automatically highlight the syntax of your files, making reading and writing computer programs much easier.

The best and quickest way to learn programming is by seeing programs. Trying to write your own code from scratch is extremely difficult. Unfortunately, economists are often not very cooperative about sharing their computer code. So Hopefully I will get round to upload some of mine soon.

Python is a useful scripting language to do repetitive tasks. As an economist, one of my basic uses of Python is to convert data from one format to a different format which can be read by another program. Often data taken from a website is not in the format you need. You could use a spreadsheet package to do this but sometimes this might require quite a bit of pointing and clicking.

But Python is much more than just a scipting language. With the Python package SciPy you won't ever use MATLAB again. To manage large amounts of data SQLalchemy is a fantastic way to create and access SQL databases from within Python. And don't forget R for anything econometrics. RPy2 is a great way to access the statistical ability of R from Python