I tried PHP during the private beta and had some problems; I don’t know if they fixed it since. I’m sure C++ would work. The MapReduce images seem to be pretty specific in their configuration, so it’s worth doing a little experiment with Perl before going to far in your implementation.

I haven’t found a lot of great links for it either, the best one is the Java documentation: http://hadoop.apache.org/core/docs/r0.15.2/api/org/apache/hadoop/mapred/lib/aggregate/package-summary.html.

When I get a chance, I’ll do another post and go through these different options.

“Amazon Elastic MapReduce supports Ruby and Python for streaming jobs.”

but AWS says:

“Amazon Elastic MapReduce allows you to implement data processing applications in many languages including Java, Perl, Ruby, Python, PHP, R, or C++. You can test these applications on different instance types and job flow sizes to pick the optimal performance settings for your specific case.”

Can I assume that I can write a Perl script as my mapper/reducer in addition to the two you mention??

Thanks,

-Rob

Couple of questions: You don’t seem to explain (unless I somehow missed it!) what the LongValueSum part of the map function output is for. I assume its for the built in aggregate function? Can you shed some light on what this is.

Also, I’ve been trying to find some more detail about this built in aggregate function on the net and seem to come up dry. (I chalk this up to my poor Googling skills - or the newness of the AWS mapreduce offering!) Do you have any good links you can post?

Thanks!

-Rob