MRPerf is a simulation tool to model MapReduce application
performance on large clusters. The current simulation mimics Hadoop, an
open-source MapReduce implementation.
Get, Compile, and Use MRPerf
The latest version of MRPerf is 0.1,
published on 2009-7-14. To get
previous versions, go to archive.
To set up and use MRPerf, read instructions in README file, or
following the steps:
Compile
Get ns-2.33 from ns-2 website.
Patch ns-2.33 with ns.patch/ns-patch file in MRPerf.
Make sure you have libxml2.
Compile patched ns-2.33 source, and set up PATH environment to
ns executable.
If everything is done correctly, you should be able to run "ns"
on command line.
Run
Copy simulator source files from src/ to a new dir, say test/
Find configuration files from exp/, and copy the ones you want
to use to test/
go to test/, and run
make topo
make gen
make
NOTE: (4/10/12) The jobs.xml file is not in use in the current version, the correct files for specifying parameters etc. are hadoop.job.tcl and parameters.tcl. The parameters we used for Terasort are already in these files.
Guanying Wang, Ali R. Butt, Prashant Pandey, and Karan Gupta.
A
Simulation Approach to Evaluating Design Decisions in MapReduce Setups.
In Proceedings of the 17th Annual Meeting of the IEEE/ACM
International
Symposium on Modelling, Analysis and Simulation of Computer and
Telecommunication Systems (MASCOTS '09),
London, UK. Sep. 2009. Best Paper Award
Guanying Wang, Ali R. Butt, Henry Monti, and Karan Gupta.
Towards Synthesizing Realistic Workload Traces for Studying the Hadoop Ecosystem.
In Proceedings of the 19th Annual Meeting of the IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS),
Singapore. July 2011
Affiliation and People
Distributed Systems and Storage Lab (DSSL), Virginia Tech