BenchLab: Realistic Internet Scale Benchmarking with Real Web Browsers
Project OverviewThis project seeks to provide the research community with more realistic benchmarks by providing real applications with real production workloads. BenchLab provides a framework to replay these workloads using real web browsers that can be deployed anywhere on the Internet. Our latest developments also provide support for mobile devices with mBenchLab. We are exploring the impact of Internet scale workloads on Web Applications including the effects of WAN latencies, load balancers, Content Delivery Networks (CDN), replication and high availability (HA) mechanisms.
BenchLab has a new dedicated web site at http://benchlab.cs.umass.edu.
BenchLab provides several components:
- Benchmark Virtual Appliances: Virtual Appliances (virtual machines with pre-installed and pre-configured software stack) of benchmarks or real applications with their database payload and workload traces
- Real Web Browser Replay: Software that allows to replay a workload trace through a real Web browser (Firefox, Internet Explorer, Chrome, Android)
- BenchLab Dashboard: A Java WebApp that acts as a control center to start and manage experiments as well as a benchmark repository for storing results of experiments
- Trace recorder: Collects live traces from an Apache server in HTTP Archive format to be replayed realistically by Web browsers
March 2015: BenchLab 3.0 release
- A new website at http://benchlab.cs.umass.edu has been launched with new releases of all the BenchLab software components
- An updated documentation as well as Virtual Machine images (VMDK or EC2 AMIs) are available on our SourceForge page.
- Both a paper and a demo have been accepted at the ACM Multimedia Systems conference in Portland OR taking place in March 2015. The papers have been added to the publication section of this page.
- New features to manipulate video players (pause, seek, change quality, skip ads...) have been added to Video BenchLab and will be soon released on SourceForge.
- Check link for software releases at the bottom of this page
- mBenchLab has been improved with a more friendly user interface with configuration screen as well as result visualization directly on the device.
- Emmanuel Cecchet
- Prashant Shenoy
- Patrick Pegus II
- Sneha Shankar Narayan
- Nikita Mehra : firstname.lastname@example.org
- Puja Mishra : email@example.com
- Seema Guggari : firstname.lastname@example.org
- Veena Udayabhanu
- Timothy Wood
- Xin He
- Robert Sims
- Anil Kumar Venkatesh
- Siddharth Gupta
January 2015: New video BenchLab paper
February 2014: New Android version available
Video BenchLab: An Open Platform for Realistic Benchmarking of Streaming Media WorkloadsPatrick Pagus II, Emmanuel Cecchet, and Prashant Shenoy
Proceedings of the ACM Multimedia Systems Conference (MMSys 2015), Portland, OR, March 2015. pdf
Video BenchLab Demo: An Open Platform for Video Realistic Streaming BenchmarkingPatrick Pagus II, Emmanuel Cecchet, and Prashant Shenoy
Demo paper - Proceedings of the ACM Multimedia Systems Conference (MMSys 2015), Portland, OR, March 2015. pdf
mBenchLab: Measuring QoE of Web Applications using mobile devices (Best Paper Award)Emmanuel Cecchet, Robert Sims, Xin He and Prashant Shenoy
Proceedings of ACM/IEE International Symposium on Quality of Service (IWQoS 2013), Montreal, Canada, June 3-4 2013
Paper (pdf) Slides (.pptx)
BenchLab: An Open Testbed for Realistic Benchmarking of Web ApplicationsEmmanuel Cecchet, Veena Udayabhanu, Timothy Wood and Prashant Shenoy.
Proceedings of 2nd USENIX Conference on Web Application Development (WebApps '11), June 15-16, 2011, Portland, OR
BenchLab: Benchmarking with Real Web Applications and Web BrowsersEmmanuel Cecchet, Veena Udayabhanu, Timothy Wood, Prashant Shenoy, Fabien Mottet, Vivien Quema and Guillaume Pierre.
Eurosys 2011, Poster and demo session, April 10-13, 2011, Salzburg, Austria
You can find the latest software releases from our SourceForge page and choose the nearest mirror at https://sourceforge.net/projects/benchlab/