BenchLab: Realistic Internet Scale Benchmarking with Real Web Browsers

 

Project Overview

This project seeks to provide the research community with more realistic benchmarks by providing real applications with real production workloads. BenchLab provides a framework to replay these workloads using real web browsers that can be deployed anywhere on the Internet. Our latest developments also provide support for mobile devices with mBenchLab. We are exploring the impact of Internet scale workloads on Web Applications including the effects of WAN latencies, load balancers, Content Delivery Networks (CDN), replication and high availability (HA) mechanisms.

BenchLab provides several components:

  • Benchmark Virtual Appliances: Virtual Appliances (virtual machines with pre-installed and pre-configured software stack) of benchmarks or real applications with their database payload and workload traces
  • Real Web Browser Replay: Software that allows to replay a workload trace through a real Web browser (Firefox, Internet Explorer, Chrome, Android)
  • BenchLab Dashboard: A Java WebApp that acts as a control center to start and manage experiments as well as a benchmark repository for storing results of experiments
  • Trace recorder: Collects live traces from an Apache server in HTTP Archive format to be replayed realistically by Web browsers


Latest News

    September 2014: BenchLab 2.2 release
    • Video support has been added to BenchLab with a new MediaDrop server virtual appliance and new extensions in the BenchLab Client Runtime for video specific metrics.
    • New enhanced analytics tools for easier comparison between experiments and visualization of all possible metrics and HAR files
    • Check link for software releases at the bottom of this page

    February 2014: New Android version available
    • mBenchLab has been improved with a more friendly user interface with configuration screen as well as result visualization directly on the device.


    Participants


    Former Participants

    • Veena Udayabhanu
    • Timothy Wood
    • Xin He
    • Robert Sims
    • Anil Kumar Venkatesh
    • Siddharth Gupta


    Collaborations

 

Publications

 
  • mBenchLab: Measuring QoE of Web Applications using mobile devices (Best Paper Award)

    Emmanuel Cecchet, Robert Sims, Xin He and Prashant Shenoy
    Proceedings of ACM/IEE International Symposium on Quality of Service (IWQoS 2013), Montreal, Canada, June 3-4 2013
    Paper (pdf) Slides (.pptx)
  • BenchLab: An Open Testbed for Realistic Benchmarking of Web Applications

    Emmanuel Cecchet, Veena Udayabhanu, Timothy Wood and Prashant Shenoy.
    Proceedings of 2nd USENIX Conference on Web Application Development (WebApps '11), June 15-16, 2011, Portland, OR
    pdf
  • BenchLab: Benchmarking with Real Web Applications and Web Browsers

    Emmanuel Cecchet, Veena Udayabhanu, Timothy Wood, Prashant Shenoy, Fabien Mottet, Vivien Quema and Guillaume Pierre.
    Eurosys 2011, Poster and demo session, April 10-13, 2011, Salzburg, Austria
    pdf
 


Software Releases

 

You can find the latest software releases from our SourceForge page and choose the nearest mirror at https://sourceforge.net/projects/benchlab/