How A Consumer Can Measure Elasticity for Cloud Platforms

1. Low-cost
2. Availability

   Annotations:

   • Disponibilidade = a proporção de tempo que um sistema está em uma condição de funcionamento http://en.wikipedia.org/wiki/Availability
3. Elasticity

   Annotations:

   • NIST definition Capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out, and rapidly released to quickly scale in. To the consumer, the capabilities avaliable for provisioning often appear to be unlimited and can be purchased in any quantity at any time
   1. Pay only for what need
   2. Automatic provisioning
   3. Quick scale
   4. Unlimited
   5. Any quantity
   6. Any Time
   7. Pay-as-you-grow
   8. Elasticity <> Availability
4. How elastic is each system?
   1. Benchmark
      1. Measures
         1. Existing today
            1. Not explicit measurement of elasticity
         2. Need to develop
            1. Appropriate measures
               1. QoS requirements
               2. Contributions
                  1. Framework to measure Elasticity
                  2. Case studies
                  3. Insights that impacts Elasticit
                  4. Understand Elasticity Behavior
               3. Compare offerings
                  1. Benchmark
                  2. According to need

Annotations:

• M. Armbrust, A. Fox, R. Griffith, A. Joeph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica amd M. Zaharia A view of Cloud Computing. Communications of the ACM 2010

Annotations:

• David Chiu - Crosswords, Vol. 16, No. 3. (2010), pp. (3-4) Ricky Ho - http://horicky.blogspot.com/2009/07/between-elasticity-and-scalability.html

Annotations:

• J. Weinman - Time is Money: The Value of "On-Demand" www.joeweinman.com/Resources/Joe_Weinman_Time_Is_Money.pdf Jan/2011

1. Demand curve (D)
   1. Time (t)
   2. Resource (R)
2. Situations
   1. Perfect Elasticity = R(t) = D(t)
   2. Overprovisioning = R(t) > D(t)
   3. Underprovisioning = D(t) < R(t)

1. Real data Workload
2. Include penalties (unsatisfactory performance)
3. QoS based
4. Allow SLA
5. allocated resources x charged resources
6. Unified metric to sumarize

1. Stantchev et al. - generic benchmark to evaluate nonfunctional properties (cost-benefit)

   Annotations:

   • V. Stantchev - Performance evaluation of cloud computing offerings.  IEEE AdvComp 2009
2. Dejun etl al. and Schad et al. - evaluate performance characteristics of cloud infrastructure (without variation during provisioning)

   Annotations:

   • J. Dejun, G. Pierre, and C. Chi - EC2 performance analysis for resource provisioning of service-oriented ICSOC Workshops 2009 J. Schad, J. Dittirich, and J.-A. Quiané-Ruiz - Runtime measurements in the cloud: Observing, analyzing, and reducing variance PVLDB 2010
3. HP Labs - measurements of quality features (cloud platforms perspective)

   Annotations:

   • C. Bash, T. Cader, Y.Chen, D.Gmach, R.Kaufman, D. Milojicic, A. Shah, and P. Sharma HPL-20110148 Cloud Sustainability of Dashboard Dynamically 2011
4. Srinivasan et al. and Huang et all. - compare data center migration techniques

   Annotations:

   • K. Sriniviasan, S. Yuuw and T. Adelmeyer - Dynamic VM migration: assesing its risks &amp; rewards using a benchmark ICPE 20111D. Huang, D. Ye, Q. He, J. Chen, and K. Ye. - Virt-LM: a benchmark for live migration of virtual machine ICPE 2011
5. Ygitbasi et al. - evaluate performance overheads with scalling lattency of VM instances

   Annotations:

   • N. Yigitbasi, A. Iosup, D. Epdema, and S. Ostermann - C-meter: A framework for performance analysis of computing clouds CCGrid 2009
6. Li et al. - propose CloudCmp: user perceived performance and cost effectiveness with fine granularity

   Annotations:

   • A. Li, X. Yang, S. Kandula, and M. Zhang - CloudCmp: comparing public cloud providers. ICM 2010
7. YCSB - evaluate performance of cloud databases (workloads and elasticity; do not evaluate de-provisioning and resource granularity aspects; not capture financial implications as well as traditional performance)

   Annotations:

   • B. Cooper, A. Silberstein, E. Tam, E. Ramakrishnan, and R. Sears - Benchmarking cloud serving systems with YCSB SoCC 2010
8. Kossmann - compare with a set of performance and cost metrics to compare throughput, performance/cost ratio and cost predictability (omit the speed of responding to change; not consider workload shrink and grow)

   Annotations:

   • D. Kossmann, T. Kraska, and S. Loesing - An evaluation of alternative architectures for transaction processing in the cloud SIGMOD 2010

1. Evaluate Elasticity from user perspective
2. Impact of Imperfection of Elasticity based on consumers' business situation.
3. Evaluate perceived performance and cost effectiveness with coarse granularity

   Annotations:

   • Could impact the metric's expression.

1. Workload Penalties
   1. overprovisioning faults
   2. underprovisioning faults

1. SLA objectives
   1. F. Nah Study and Weinman

      Annotations:

      • F. Nah. A suty on tolerale waiting time: how long are web users willing to wait?  Behaviour &amp; Information Technology 2004 J. Dejun, G. Pierre, and C. Chi. EC2 performance analysis for resource provisioning of service-oriented applications. ICSOC 2009
2. Metrics
   1. EC2 - CPU capacity
3. Workloads
   1. Sinusoidal
   2. Sinusouidal with Plateau
   3. Exponentially Bursting
   4. Linearly Growing
   5. Random
4. Penalties
   1. Latency
   2. Aviability
5. Workload characteristics
   1. Periodicity
   2. Growth and decay rate
   3. Randomness