PowerPoint-Präsentation - ITU

PowerPoint-Präsentation - ITU

Workshop on Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks (Maputo, Mozambique, 14-16 April 2014) Perceptual Quality Evaluation Of OTT Streaming Video Services Joachim Pomy [email protected] OPTICOM, Germany Perceptual Quality Evaluation Of OTT Streaming Video Services Dipl.-Ing. Michael Keyhl, MBA Founder & CEO, OPTICOM, Germany Session > OTT Technologies and Services < Broadcast Engineering Conference | NAB Show 2014, Monday April 7 Who We Are

Founded 1995 in Erlangen, Germany Spin-Off from the Fraunhofer-Institute IIS (Home of MP3) Originators of Perceptual Audio Quality Measurement (1988) Co-Authors of Six Major International Standards Leading Source of Voice and Video Quality Test Products and OEM Technology for Mobile and IP-based Network Testing > 100 Licensed OEM Customers > 40.000 Licensed PESQ Products NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 3 How Do You Measure Video How Do You Measure Video Quality? Quality.. ...to benchmark Multi-Screen QoE of ABR VoD OTT

Services as perceived by subscribers? NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 4 How Do You Measure Video Quality.. ...to benchmark Multi-Screen QoE of ABR VoD OTT Services as perceived by subscribers? Conclusion 1: Clear Need for a New Streaming Video Quality Measurement Technique Conclusion 2: Newly Interpret Video Quality in the Context of Multi-Screen Use Scenarios NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 5 Overview 1. 2. 3. 4. 5. 6. ABR and Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de

6 Overview 1. 2. 3. 4. 5. 6. ABR and Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 7

Testing Typologies Full Reference (FR) No-Reference (NR) Principle: Differential Analysis comparing Degraded with the Original Source Video Access to Reference/Studio Source Pixel-based, Frame-by-Frame Comparison e.g. OPTICOMs PEVQ/ITU-T J.247 + Highest Accuracy and Repeatability + Standardized Processing Intensive Principle: Single-ended Analysis, NO Information on the Original Source Video Pixel-based, or NR/IP Transport Stream Analysis, only,

no Payload decoding Various Proprietary Metrics Low Accuracy Estimates only Failed in Standardization + Lightweight, easy to apply NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 8 Video Quality Standard s develope d by VQEG and ITU based on

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 9 ABR Video Streaming Techniques (based on Jan Ozer: Video Compression for Flash, Apple Devices and HTML5 [2011]) NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 10 Example: YouTube Upload Upload: V_Res 1920x1080, V_Fps 24, V_BitRate 8Mbit/s

Protocol PD PD PD PD PD PD PD ABR ABR ABR ABR ABR ABR ABR V_Codec MPEG4 MPEG4 FLV VP8 H.264

H.264 H.264 H.264 H.264 H.264 H.264 H.264 V_BitRate 60kbit/s 200kbit/s 300kbit/s 600kbit/s 500kbit/s 2Mbit/s 100kbit/s 250kbit/s 400kbit/s 800kbit/s 1.6Mbit/s 3Mbit/s

V_Res 176x144 320x180 426x240 640x360 640x360 1280x720 256x144 426x240 640x360 854x480 1280x720 1920x1080 V_Fps 12 24 24 24 24 24 15

24 24 24 24 24 A_Codec AAC AAC MP3 OGG OGG AAC AAC AAC - A_Bitrate 24kbit/s 32kbit/s 64kbit/s 96kbit/s

110kbit/s 96kbit/s 192kbit/s 128kbit/s - NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 Reference/Source PD or ABR Protocols Format 3GP 3GP FLV WEBM WEBM MP4 MP4 MP4 MP4 MP4

MP4 MP4 MP4 MP4 PEVQ-S Various Codecs at Various Bitrates Compare Various Frame Sizes Compare Different Fps

Handle Common Bitstream Formats 2014 Michael Keyhl, www.opticom.de 11 Only PEVQ was validated for QCIF, CIF, VGA, SD, and HD based on BT.500/P.910 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 PEVQ PEVQ PEVQ

2014 Michael Keyhl, www.opticom.de 12 Validation by Subjective Testing ITU-R BT.500: Originally CRT TV-Quality Testing (SD) ITU-T P.910: Multimedia (QCIF, CIF, VGA) and IPTV (HD 720/1080) Mostly ACR Tests Test Sequences 8 15 sec 5-Point Opinion Scale Averaging over a Large Enough Test Population MOS Conclusion 4: Need for New Subjective Testing Snapshots of 10 s Video are Inadequate to Assess Re-Buffering and Long-term Behavior of Streaming Video

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 13 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 14 Stakeholders involved in a QoE Rivalry NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 15 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry

Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 16 Defining a 4layered OTT Quality Model NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 17

Stakeholders Different Quality Views Stakeholders for OTT QoE and related Quality Layers NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 18 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry

Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 19 OTT VQM Requirements VQM Should be related to the Content Quality as a Reference Accurately score encoding and transcoding Artifacts = Media Stream Quality Measure and compare the Picture Quality for different Frame Sizes and Frame Rates = Media Stream/Transmission Quality Continuously track the different Bit Rates and evaluate how smooth the Player is able to interact with the Server in a Congested Network = Transmission Quality Take into account Player and Device Characteristics as well

as Viewing Environment = Presentation Quality 20 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014

2014 Michael Keyhl, www.opticom.de 21 NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 22 Overview 1. 2. 3. 4. 5. 6. Existing Video Metrics and Standards Stakeholders in the OTT QoE Rivalry Defining an >OTT Quality Model< Designing a Video Quality Measure for OTT

Introduction to PEVQ-S Conclusions NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 23 Conclusions (1/2) Clear Need for a Streaming Video Quality Measurement Technique which accurately evaluates Subscriber QoE We must reformulate QoE Concepts and newly interpret Video Quality in the Context of Multi-Screen Use Scenarios Currently, no Standardized Subjective and Objective VQM Standard exists for ABR Video Streaming PEVQ-S is proposed to leapfrog this Shortcoming, based on advancing existing Standards, while maintaining maximum Backward-Compatibility and Validated Accuracy PEVQ-S is suited to evaluate all 4 OTT Quality Layers:

Content, Media Stream, Transmission and Presentation Quality NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 24 Conclusions (2/2) PEVQ-S allows for Analysis of Common ABR Protocols and Formats, Various Codecs at Various Bitrates PEVQ-S can analyze Video at Different Frame Sizes and Fps An Advanced Hybrid Architecture: (a) the PEVQ-S Media Stream Quality Database and (b) the PEVQ-S Client Probe combines the high Accuracy of a FR Measure with a Light-weight Realtime Client Implementation, even on Android Smartphones Measures Current Video Quality (CVQ) MOS and Headroom for Optimization for the Best Video Quality Available (BVQA) MOS PEVQ-S is made available as an OEM-Technology to leading OTT, Middleware and T&M Vendors and soon to be found in Products

NAB Show 2014, Broadcast Engineering Conference, 07 April 2014 2014 Michael Keyhl, www.opticom.de 25 Any questions ? Joachim Pomy [email protected]

Recently Viewed Presentations

  • Hormonal Control of Flowering - Gavilan College

    Hormonal Control of Flowering - Gavilan College

    Chapter 28 External factors and Plant growth Nastic Movement Nastic Movements- plants movement that occur in response to a stimulus independent of position of stimulus ( leaves light and dark cycle) Phototropism- response to light- caused by elongation- under the...
  • Heat Recovery Technologies for Combined Heat and Power in Oil ...

    Heat Recovery Technologies for Combined Heat and Power in Oil ...

    The 5.5 megawatt system converts waste heat from the compressor into electricity using an Organic Rankine Cycle based power plant. ORMAT Technologies, Inc. owns and operates the plant, purchasing the waste heat from NBP and selling the electricity to nearby...
  • Taiwan I&amp;C Standard Committee Liaison Report

    Taiwan I&C Standard Committee Liaison Report

    Doc #5740 -Test Method of Electrochemical Corrosion for PV Module. Doc #5979 -Specification of indoor lighting simulator requirements for emerging PV. Doc #5980 -Test method of current-voltage (I-V) measurement at indoor lighting for DSC and OPV. Forming . New ....
  • Part I

    Part I

    Arial MS PGothic Arial Unicode MS template 1_template 2_template 3_template Slide 1 International Context National Context Caste based Discrimination and Untouchability (Offence and Punishment) Act 2011 Major Provisions of the Act What is untouchability and CBD? (Section 4) What is...
  • The HWNZ work plan 2013/2014 - GP CME

    The HWNZ work plan 2013/2014 - GP CME

    Core HWNZ Strategies 2013/2014. Finally, some deliverables are direct and essential responses to the extraordinary change in medical workforce dynamics that have occurred over the last three years - and one is the necessary ongoing work to ensure that the...
  • Der Philhellenismus in der westeuropäischen Literatur von ...

    Der Philhellenismus in der westeuropäischen Literatur von ...

    Melodramma serio, musique de Giovanni Pacini (Milan 1827) Felice Romani: La straniera, musique de Vincenzo Bellini (Milan 1829) La Revue des romans, 1831: Aujourd'hui presque impossible à lire sans s'endormir, ou tout au moins sans bâiller à se rompre la...
  • THE SONNET - Worth County Schools

    THE SONNET - Worth County Schools

    The English word "sonnet" comes from the Italian word "sonetto," meaning "little song." Some early sonnets were set to music, with accompaniment provided by a lute. The Italian poet Petrarch (1304-1374), a Roman Catholic priest, popularized the sonnet more than...
  • Careers in Geosciences: Employment Trends, Marketable Skills, and

    Careers in Geosciences: Employment Trends, Marketable Skills, and

    Get your HAZWoPER 40-hour certification. Get a first aid and CPR certification. Check your social media accounts for obvious stupidity. Before you apply, become familiar with the company. Identify key personnel. Learn about specific projects of interest. Check their social...