Admin - WPI

Admin - WPI

Introduction IMGD 2905 What is data analysis for game development? What is data analysis for game development? Using game data to inform the game development process Where does this data come from? Users playing game Quantitative (instrumented) Qualitative (subjective evaluation) (But often lots more of the former!) What can game analysis do for game development? What can game analysis do for game

development? Improve level design e.g., see where players are getting stuck Focus development on critical content e.g., see what game modes or characters are not used Balance gameplay e.g., tune parameters for more competitive and fun combat Broaden appeal e.g., hear if content/story is engaging or repulsing Note: game data often informs players, too Analytics not dissimilar Why is data analysis for game development needed? Why is data analysis for game development needed? Challenge

Games gotten larger and more complex Number of reachable states, characters Game balance harder to achieve Need for metrics to make sense of player behavior has increased Opportunity New technologies enable aggregation, access and analysis IMGD 2905 Doing Data Analysis for Game Development Data analysis pipeline get data from games, through analysis, to stakeholders Summary statistics central tendencies of data Visualization of data how to display analysis, illustrate messages Statistical tests quantitatively determine relationships (e.g.,

correlation) Probability needed as foundation Regression model relationships More advanced topics (e.g., ML, Data management ) For this class: Described in lecture Read about in book Applied in projects Foundations for Data Analysis for Game Development @ WPI Statistics classes MA 2610 Applied Statistics for Life Sciences MA 2611 Applied Statistics I MA 2612 Applied Statistics II

Probability classes MA 2621 Probability for Applications Data Science minor Note other Stats and Probability classes are primarily geared for Math majors MA, CS, BUS DS 3001 Foundations of Data Science Data Mining CS 4445 Data Mining and Knowledge Discovery in Databases

Other CS 1004 Introduction to Programming for Non-Majors CS 3431 Database Systems I Outline Overview (done) Game Analytics Pipeline (next) Game Data Analysis Examples Sources of Game Data Quantitative (Objective) Internal Testing Developers QA External Testing

Qualitative (Subjective) Surveys Reviews Online communities Post mortems Usability testing Beta tests Long-term play data How to get from data to dissemination? Game analytics pipeline

Game Analytics Pipeline Game Extracted Data Raw Data Charts and Tables Dissemination Presentation Exploratory Graphs/Stats Report Analysis

Statistical Tests Game Analytics Pipeline - Example Analysis fifa-18.csv Dissemination Project 1! Game Analytics Pipeline - Example Analysis Dissemination Project 4!

Game Analytics Tools Games breadth of experience with games, specific experience with game to be analyzed Tools import, clean, filter, format data so can analyze Statistics measures of central tendency, measures of spread, statistical tests Probability rules, distributions Data Visualization bar chart, scatter plot, histogram, error bars Technical Writing and Presentation white paper, technical talk; audience is peer group, developers, boss Outline Overview (done) Game Analytics Pipeline (done) Game Data Analysis Examples (next)

Example: Project Gotham Racing 4 K. Hullett, N. Nagappan, E. Schuh, and J. Hopson. Data Analytics for Game Development, International Conference on Software Engineering (ICSE), May, 2011, Waikiki, Honolulu, HI, USA http://dl.acm.org/citation.cfm?id=1985952 Publisher Microsoft 2007 134 vehicles, 9 locations, 10 game modes Analyzed data (Authors worked at Microsoft) 3.1 million log entries, 1000s of users Project Gotham Racing 4: Results

Game Mode Races OFFLINE_CAREER 1479586 PGR_ARCADE 566705 NETWORK_PLAY 584201 SINGLE_PLAYER_PLAY 185415 . NET_TOURNY_ELIM 2713 % Total 47.63% 18.24% 18.81% 5.97%

Group STREET_RACE NET_STREET_RACE ELIMINATION HOTLAP TESTTRACK_TIME CAT_N_MOUSE_FREE CAT_N_MOUSE % Total 25.60% 17.50% 6.95% 6.31% Races

795334 543491 216042 195949 7484 3989 53 0.09% 0.24% 0.13% 0.00% Thoughts? What are some main messages?

Project Gotham Racing 4: Results Game Mode Races OFFLINE_CAREER 1479586 PGR_ARCADE 566705 NETWORK_PLAY 584201 SINGLE_PLAYER_PLAY 185415 . NET_TOURNY_ELIM 2713 % Total 47.63%

18.24% 18.81% 5.97% Mode 0.09% Events Group STREET_RACE NET_STREET_RACE ELIMINATION HOTLAP TESTTRACK_TIME CAT_N_MOUSE_FREE

CAT_N_MOUSE % Total 25.60% 17.50% 6.95% 6.31% Races 795334 543491 216042 195949 7484 3989 53 0.24%

0.13% 0.00% Offline career dominates Network tournament hardly used Street race and network street race dominate Cat and mouse never used Vehicles (not shown) 1/3 used in less than 0.1% of races Project Gotham Racing 4:

Conclusion Content underused - 30-40% of content in less than 1% of races Use to shift emphases for DLC, next version Asset creation costs significant, so even 25% reduction noticeable Other (not shown) Encouraging new players to play career mode Increasing likelihood of continuing play Encouraging new players to stay with F Class longer Rather than move to more difficult to control A Class Example: Halo 3 B. Phillips. Peering into the Black Box of Player Behavior: The Player Experience

Panel at Microsoft Game Studios, Game Developers Conference (GDC), 2010. http://www.gdcvault.com/play/1012387/P eering-into-the-Black-Box Publisher Microsoft 2007 Achievements: single player missions, challenges such as finding skulls, multiplayer accomplishments Analyzed data (Author worked at Microsoft) 18,0000 players Halo 3: Results Thoughts? What are some main messages?

Halo 3: Results 73% of players completed campaign Can compare to other Xbox games Took 26 days to accomplish Double that time for all original content DLC provides users up to 2 years of content Good Descriptive Example

Example: League of Legends Mark Claypool, Jonathan Decelle, Gabriel Hall, and Lindsay O'Donnell. Surrender at 20? Matchmaking in League of Legends, In Proceedings of the IEEE Games, Entertainment, Media Conference (GEM), Toronto, Canada, October 2015. Online at: http://www.cs.wpi.edu/~claypool/papers/lol-matchmaking/ ??? Publisher Riot Games 2009 User study (52 players) Play LoL in controlled environment Record objective data Fun Rank: ~5 Tiers, 5 divisions each 25

Sweet spot (e.g., player rank and game stats) Provide survey for subjective data (e.g., match balance and enjoyment) Too hard! Just right! Game Balance Too easy! League of Legends: Results Main

messages? Main messages? Objective Subjective Main messages? Main messages? League of Legends: Results Most teams are balanced But about 10% more than 3 from mean Win? Game is balanced Lose? Game is

imbalanced Objective Subjective Most games evenly matched But about 5% difference of 2 from mean Win? Game is fun (70%), never not fun Lose? Game is almost never fun (90%)

Fun League of Legends: Results Sweet spot Game Balance Fun Sweet spot? Game Balance Imbalance in players favor the most fun! Matchmaking systems may want to consider - e.g., balance not so important, as long as player not always on imbalanced side 27

Summary Data analysis for games increasingly important Has potential to improve game development Knowledge and skills required Scripting Statistics Data analysis Writing and presentation Lets get to it, already! -- Tracer (Overwatch)

Recently Viewed Presentations

  • The National Institute on Disability and Rehabilitation ...

    The National Institute on Disability and Rehabilitation ...

    The National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR) Brian Bard, Program Specialist and SBIR Program Coordinator. Administration for Community Living (ACL)U.S. Department of Health and Human Services
  • Nuclear Chemistry

    Nuclear Chemistry

    Nuclear Chemistry Chemistry
  • Project:PRINCIPLES OF FAME: FAMOUS PEOPLE OF MY COUNTRY THAT ...

    Project:PRINCIPLES OF FAME: FAMOUS PEOPLE OF MY COUNTRY THAT ...

    Project:PRINCIPLES OF FAME: FAMOUS PEOPLE OF MY COUNTRY THAT MADE A POSITIVE CONTRIBUTION TO THE WORLD. Founding Schools: 7th Primary School of Agii Anargiri, Greece (Ms Thalia Hadzigiannoglou) Scoala cu clasele I-VIII Nr.4 Nr.5 Gheorghe Magheru, Caracal, Romania (Mr Dan...
  • Sound Reasoning 3 - Speech-Language Therapy

    Sound Reasoning 3 - Speech-Language Therapy

    We "prefer" to articulate words with a rise and fall in sonority; p ... This rise - fall tendency is called the Sonority Sequencing Principle (SSP) [ ] and [ ] are more 'natural' for us to say than[ ]...
  • #cciolip The role of the CNIO Fiona Stephens,

    #cciolip The role of the CNIO Fiona Stephens,

    The role of the CNIO Fiona Stephens, Jon Hoeksma. #cciolip. EHI CCIO Leaders Network. Celebrating Leadership in Nursing Informatics. 4th July 2012. Jon Hoeksma, Editor EHI. CCIO Leaders Network aims "To support and develop current and future clinical information leaders...
  • The Insecurity of Home Digital Voice Assistants Vulnerabilities,

    The Insecurity of Home Digital Voice Assistants Vulnerabilities,

    Dynamic baseline: the baseline it the average over the recent 1 hour. The outlier is detected based on its variation from the baseline. It makes our VSButton more adaptive in the home use scenario. We don't recollaborate our VSButton even...
  • QUIT CHAPTER 8 African Civilizations 1500 B.C.A.D. 500

    QUIT CHAPTER 8 African Civilizations 1500 B.C.A.D. 500

    Migration continues to shape the modern world. Overview Assessment • Bantu-speaking peoples MAIN IDEA WHY IT MATTERS NOW TERMS & NAMES 1. Look at the graphic to help organize your thoughts. List five reasons why people migrate. Give an example...
  • STAAR Preparation 2017

    STAAR Preparation 2017

    Chris paul was shoved by a player on the Minnesota timberwolvesGorguiDieng. Gerald green then shoved gorguidieng to stand up for his teammate chrispaul. green knew he would get punished but this is didnt effect his decision. chrispauls comment on the...