ACM SIGCSE 2003: Multimedia Construction Projects

ACM SIGCSE 2003: Multimedia Construction Projects

Introducing Computer Programming with Multimedia Barb Ericson Institute for Computing Education Georgia Institute of Technology [email protected] Computer Science Crisis 40% drop in majors since 2001 Nationwide Up to 60% drop in some places Percentage of women is down to about

12% From a high of 30% in the 1980s Never attracted many African Americans, Hispanics, Native Americans Why the Decline? Dot com crash in 2001 Loss of jobs AP Exam switched from C++ to Java Many schools quit teaching AP Computer Science

Visual Basic changed In .net to be object-oriented Perceptions of Computer Science boring, antisocial, not creative, and just programming Looming Shortage Only a few jobs have gone overseas About 2-3% We have recovered from the dot com

crash There are now more jobs in this area than in 2001 Computing jobs are some of the fastest growing US Dept of Labor predicts 45.93% growth Reversing the Trend We need to get more students interested in computer science Not drive them away

We need to appeal to a diverse group of students Not just geeks All students need to have some understanding of computer science Our society relies on technology Our Solution Use multimedia projects to introduce computing concepts

Pictures, sounds, movies, text Use open-ended projects to encourage creativity Create an image collage, create a sound collage, create a movie Make computing social Encourage students to show off their work Choice of Languages Python

Used by many companies: IL&M, Google, Nextel, etc. Easier syntax than many languages Can use to teach functional, procedural, or objectoriented programming Used as a scripting or extension language for many applications Growing in popularity Java Popular language used for AP CS Digitizing Pictures We digitize pictures into

lots of little dots Enough dots and it looks like a continuous whole to our eye Each picture element is referred to as a pixel Each pixel knows its color It also knows where it is in its picture Encoding Color: RGB In RGB, each color has

three component colors: Red, Green, and Blue In most computer-based models of RGB, a single byte (8 bits) is used for each So a complete RGB color is 24 bits, 8 bits of each 0,0,0 is black 255,255,255 is white 255,0,0 is red

Make and Show a Picture Use pickAFile function To select a picture file (ends in .jpg) Use makePicture(file) to create the picture Use show(picture) to display the picture Use MediaTools Click on MediaTools

Select the picture to explore Move the cursor while holding down the left mouse button To see x,y,r,g,b Click on zoom To zoom in or out Creating a Python Function Define

a function Load the function Try the function Rules for Python Functions Use the keyword def to define a function

def pickAndShow(): myFile = pickAFile() Followed by the myPicture = function name makePicture(myFile) Use parentheses after show(myPicture) the name Name parameters End with a colon

Indent the body of the function Using a Loop def decreaseRed(picture): for p in getPixels(picture): value = getRed(p) setRed(p,value * 0.5) More Picture Functions def greyscale(picture): for p in getPixels(picture):

redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p, makeColor(luminance,luminance,luminance)) def negate(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255blue) setColor(px,negColor)

How Sound Works: Sounds are waves of air pressure Sound comes in cycles The frequency of a wave is the number of cycles per second (cps), or Hertz The amplitude is the maximum height of the wave

Volume and Pitch Our perception of volume is related (logarithmically) to changes in amplitude If the amplitude doubles, its about a 3 decibel (dB) change. Our perception of pitch is related (logarithmically) to changes in frequency Higher frequencies are perceived as higher pitches We can hear between 5 Hz and 20,000 Hz (20 kHz) A above middle C is 440 Hz

Digitizing Sound We can estimate the sound curve Analog-to-digital conversion (ADC) will give us the amplitude at an instant as a number: a sample How many samples do we need? 2 x max frequency

CD quality is 44,100 samples per second Digitizing sound in the computer Each sample is stored as a number 16 bits, 216 = 65,536 combinations But we want both positive and negative values To indicate compressions and rarefactions. Each sample can be between -32,768 and 32,767

Exploring a Sound Use MediaTools to explore a Sound Working with Sounds Well use getSamples to get all the sample objects out of a sound Use getSample to get the value from a Sample obj We can also get the value at any index with

getSampleValueAt Sounds also know their length (getLength) and their sampling rate (getSamplingRate) Save a sound to a file using writeSoundTo(sound,file.wav) Decreasing the volume def decreaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 0.5)

This works just like decreaseRed, but decreasing the amplitude (volume) instead of the red color. Multimedia Java CS1 We use DrJava A free and easy development environment With an interactions pane

We have created Java classes For the students to use and modify World Turtle Picture Sound

FileChooser Java Syllabus Introduction to Java Math operators, printing results, data types, casting, relational operators, Strings, variables Introduction to Programming Creating and naming objects Using a turtle and a world Creating new Turtle methods

Draw simple shapes Using parameters Syllabus - Continued Modifying Pictures using Loops One-dimensional arrays Use for-each, while, and for loops to Increase/decrease colors, fake a sunset,

lighten and darken, create a negative, and grayscale Create and Show a Picture Java Negate Method /** * Method to negate the picture */ public void negate() {

Pixel[] pixelArray = this.getPixels(); Pixel pixelObj = null; int redValue, blueValue, greenValue = 0; // loop through all the pixels for (int i = 0; i < pixelArray.length; i++) Java Negate Method - Cont { // get the current pixel pixelObj = pixelArray[i]; // get the values redValue = pixelObj.getRed();

greenValue = pixelObj.getGreen(); blueValue = pixelObj.getBlue(); // set the pixel's color pixelObj.setColor( new Color(255 - redValue, 255 - greenValue, 255 - blueValue)); } } Testing Negate

Syllabus - Continued Modifying Pixels in a Matrix Two-dimensional arrays Nested loops Copying, mirroring, blending, rotation, scaling Syllabus - Continued Conditionally Modifying Pixels

Replacing a color, reducing red-eye, edge detection, sepia-toned, posterize, highlight extremes, blurring, background subtractions, chromakey Syllabus - Continued Drawing on Pictures

Using existing Java classes Inheritance Interfaces Drawing simple shapes, drawing text, general copy, general scale, shearing, gradient paint, general blending, clipping

Syllabus - Continued Modifying all Samples in a Sound 1D arrays Loops Conditional execution Change volume,

normalizing a sound (make it as loud as possible), force to extremes Syllabus - Continued Modifying Samples using Ranges Loops Clipping, splicing, reversing, mirroring

Syllabus - Continued Combining and Creating Sounds Class and private methods Composing sounds, blending sounds, changing frequencies, and creating echoes MP3 and MIDI

Syllabus - Continued Creating Classes Defining a class Overloading constructors Creating and initializing an array Creating getters and setters Creating a main method Javadoc comments Using Inheritance Turtle ConfusedTurtle

Syllabus - Continued Creating and Modifying Text Reading from and writing to files Handling Exceptions

Modifying programs Getting text from networks Creating random sentences Using text to shift between media Syllabus - Continued Making Text for the Web Throwing exceptions, HashMap, Generics, and

Iterators Create a web page from a directory Create a web page from other web pages Creating a web page from a database Syllabus - Continued Encoding, Manipulating, and Creating Movies

Frame-based animations with simple shapes and text Special effects fade out, fake sunset, and chromakey Syllabus - Continued Speed What makes programs fast? Compilers and Interpreters

Searching Algorithms that cant be written What makes computers fast? Clock rates, Storage, Display Syllabus - Continued Javascript Syntax User Interfaces

Multimedia Multimedia CS2 in Java Driving question: How did the wildebeests stampede in The Lion King? CS2 Ideas Teach data structures using multimedia Stacks and queues for simulations

Linked lists of pictures Order in linked list is order in scene Order in linked list is the layer the picture is in Circular linked list for animation Linked lists of sounds Weave and repeat musical phrases Trees of pictures Scene graph Trees of sounds

Different traversal gives different result Examples of Student Work SoupAudio Collage CanonLinkedList of (MIDI) Music Results Python CS1 is used for non-majors at Georgia Tech

Business students went from 49% to 88% pass rate Similar results at Gainesville and U.Ill-Chicago Java CS1 is used in Georgia High Schools Both in an introductory course and in Advanced Placement courses Also used at Dennison and Duke Java CS2 is used at Georgia Tech Students are transferring into computer science Why Media Computation?

Works on nearly all computers Unlike Alice which requires newer computers Low cost Unlike LEGO Mindstorms or other robots Students can see (and hear) if their programs work Computer Science concepts are introduced to solve problems Looping instead of changing thousands of pixels by hand Open-ended assignments allow students to be creative

Students can use their own media Students can publicly share their media artifacts with others Whats on the CD CS1-Python materials

Chapters from the Media Computation book in Python Course slides Free Jython Environment for Students (JES) Instructor Resources CS1-Java materials Chapters from the Media Computation book in Java Course Slides Java Classes Whats on the CD - Continued CS2-Java materials

Course notes Course slides Java Classes DrJava Free Development environment for Java Mediasources Royalty free pictures and sounds Additional Resources Web Site for Media Computation CS1 Python Book Introduction to Computing and Programming in Python, A Multimedia Approach by Mark Guzdial, ISBN: 0131176552 CS1 Java Book Introduction to Computing and Programming with Java, A Multimedia Approach by Mark Guzdial and Barbara Ericson, ISBN: 0131496980

CS2 Java Book Coming in 2007 Funding Sources National Science Foundation Georgia Tech's College of Computing Georgias Department of Education Al West Fund Toyota Foundation

Recently Viewed Presentations

  • Eastern and Southern Asia

    Eastern and Southern Asia

    Japan is an archipelago. Archipelago—a chain of islands. Koreas (North and South) Peninsula. North and South Korea are located on the Korean Peninsula. Peninsula—a body of land surrounded on three sides by water. Major Deserts in East Asia. Gobi Desert.
  • Earned Value Management (EVM) for Government Agencies

    Earned Value Management (EVM) for Government Agencies

    What is Earned Value Management? Earned Value Management (EVM) is the systematic integration and measurement of cost, schedule, and technical (scope) accomplishments in a project or task. EVM provides both the government and its contractors the ability to examine detailed...
  • ARK OF THE COVENANT - South Walton church of Christ

    ARK OF THE COVENANT - South Walton church of Christ

    ARK OF THE COVENANT By HAROLD HARSTVEDT WHO TOLD MOSES TO BUILD THE ARK? EXODUS 25:10-16 10 You shall make an ark of acacia wood; two and a half cubits in length, a cubit and a half in width, and...
  • Addressing the high levels of psychological distress in law ...

    Addressing the high levels of psychological distress in law ...

    UWA - Jill Howieson. UNSW - Prue Vines et al. Wendy Laucombe - Melbourne University. The Problems Begin in the First Year of Legal Education. The literature suggests that psychological distress is experienced as a direct result of the way...
  • Management of Common Medical Conditions Observed During ...

    Management of Common Medical Conditions Observed During ...

    UI adds to the care partner burden, is often very distressing, and influences the decision to consider early institutionalization (Drennan et al., 2011; Grant, et al., 2013). UI can begin during the early stages of dementia, although it is more...
  • Volume of a Cylinder How much will I hold?

    Volume of a Cylinder How much will I hold?

    d = 8 cm h = 11 cm d = 10 cm V = 275 cm³ Find the height of the cylinder. Cone - Is "pointed" like a pyramid, but its base is a circle. h r V = ⅓Bh...
  • Five Biblical Images of Church and World The

    Five Biblical Images of Church and World The

    Five Biblical Images of Church and World. The slide summarizes five Biblical views of the role of the Christian in the world and the identity of the church in the world as these relate to God's two kingdoms.
  • Social Psychology - Frankumstein

    Social Psychology - Frankumstein

    This is a classic social dilemma because it is to each individual farmer's benefit to let his or her sheep graze as much as possible, but if all farmers do this, the commons will be overgrazed and the grass will...