Processing Images and Video for An Impressionist Effect

Processing Images and Video for An Impressionist Effect

Processing Images and Video for An Impressionist Effect Automatic production of painterly animations from video clips. Extending existing algorithms for producing painterly pictures to create temporally coherent animations. Reduce user interaction to a bare minimum. Methodology Overview Brush strokes are clipped to edges detected in the original

image sequence and are oriented normal to the gradient direction of the original image. Scattered data interpolation is used when gradient is near zero. Brush stroke list is maintained and manipulated through the use of optical flow fields to enhance temporal coherence. Rendering Strokes Rendered with an anti-aliased line centered at a point, with a given length, radius, and orientation.

Color is determined through bilinear interpolation of the color at the point in the original image. Strokes are spaced apart by a user specified distance. Strokes are drawn using a random ordering to help achieve a hand-drawn look. Randomizing Strokes Random amounts are assigned to the length and radius of the stroke, within a user specified range. Colors are perturbed by a random amount usually between [-15,15] for r,g,b. Colors are then scaled by an intensity

usually within [.85,1.15] and then clamped to [0,255]. The orientation angle is randomized by adding a random angle amount between [-15,15]. All these randomizations are then stored in a data structure for each stroke and are not regenerated on a frame to frame basis. Clipping and Rendering 1. 2.

3. An intensity image is created from the original color image using the equation: (30*red + 59*green + 11*b)/100 The intensity image is blurred using a Gaussian kernel, or a B-spline approximation to the Gaussian. The resulting blurred image is Sobel filtered where the value of the Sobel filter is: Sobel(x,y) = Magnitude (Gx, Gy)

Sobel Filtering The Sobel filter consists of two kernels which detect horizontal and vertical changes in an image. The magnitude of an edge: MSobel[x][y] = |Gh[x][y]| + |Gv[x][y]| The direction of an edge: Sobel[x][y] = tan-1(Gv[x][y] / Gh[x][y]) Clipping and Rendering Cont 4.

5. Start the line at the center point and grow the line in its orientation direction until either an edge is reached or maximum length obtained. An edge is reached if the Sobel value decreases in direction of orientation that the line is growing. Once we have determined the endpoints, the stroke is drawn using the perturbations stored in our stroke data structure. A linear falloff is used in a 1.0 pixel radius region ( alpha transition from 1 to 0 ).

Brush Stroke Orientation Strokes are drawn with a constant color in the gradient-normal direction. Brush strokes in a region of constant color smoothly interpolate the directions defined at the regions boundaries. A thin-plate spline is used for the interpolation

because it cannot be assumed that the data lies on a uniformly spaced grid. The gradient at the center point is bilinearly interpolated and the direction angle is computed using: Arctan(Gy/Gx) + 90 ( normal to the gradient ). The perturbations are added and the result is stored as this strokes direction angle. Animating the Images Since there is no a priori information about pixel movement, vision techniques are applied to guide the brush strokes. Optical flow vector fields are utilized to determine pixel

movement. Brush stroke centers are mapped to subpixel locations in the new image. Handling Sparse Regions After using the vector flow field there may be regions which are too sparsely populated. Sparse regions dont provide the coverage we wish to have. Use Delaunay triangulation to find triangles with area greater than maximum specified. Add pixels to reduce the area of these triangles.

The Delaunay triangulation has the property that the circumcircle (circumsphere) of every triangle (tetrahedron) does not contain any points of the triangulation. Delaunay Triangulation Handling Dense Regions Dense regions will greatly slow down the rendering process. The edge list of the triangulation is traversed. If the distance between points is less than a user-specified distance, the

stroke that is drawn closer to the back is removed. Display list of strokes determines stroke order. Potential Problems The method of optical flow estimation employed by Litwinowicz assumes that lighting is constant and that occlusion may be ignored, resulting in objects appearing and disappearing. Stroke movement is only as good as the motion estimate technique. Noise in the images will cause noise in the animations.

Possible Enhancements and Future Work Scale the offset of the stroke based on the length of the stroke. Implement additional rendering styles to produce other artistic styles. This would involve modifying the stroke drawing algorithm to emulate the different styles. Apply this technique to 3D models where the motion of the objects in the scene would be known a priori. This could be accomplished by removing most of the vision

algorithms from the system and replacing them with data structures that would hold all the necessary information for all the models of the items in the animation.

Recently Viewed Presentations

  • Implementing interventions for child neglect

    Implementing interventions for child neglect

    Behavior of Caregivers. More verbal aggression/lower verbal accessibility. Lower positive behavior/ more negative behavior. Schumacher et al., 2001. Connell-Carrick, 2003 Lower self-esteem, less confident, less outgoing, poor social skills. Neglectful mothers view themselves as less supported. Several studies have found...
  • Learning Objectives (cont'd.) - East Carolina University

    Learning Objectives (cont'd.) - East Carolina University

    Symmetric Encryption (cont'd.) Symmetric encryption problems. Difficult to distribute new keys to authorized parties while maintaining security, control over keys. Private keys do not work well in large environments. Data Encryption Standard (DES) Encryption algorithms adopted by U.S. government ....
  • The Trial Of Cardigan Jones

    The Trial Of Cardigan Jones

    or innocent? Honest. Truthful. When you are on the witness stand you must. be . honest. and truthful. ... Pointed. Used a finger to show where something was. The man . pointed. to the paper in his ear! Murmur. The...
  • Multimedia Devices

    Multimedia Devices

    Multimedia Overview. The term multimediahas different meanings for people because there are many types of multimedia devices.This chapter focuses on the most popular areas—CD and DVD technologies, sound cards, cameras, and scanners.
  • Response To Intervention and Early Intervening Services ...

    Response To Intervention and Early Intervening Services ...

    Implications for Implementing an RTI Model (cont.) This data-based documentation, reflecting formal assessment of student progress during instruction, is a component of RTI models; however, this regulatory requirement is mandated, whether or not a state chooses to implement an RTI...
  • On Your Map - PCMS

    On Your Map - PCMS

    On Your Map - Refer to Page 39 in the Textbook. Cut Map to fit on page 11 of IN. Outline, c. olor in and label the 4 settlements. Four settlements should include: New . Spain, New France, New England,...
  • Landforms - ecboe.org

    Landforms - ecboe.org

    A fan-shaped deposit of sand and mud at the mouth of a river. Delta
  • First Aid Jeopardy - miningquiz.com

    First Aid Jeopardy - miningquiz.com

    First Aid Jeopardy Game Rules 2 teams, winner determined by highest point total Teams select a captain that will give the final answer No set time limit on answering A team selects a category and desired point total For a...