A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles ...

A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles ...

A Faculty-Student Dialogue on Assessment: Rubrics as Vehicles for Communicating Learning Expectations in the Field Eric J. Pyle James Madison University Assessment vs. Evaluation Assessment involves comparing information gathered from subjects relative to some established goal or objective (Kizlik, 2009). These goals, objectives, or outcomes are set in advance, and should be clear to both instructors and students. Evaluation allows the establishment and communication of the worth of an activity (Kizlik, 2009). This worth can be determined by the extent to which decisions of instructional approaches, arrangements, organization, etc., are effective in aiding students into reaching the desired outcomes.

Goals vs. Objectives Goals reflect the overall outcome expected of an learning activity, such a capacity to generate geologic descriptions of areas; Objectives are the specific means by which students can demonstrate that they have met the goal expected of them, such as generating a geologic map, cross-section, and lithologic descriptions for a given area. Objectives Objectives consist of three parts: Condition refers to the context of the learning experience, the setting, the materials, available or to be used; Performance is a statement of what students are supposed to do, a description of the task or demonstration; Criterion is the indication that the particular objective has been met, and to what degree it has been met.

Example: Given a base map and aerial photograph of an area, the student will produce a geologic map and crosssection that describes the relative sizes and orientations of the lithologies and structures for the region. Chiappetta & Koballa, 2006 Assessments and Objectives Knowledge specific knowledge, recall, terminology, etc.; Skills specific performances, equipment, tool-use, etc. Dispositions scientific habits of mind, persistence, question-posing, etc. The struggle is to ensure that the entire range of learning described in the goals is expressed through appropriate objectives and related assessment tasks validity. Types of Assessment Objective/Knowledge items - multiple choice,

fill-in-the-blank; uses binary, nominal criterion Prescribed/Lexical tasks - problems, short answers; procedures as well as the task are of value; uses ordinal level criterion measure; Open-Ended/Prototype tasks - essays, artifacts; uses interval level criterion measure on 0-100 scale. The struggle in each form is to provide a clear and consistent determination of mastery of the related objectives reliability The special case of field learning Expectations are not just limited expressions of knowledge but the application of that knowledge; Expectations include acquisition, demonstration, and mastery of skill sets;

Expectations include consistency, persistence, attention to detail, and synthesis of working hypotheses. The special case of field learning: Knowledge Cognitive Earth Science level Sample Verbs Define, describe, Knowledge identify Interpolate, Comprehension estimate, predict Compute, modify, Application relate, use Diagram, divide, Analysis infer Arrange, generate,

Synthesis design Contrast, interpret, Evaluation appraise Based on Bloom (1956) concept rock texture, RFM identification Draw contour lines from elevation data Graph a topographic cross-section Plot fold axis on a map Construct a geologic map from field data Assess landslide hazards from map data The special case of field learning: Psychomotor Skills Sample Indicators Earth Science action level

Imitation Manipulation Precision Articulation Naturalization Based on Dave (1975) Crude reproduction of action based on observation and minimal practice Performance from instruction with attention to form Accuracy, proportion, and exactness in performance, with minimal error Coordinating a series of acts with harmony and consistency Determination of mineral sample physical properties, such as hardness, streak, or observing

cleavage Measurement and data encoding using a Brunton compass or Jacobs staff Collection of physical and chemical data at several points along a stream Generating a map from a series of station measurements, plotted on a base map Generation of finished maps that Smooth, natural reflect multiple layers of data performance with minimum collection and procedures and of psychic energy coordinate well with field notes and diagrams. The special case of field learning: Dispositions Dispositional Level Receiving Responding

Valuing Organizing Sample Indicators Earth Science action Follow directions, locate, identify Completes assigned tasks at or above level required, or for selfsatisfaction Accepts, prefers, and commits to scientific values Following along with a field trip guidebook as a part of a field trip Once a local geologic map has been studied, a student seeks out a regional geologic map to see larger context More than one measurement of a particular parameter is sought in each location Personal values are

brought into line with scientific values Each field investigation is approached with a set of questions framed on methodologies and possible outcomes Lifestyle adoption indicative of a ased on Krathwohl, Bloom, & Masia (1973) Characterizing preference for scientific Active seeking of communications with other students and faculty on Combining Domains in Assessments

Structure Sample Task Domain-Based Rationales Objective/ Knowledge Based on observations of current stream conditions and local erosion and sedimentation patterns, make a prediction of how the stream changes when flow reaches flood conditions. Prescribed/ Lexical Knowledge recall and appropriate Correctly applying application of terminology; terminology in a lithologic Skills communicating descriptions in description using texture, written or oral form;

and mineralogy, and internal Dispositions Use of a variety of descriptive features or structures. terms in a manner that reflects possible contexts. Knowledge beliefs from prior knowledge, applied to novel situation; Skills measurements and observational descriptions of setting; Dispositions use of more than one parameter in making the prediction. Knowledge synthesizing an analogy representing the distribution and orientation Constructing an accurate of materials; cross-section from a map, or Skills drafting the cross-section with Opendistinguishing the correct consistency of measurement, to scale, from Ended/ cross-section from the map; Prototype distracters, stating reasons Dispositions Cross-section contains all for rejection. necessary detail, on Smith (1995), Lawrence

& Margolis (1999), Murphy (2002), and Sibley (2005)drafted in a manner that communicates clearly the interpretations Application: Field Notes Rubric Condition: note-taking of field situations, with general expectations of description and supporting other documentation, such as maps or lithologic descriptions Where is the Dialogue? Exchange begins with sharing the rubrics ahead of time can be stated a priori by instructors, which allows reliable Criterion measurement; or with students, which allows for closer validity of Negotiated measurement to particular or unique contexts.

Student first (and continuing) response is usually a request for clarification is finer grained not what do you want, but what do you mean Reply by, allowing more specific examples to be generated. Next exchange is the assignment of scores are broken down by the level of mastery for each dimension or Scores objective; Narrative feedback becomes focused on stated criteria. Next exchange is student formative response what do they do next time to improve, or in some cases, revision of prior submission; Less prominent is the student of response of why did I get this grade, when subscores relative to descriptors are provided. Other Applications in Field Courses: Maps and Cross-Sections Other Applications in Field Courses:

Lithologic Descriptions, Memoirs Uses and Extensions These rubrics have been used with two successive JMU field courses; In general, their use allows for quicker and more consistent turn-around of student assignments; But continued discussion among participating faculty is needed to fine-tune language relative to expectations; The Knowledge-Skills-Dispositions format is also employed in student evaluations of instruction, specific to the individual assignments in the field course. Data analysis is on-going; Rubrics are frequently uses on assignments in BA-Earth Science classes at JMU; A general rubric is in use with the JMU program evaluation Students participate in a performance task, with differential expectations based on student standing in program. References Bloom, B. S. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. New York: David McKay Co., Inc.

Chiappetta, E. L. & Koballa, T. R. (2006). Science instruction in the middle and secondary schools: Developing fundamental knowledge and skills for teaching, 6 th ed. New York: Allyn & Bacon. Dave, R. H. (1975). Developing and writing behavioral objectives. R. J. Armstrong, ed. Educational Innovators Press. Kizlik, B. (2009) Measurement, Assessment, and Evaluation in Education. Website: http://www.adprima.com/measurement.htm, retrieved 3/10/09. Krathwohl, D. R., Bloom, B. S., and Masia, B. B. (1973). Taxonomy of education objectives, the classification of educational goals, handbook II: Affective domain. New York: David McKay Co. Inc. Lawrence, S., and Margolis, E. (1999). Concepts and cognitive science. In E. Margolis and S. Lawrence (eds.), 3-83. Concepts. Cambridge, MA: MIT Press.

Murphy, G. L. (2002). The big book of concepts. Cambridge, MA: MIT Press. Pyle, E. J., & Brunkhorst, B. (2009). Developing and applying the knowledge, skills, and dispositions needed for effective Earth science teaching. In A. Collins & N. Gillespie (Eds.) The Continuum of Secondary Science Teacher Preparation: Knowledge, Questions, and Research Recommendations. Rotterdam: Sense Publishers. Sibley, D. F. (2005). Visual abilities and misconceptions about plate tectonics. Journal of Geoscience Education, 53(4), 471-477. Smith, E. (1995). Concepts and categorization. In E. Smith and D. Osherson (eds.), Thinking: An invitation to cognitive science, vol. 3., 3-33. Cambridge, MA: MIT Press. Pyle, E. J. (2009). The evaluation of field course experiences: A framework for development, improvement, and reporting. In S. J. Whitmeyer, D. Mogk, & E. J. Pyle (Eds.) Field Geology Education: Historical Perspectives and Modern Approaches. GSA Special Paper 461. Boulder, CO: Geological Society of America.

Recently Viewed Presentations

  • Sixth Sense Device - METU Ceng Demo Day 2017

    Sixth Sense Device - METU Ceng Demo Day 2017

    Which different and unique things do we offer? Which technologies do we use? What was done about the project till this moment? What do we suggest to do later? REFERENCES Strattman E.,Ousterhout J.,Madan S. (n.d.). Integrating Long Polling with an...
  • KPMG Talkbook template

    KPMG Talkbook template

    Immediate payments are possible with closed-loop Wallets - a touchless solution. Providing a digital identity. Free Bank Account & ATM through PradhanMantri Jan DhanYojana. BPL App for BPL Schemes & Benefits. ... Real time Identification - Touch & Go.
  • Chapter 1

    Chapter 1

    This type of cable is often needed when you connect two devices of the same type to one another. For a 1000BaseT crossover cable, you have to cross the blue and brown pins because they're used in 1000BaseT. Guide to...
  • Aeries Web Overview - Alvord Unified School District

    Aeries Web Overview - Alvord Unified School District

    ALVORD UNIFIED SCHOOL DISTRICT PROMISE: ALL STUDENTS WILL REALIZE THEIR UNLIMITED POTENTIAL. 19. Opening Multiple Windows. While on any Form/Page, right-click on the Form/Page you would like to open in a new tab or new window.
  • Cook. 2010. The influence of metritis and mastitis on dairy ...

    Cook. 2010. The influence of metritis and mastitis on dairy ...

    Giri et al., 1991.Role of Eicosanoids in Abortion and its Prevention by Treatment with FlunixinMeglumine in Cows During the First Trimester of Pregnancy Journal of Veterinary Medicine A.38, Issue 1-10: 445-459,
  • The derivative of a function Heydar Zahedani CSU San Marcos

    The derivative of a function Heydar Zahedani CSU San Marcos

    He claimed that any analytic function had a power series expansion . Cauchy (1823) There is everywhere continuous, nowhere differentiable function. ... The derivative of a function Heydar Zahedani CSU San Marcos
  • Perspective Construction - St. Bonaventure University

    Perspective Construction - St. Bonaventure University

    Perspective Construction One point at a time. Defining the Station Point, Vanishing Point, Horizon Line, Ground Line & Projection Plane. * * View from the front * View from above * Top and front combined * Line A * Elevation...
  • Hazardous Waste Generator Improvement Rule and VSQGs Rule

    Hazardous Waste Generator Improvement Rule and VSQGs Rule

    Noncompliance by any generator with an applicable condition for exemption from storage permit and operations requirements means that the facility is a storage facility operating without an exemption from the permit, …, and operations requirements in 40 CFR Parts 124,...