Creation of a Statewide Survey of Incentives & Disincentives for the use of OpenCourseWare in Utah Anne Arendt Research Questions

(a) Perceived incentives for use of OCW by the Utah adult population? (b) Perceived disincentives that prevent use of OCW by the Utah adult population? (c) Diffusion attributes that contribute to the adoption (incentives) of OCW in Utah? (d) Diffusion attributes that contribute to rejection (disincentives) of OCW in Utah? Overall

Descriptive study that employed a survey method. Consisted of three stages: a preliminary Delphi technique questionnaire based on Rogers (2003) attributes of innovation, a pilot study, and the primary study. Delphi Technique

Survey instrument developed using attributes that emerged from a Delphi technique Input from experts in the OCW field. Eleven experts where asked to participate and five were actively involved: Marion Jensen, Ahrash Bissell, Terri Bays, Steve Carson, and Andy Lane. Delphi Initial

identify potential incentives and disincentives for end-user OCW use in the form of two questions: (a) in your opinion, what are incentives for potential users of OCW to make personal use of the resource? (b) in your opinion, what are disincentives for potential users of OCW to make personal use of the resource?

Delphi Timeframe The Delphi analysis occurred in three phases: (a) initial question creation, which began on June 12, 2008; (b) compilation and rating, which began on July 3, 2008; and (c) review, which began on July 17, 2008 and ended on July 25, 2008.

Phenomenological Method To compile the results of the experts a phenomenological research method, which describes the lived experiences for several individuals about a concept or phenomenon, was used. Phenomenological Method (a) list incentives or disincentives, (b) determine the meaning of the statements,

(c) identify common characteristics or essentials, and (d) create descriptive statements of essential themes. Throughout this process the experts were asked for input a total of four times. Delphi Step 2 compiled responses sent to expert

rate their agreement with each statement on a five-point Likert scale where 1 represents strongly agree and 5 represents strongly disagree. Statements with a mean of 3.5 or higher and a standard deviation below 1 were designated as areas of agreement. Delphi Step 3 Areas of agreement were compiled and

sent to the group of experts for review with an area allocated where they could identify additional items they believe should have been included that differ from the general tendency but have justification. Delphi Step 4 Finally, additional items for the survey were again sent out for final review and commentary.

Categorization The results were then distributed into categories of Rogers innovation attributes: (a) relative advantage, (b) compatibility, (c) complexity, (d) trialability, and (e) observability.

Categorization Although the items were not separated into Rogers innovation attributes on the end-user distribution survey, these categorizations were used when statistical analysis is performed. Pilot Test Pilot tested with 44 individuals. Cronbachs alpha was calculated to

assess inter-item consistency for the pilot test and required a reliability of .70 or higher before the survey instrument would be used (Schumacker, 2005). Cronbachs Alpha Cronbachs alpha was calculated to assess inter-item consistency for the N = 44 pilot test and required a reliability of .70 or higher before the survey instrument

would be used (Schumacker, 2005). Modifications to the questions were made as necessary based on the results of pretesting. Cronbach Results Based on the pilot study (M = 44) categorization by Rogers attributes and Cronbachs alpha assessment, two questions were removed.

(1) availability of alternative methods to find information online, and (2) the need to purchase books or items not provided online. Formal Survey Survey sent via postal mail to a randomized group of 753 individuals residing in Utah between the ages of 18 and 64. Names and addresses, along with

associated gender, ethnicity, income, age, education, and occupation were obtained from Alesco Data Group, LLC of Fort Myers, Florida. Sample Size the survey sample size was based on numbers used for inferential statistics. a minimum of 95% confidence level needed to be obtained with a confidence

interval, otherwise known as a range of scores or margin of error, of 8%. Sample Size Based on the Utah population of 1,383,605 for high school graduates ages 18 to 64 in 2006 (U.S. Census Bureau, 2007), a sample size of 150 was necessary to achieve a confidence level of 95% and a confidence interval of 8%.

Therefore, 753 surveys were sent out with three follow-up letters. Sample Size The percentage response rate needed from the 753 surveys was 19.92%. It should also be noted that Alesco data is deemed 90% deliverable, so with that in consideration a percentage response rate of 22.16% would have been required of

the deliverable 678 surveys. Survey Package (a) a cover letter: importance of the participant, incentives offered, purpose, confidentiality, & completion time; (b) a statement of consent, (c) the survey with a unique identification number that tied the survey results back to the demographic variables, and

(d) a prepaid addressed envelope for return of the survey (e) a monetary incentive (in this case $.50) Reminder 1 The first follow up letter was sent via postal mail two weeks after the study introduction. The purpose of this letter was to thank those who have already completed and returned their survey

package and remind those who had not yet done so. Reminders 2 and 3 Second and third follow-up letters were delivered via postal mail to nonrespondents on the third and fourth weeks after the study introduction. In the last follow-up letter, instructions were included for requesting another copy of the survey. Three individuals requested new

copies of the survey via the email method specified. Received Back Of 753 surveys set out across Utah, 35 were returned as undeliverable, leaving a total of 718 deliverable surveys. Of the deliverable surveys, 180 responses were received, for an overall response rate of 25.06%.

Of the 180 received, 140 were deemed usable. Received Back Five survey responses were removed at the request of the recipient leaving 175. Ten of the remaining 175 responses were removed due to missing over 20% of the survey answer values, leaving 165 total responses.

A category of do not know eliminated another 25 responses, leaving 140 total responses. Confidence With only 140 of the surveys being deemed usable, that number dropped to a confidence level of 95% and a confidence interval of 8.28%.

During Survey A Cronbachs alpha was also run at completion of the survey data collection to assess the categorization by Rogers attributes. A Cronbachs alpha over .70 was the target. This was achieved for all categories for both incentives and disincentives on all Rogers attributes. During Survey

Wave Analysis On order to assess the consistency of data throughout the survey period, a wave analysis was done for incentives and disincentives by week for overall means by Rogers attributes. Although there was some variance in the means and standard deviations, overall the results were consistent.

Results Assessment In addressing the perceived incentives for use of OpenCourseWare by the Utah adult population, statistics were given for overall incentives and frequency for each independent incentive. Descriptive statistics of survey responses for incentives were also supplied. The same was done for disincentives.

Correlations In addressing incentives in the Use of OpenCourseWare in Utah by age, income, gender, education, county, occupation, and ethnicity a variety of correlations were used based on the data type. . Correlations Pearson product-moment correlations were used between incentives and age and also incentives and

income. Point bi-serial correlation coefficients were used between incentives and gender. Spearmans RHO was used for correlation coefficients between incentives and education. Eta Correlation was used between incentives and county, incentives and occupation, and incentives and ethnicity. The same was done for disincentives.

Rogers Attributes In considering the diffusion attributes that contribute to the adoption (incentives): Descriptive statistics for incentives as categorized by Rogers attributes of innovation, overall means for incentives by Rogers attributes, Rogers Attributes

frequency for incentives based on Rogers attributes of innovation, multiple regression analysis on incentives and Rogers Attributes, and multiple regression analysis of weighted incentive mean and Rogers attributes. The same was done for disincentives. Drawback One drawback of this study, however, was that

there were not enough results to make use of logistical regression a possibility. As it stands, the multiple regression analysis was not statistically valid; however, the participant numbers were not high enough to constitute logistical regression, although it would have been ideal. Due to this, attempts to quantify the role Rogers attributes played in predicting user response were not possible.

Open Ended Questions Results of Open-Ended Questions The results of open-Ended questions were compiled in their own section. It was anticipated that results would be categorized but that was not necessary as there were only three open ended comments. Threats to Validity & Reliability

Statistical Validity Threats to external validity involve the extent to which the results can be generalized to other people and places Threats to internal validity have do with selection, where the groups end up not equivalent, and attrition, where some participants do not complete the survey. Threats to Validity & Reliability

History As each participant only completed the survey once there was not a possibility for validity errors between surveys, but there may be events that occur in the 6-week duration of the study itself-which could affect early responders as compared to late responders. Due to this, a wave analysis was done

Threats to Validity & Reliability Sampling Error Concerning sample selection, it may be faulty to assume the similarity of the participants involved. One way to remedy this would be to perform additional surveys of other groups and compare results. For the purposes of this study sampling error is reduced through the use of a large sample.

Threats to Validity & Reliability Response Error According to Dillman (2000), the single most serious limitation to direct mail data collection is the relatively low response rate. Response rates are often only about 5 to 10 percent. Individuals may choose not to respond, which could cause bias in the responses that are received and cause non-response error.

Threats to Validity & Reliability To discourage this from occurring, the survey package included a monetary incentive of 50 cents in the initial survey package because research has shown consistently that inclusion of a small financial incentive can improve response rates significantly over promised incentives, nonmonetary incentives or no

incentives (Dillman, 2000; StatPac, 2007). Threats to Validity & Reliability Although higher monetary rewards may have created even higher response rates, this figure was within the budget allocated for this study. Three follow up letters were also sent to those who had yet to respond, encouraging their participation (Dillman; Miller & Smith, 1983). For this study, a response rate of 25.06% was

achieved, well beyond typical response rates noted above. Threats to Validity & Reliability To address any potential non-response error that still might occur, a comparison of non-respondents and respondents was performed to assess similarities in the demographics in an effort to address any potential discrepancies in the responding

sample. Threats to Validity & Reliability If the non-respondents do not appear different then the results can be generalized to the sample and population (Lindner, Murphy, & Briers, 2001; Lindner & Wingenbach, 2002; Miller & Smith, 1983). There were no differences greater than 10% between respondents and nonrespondent demographics.

Threats to Validity & Reliability Response error could also be due to uninformed response error as well. where individuals feel obligated to express an opinion for which they have little or no information was minimized by offering a do not know option. Twenty-five responses were not included in the overall results due to respondents

making use of the do not know option. Threats to Validity & Reliability Reliability internal consistency reliability because the instrument was only administered once, in a singular version, and each participant completed the survey only once. Cronbachs alpha calculated pilot test and required a reliability of .70 or higher before

the survey instrument was used (Schumacker, 2005). Threats to Validity & Reliability Measurement Error questionnaire had clear, unambiguous questions and response options. Does not completely remove the potential of measurement error, which results when respondents fill out surveys, but do not

respond to specific questions. Threats to Validity & Reliability In order to avoid processing errors the data will be backed up and all calculations, sorts or summaries were run twice to ensure the same results are obtained. Threats to Validity & Reliability Coverage Error

In order to avoid coverage error the sample was obtained from Alesco data, which is deemed 90% deliverable. An updated randomized list was obtained in April 2008 to ensure the timeliness of data. Issue One challenge with this technique was encouraging and maintaining expert involvement (5 of 11 and some limited)

Process of developing the questions took 42 days due to waiting for responses from experts who stated they would submit feedback. Issue Of the 11 individuals asked to participate in the Delphi technique, five opted to participate: Marion Jensen, Ahrash Bissell, Terri Bays, Steve Carson, and Andy Lane.

Andy Lane and Steve Carson participated in all phases Issue A second limitation was the number of participants in the survey; this occurred in part because the study was self-funded. At the same time the self-funding was a limitation, however, it was also a benefit in that there were no incentives to obtain any

particular set of results. Ideally, additional studies can be done with larger sample sets in the future. The Results Full results of the survey can be found at: ARENDT, A., SHELTON, B.. Incentives and Disincentives for the Use of OpenCourseWare. The International Review of Research in Open and Distance

Learning, North America, 10, oct. 2009. Available at: http://www.irrodl.org/index.php/irrodl/article/ view/746/1393 . The Results Printable results: http://www.irrodl.org/index.php/irrodl/rt/prin terFriendly/746/1393

The full dissertation for which this research was done can be found at: http://digitalcommons.usu.edu/cgi/viewcon tent.cgi?article=1379&context=etd Themes that Emerged In regard to incentives three themes emerge: (a) self-directed knowledge and learning, (b) convenience, and (c) quality. In regard to disincentives five themes

emerge: (a) lack of support, (b) no valid certification, (c) topic issues, (d) lack of content, and (e) lack of resource knowledge. Outcomes Learning and knowledge are perhaps the most significant incentives for using OpenCourseWare (OCW). However, based on this study, individuals are not

driven to use OCW as a precursor to attending a particular institution or to taking a particular traditional class as these questions were asked specifically on the survey Thoughts Institutions offering OCW could perhaps work to transition some OCW users into degree-granting paid programs by (a)

noting available degrees or courses associated with the class the individual is reviewing or (b) permitting a more flexible model of institution entry where individuals could enter into a program at their level of competency. Thoughts A test drive model can be developed to promote or market an institution, using

OCW as a maven trap (Gladwell, 2002). Implementing this model would help users to keep their educational costs down, while receiving desired support and valid certification. Thoughts Offering a flexible entry model into traditional at-a-cost education could be accomplished by offering some type of

testing to determine if the OCW user comprehended and mastered the course objectives. Thoughts If testing is offered, the OCW website could suggest other OCW courses of potential interest as well as provide information about associated degrees or traditional instructor-led courses that seem

to be a good fit. Thoughts The users could find their personal level of competency using measurable assessments. Once the users reached their maximum capability, the results message could explain the potential benefits of traditional instructor-led education for areas they need more help with, noting that although there would now be a cost

there would also be support as well as acknowledged and accredited certification or degrees granted. Thoughts The site could also note traditional at-acost classes for which there are no OCW alternatives, yet are practical for their area of interest. This may include classes for which there is extensive lab time, expensive equipment requirements, or

requisite instructor-led time. Thoughts It should be noted that according to this study there is no direct relation between the amount of education a potential OCW user has and the incentives for OCW use, so institutions might also want to reassess their presumptions relating to prior educational attainment in relation to who may be using, and potentially mastering,

OCW materials. Lack of content or topic issues is another area that surfaced as a disincentive. This disincentive could, in part, be remedied by elevating the status of current OCW/OER recommendation engines such as ccLearn DiscoverEd Thoughts

CW websites should make their content available to these recommendation engines via tags for their content and should reciprocally link to one or more of these sites. Thoughts Although users might leave one particular institutional site in favor of content in another, they are encouraged to continue

their pursuit of knowledge, and this is one of the ultimate goals of OCW and the open educational resources movement. Thoughts In marketing efforts it is suggested that institutions follow Rogerss advice for campaign communications. Campaign communications include (a) using formative research to understand the intended audiences and

campaign messages, (b) setting specific and realistic campaign goals, (c) using audience segmentation to create more homogenous audience groups, and (d) designing mass media messages that trigger interpersonal network communication to occur. Thoughts Equally, institutions will want to identify potential opinion leaders, change agents, and

champions. As Rogers notes, opinion leaders provide information and advice about innovations to many individuals in the system (2003, p. 27). Change agents influence an individuals decisions toward the innovation (2003, p. 27). Champions put their weight behind an innovation, thus overcoming indifference or resistance (2003, p. 414). Thoughts

Rogers asserts that mass media is best for communicating at the knowledge acquisition stage to inform potential users of the innovation, and interpersonal communications are best used at the persuasion stage to influence potential users. Institutions will want to consider marketing OCW and other related open educational resources as technology clusters to encourage more rapid diffusion results.

Thoughts This study is meant to spark thinking not necessarily conclusively answer questions. It is my hope that this presentation and information has sparked ideas and action toward further research in this area.

Recently Viewed Presentations

  • Meta-Cognitive Markers

    Meta-Cognitive Markers

    Alone, mark and solve word problems #1-3 using the markers. Explain how you got your answers to the person sitting next to you. Did you get the same answer? If not, figure out who is correct. In groups of four,...
  • Studi Hadist 2 Kelompok 8 Fitri Hidayati (D71214039)

    Studi Hadist 2 Kelompok 8 Fitri Hidayati (D71214039)

    Adapun Jumhur Ulama berpendapat mustahab (sunnah). Mereka mengatakan: "perintah yang disebutkan dalam hadist yang terdahulu-terdahulu dipalingkan dari hukum wajib (kepada hukum sunnah). Dan juga fatwa sahabat kepada Asma' bintu Umais, istri Abu Bakar, ketika selesai memandikan jenazah ...
  • Chapter 6 Population and Community Ecology Friedland and

    Chapter 6 Population and Community Ecology Friedland and

    The Exponential Growth Model. The exponential growth model. When populations are not limited by resources, their growth can be very rapid. More births occur with each step in time, creating a J-shaped growth curve.
  • Lesson 17: I can use basic facts to estimate quotients with ...

    Lesson 17: I can use basic facts to estimate quotients with ...

    Lesson 17: I can use basic facts to estimate quotients with two-digit divisors. 5th Grade Module 2 - Lesson 17 Created by Leigh-Ann Pieragostini
  • KS1 Maths Evening - KCPE-KCSE

    KS1 Maths Evening - KCPE-KCSE

    Practical Subtraction (using objects and pictures) Subtraction (Find the difference) Subtraction (using a number line) Subtraction (using a number square) Subtraction (using an 'empty' number line) Subtraction *Apply what you know within a game* Multiplication (repeated addition) Multiplication (drawing and...


    Standard PPBE. Delinquent Debt Management. Treasury System Roll-Out. ... System Posting Chart of Accounts to the DoD Chart of Accounts. Posting Logic. ... Standards (data, process and controls) are imperative to unify enterprise systems and processes.
  • Clinical Consequences of Chimerism - Denver, Colorado

    Clinical Consequences of Chimerism - Denver, Colorado

    Clinical Consequences of Chimerism. Nicole L. Draper, MD. Assistant Professor, Department of Pathology ... Rh-negative with antibody against group B. ... her antibody screen revealed strong 3+ to 4+ anti-D and anti-C antibodies, with a negative autocontrol ...
  • Graduation Project

    Graduation Project

    GSM-900 Mobile JAMMER EE592:Graduation Project Ahmad Jisrawi