The Quest for Quality: Monitoring and Measuring Outcomes

The Quest for Quality: Monitoring and Measuring Outcomes

The Quest for Quality: Monitoring and Measuring Outcomes in Continuing Nursing Education Session 3: Monitoring and Measuring Outcomes for Provider Unit Effectiveness Pamela S. Dickerson, PhD, RN-BC, FAAN Director of Continuing Education, Montana Nurses Association President, PRN Continuing Education Presented by Ohio Nurses Association and Montana Nurses Association 1

Disclosures Purpose: Participants will effectively develop learning and provider unit processes that lead to measurable outcomes demonstrating quality patient care and/or enhanced professional development of registered nurses. Criteria for Successful Completion: Participate in the entire webinar. Complete the evaluation and registration information and return to ONA as directed. Conflict of Interest: There is no conflict of interest for any planner or presenter for this activity. 2

Objective Examine approaches to data collection and analysis related to the effectiveness of a provider unit in improving patient care and enhancing nursing professional development. 3 Reflection If you have participated in sessions 1 & 2, please rate your current ability to do the following: Strong = 3

Moderate =2 Weak = 1 Conduct a needs assessment Analyze a gap Measure an activity outcome Write a

provider unit outcome Your Score: 4 Reflection Data My score is closest to:

4 7 9 12 5 Self-Assessment I know how to Strong =3 Moderate = Weak =

2 1 Develop goals and outcome measures for my provider unit Engage stakeholders in provider unit evaluation Evaluate provider unit data Measure provider unit effectiveness in relation to improved patient care or enhanced professional development of registered nurses

Your Score 6 My total score was closest to a: 4 7 9 12

7 Accreditation Conceptual Framework ANCC 2013 used with permission Structural Structural Capacity Capacity Commitment Commitment Accountability Accountability Nursing

Leadership Profession Leadership Resources al Resources Developme nt: Continuin g Nursing Education Educational

Educational Design Design Process Process Assessment Assessment of of Learning Learning N

N Planning and Faculty Planning and Faculty Design Design Principles Principles Achievement Achievement of of Objectiv Objectiv Quality

Quality Outcomes Outcomes Evaluation Evaluation Process Process Evaluation Evaluation Participation Participation Goals Goals for for Improvement Improvement Value/Benefit Value/Benefit to

to Nursing Nursing Professional Professional Development Development 8 Provider Unit Outcomes Measures Outcome: The impact of structure and process on the organizationand the value/benefit to nursing professional development. (ANCC,

2013). Outcome Measurement: The process of observing, describing, and quantifying predefined indicators of outcomes of performance. (ANA/NNSDO, 2010) 9 Evaluation in Context Goals for Provider Unit Outcome

Measurement for Provider Unit Outcome Measure s Provider Unit Structure & Function Activity Evaluati on

Learning Activities PSD, 2013 10 Why Does Your Provider Unit Exist? Single- focused organization: o Why is continuing education important enough to make it your business? o What does success for your organization look like? Multi-focused organization:

o Why is continuing education part of what your organization does? o How does continuing education benefit the organization? 11 Lets Look at the Organization What is its mission? What are its goals? Who are its stakeholders? (Who cares about the organization, its success, and its functions?) 12

Your Provider Unit What are the goals for your provider unit? How do your goals match those of organization? (If they dont, its time for some serious reassessment.) 13 Think Like A Reporter o h W

en h W y h W Ho w W ha t

Wh er e 14 What Is The Best Time To Evaluate Your Provider Unit? Quarterly

Annually When requested by administration When the provider application is due 15 16 What is the best way to frame Provider Unit Evaluation? How effectively do our structure and processes support our efforts in achieving our goals? Did we accomplish what we planned in our learning activities as a whole?

Do we have administrative support and funding for our next provider application? When is our next provider application due? 17 Evaluating Your Provider Unit: Who Structural Capacity o People Right mix of people? Right qualifications of people? Enough people? People doing the right things? Do people know how to do whats needed as

the system evolves? 18 Evaluating Your Provider Unit: What Structural Capacity: Physical and Material Resources o Office space o Office supplies o Classroom space o Equipment Up-to-date? Functional?

19 Evaluating Your Provider Unit - What Structural Capacity Financial and Administrative Support o Does primary nurse planner have authority to ensure adherence to criteria? o Is there administrative support for the work of the provider unit? o Are there financial resources for support of structure and processes of the provider unit as well as the educational activities? 20

Evaluating Your Provider Unit - When It depends o Some factors may need to be evaluated monthly or quarterly o It may take time to collect data for some outcomes measures, ranging from several weeks to a year or more o Organization may require an annual report and annual goals o Provider renewal time is an excellent opportunity for a comprehensive review and analysis 21 Evaluating Your Provider Unit - Where

Within the context of the provider unit itself Within the context of the organization Each will require unique perspectives, data, and analyses 22 Evaluating Your Provider Unit How Look at components of structural capacity Look at components of processes provider unit functions AND educational design processes Measure against your goals, which are supported by your predetermined outcomes measures

Engage appropriate stakeholders 23 Stakeholders Who are the people who care? o Learners o Faculty o Provider Unit Nurse Planners & Key Personnel o Organizational Personnel Administration Finance Risk Management Others

o Community representatives o Advisory Committee or Board o Stockholders or owners, Board of Directors o Others 24 Evaluate Your Provider Unit: Why Prove that you make a difference o Validate your contribution to quality patient care o Demonstrate your commitment to professional development of registered nurses

25 Example: Provider Unit Effectiveness (how stakeholder input helped improve a quality outcome measure) Outcome Measure: diversity of learning activities. Provider Unit Goal: Implement at least 5 web based learning opportunities for nurses in our hospital during calendar year 2013. 26 Stakeholder Input

o Learners requested opportunity for web-based learning activities they could access via the hospitals intranet and complete at their convenience. o Managers requested that contact hours be awarded for activities that did not require nurses to be away from the practice setting for the time required to attend classes. 27 Data In calendar year 2012, we offered 53 continuing education classes for our nurses all were realtime learning opportunities.

In calendar year 2013, we offered 3 web-based independent study activities. Our goal was 5, but the IT person resigned and has not been replaced. 28 Your Thoughts? Does this evidence address the question of how stakeholder input has helped to improve a quality outcome measure? Yes No

29 30 What Could Make It Stronger? Did learners feel that their needs were met with these 3 activities? What is the plan to move forward? Need to advocate for IT resources? What feedback has been received from managers in relation to their original request? 31

Example: Professional Development Outcome Measure: leadership development Provider Unit Goal: Enhance effectiveness of charge nurses and managers through a series of leadership classes and organizational initiatives. 32 Background Reason for this goal: Fits with organizational goal of improving staff satisfaction and retention through better relationships with front-line leadership. Evidence: o Research demonstrating that new nurses leave 1st

positions in significant numbers, primarily due to unsatisfactory relationships with front-line leaders. o In our facility in 2012, 36% of new RNs left their positions within 18 months of hire. o In our facility in 2012, 64% of RN staff rated their satisfaction as moderate to low and indicated relationship with management a key factor 33 Action Plan Survey staff nurses to get their perspectives on expectations of charge nurses and managers Survey charge nurses and managers to get their perspectives on their expectations of their roles

Survey nursing administration to get their perspectives on expectations of charge nurses and managers 34 Action Plan Based on survey results, meet with nursing administration to develop a plan for role clarification, position description revisions, and other organizational initiatives to support more effective functioning of charge nurses and nurse managers. Plan coordinated educational initiatives. Establish planning committee to develop a series of

leadership classes for front-line managers. Timeline for combined interventions March through June, 2013 35 Educational Interventions Six half-day workshops for front-line managers over a three month period of time. Content from each session built on previous sessions Content addressed key areas of concern noted in surveys: staff team communication, interprofessional communication, conflict resolution, time management, workflow

management, crisis intervention, dealing with bullying/intimidation, feedback & evaluation 36 Outcomes In the last 6 months of 2013, only 8% of new nurses (defined as within 18 months of hire) left their positions. In the staff satisfaction survey conducted in October of 2013, 72% of RNs ranked their satisfaction as moderate to high. Anecdotal comments indicated that the concentrated effort to better prepare and support front-line managers was a key factor.

37 Demonstrating the Evidence % leaving 40 35 30 25 % leaving 20 15

10 5 0 2011 last half 2012 38 More Evidence 100% 90% 80% 70%

60% 2013 by % 2012 by % 50% 40% 30% 20% 10% 0% Low Moderate High 39

So What? Has the provider unit contributed to the professional development of the front-line managers? Has the provider unit contributed to the professional development of ALL nurses in the organization? 40 Leadership: Another Example A college of nursing provider unit is aware of national data, supported by the IOM report on the future of nursing, that there is increased need for

leadership education for registered nurses. A focus group of CNOs from area healthcare facilities supports the need at the local level. 41 Leadership Example Provider Unit Goal: Increase leadership capability of alumni and other participants in our CNE activities Outcome Measure: o Professional Development - Leadership 42

Framing Your Plan For Goal Achievement Use of an evaluation model o Revised Kirkpatrick o Logic o Moore Have a clear plan in place for what youre going to do, what resources will be employed, and what outcome measurement data you will collect begin with the end in mind! 43

Using the Revised Kirkpatrick Model What outcomes do we want? What behaviors are required of these nurse to increase their leadership capability? Where are the gaps? What are current capabilities and what are the desired capabilities? What KSAs do learners need to fill these gaps? What is the best learning strategy to address these needs? 44 Using the Logic Model What do we need? (better prepared leaders)

To address this need, we must: If we do this effectively, the outcome will be: When leaders implement this outcome, we will see: When this happens, organizations will be better because: 45 Using The Moore Model Does the learner know? (is there a knowledge deficit?) Does the learner know how? (is there a skill deficit?)

Can the learner show that he/she can use the knowledge? (is there a performance deficit?) Is the learner able to apply the knowledge? (is there a critical thinking/clinical judgment/application deficit in practice?) 46 The Outcome for The Provider Unit Provider Unit Goal: Increase leadership capability of alumni and other participants in our CNE activities Provider Unit Outcome Measure: professional development leadership Stakeholders: CON leadership, CNO focus group, alumni,

other learners in related activities Wheres the evidence? o Self-report from participants o Data from CNOs regarding leadership evidence in their organizations among nurses who participated in these activities o Number of nurses engaged in leadership roles in practice or professional settings 47 Self-Assessment I know how to Strong

=3 Moderate = Weak = 2 1 Develop goals and outcome measures for my provider unit Engage stakeholders in provider unit evaluation Evaluate provider unit data Measure provider unit effectiveness in relation to

improved patient care or enhanced professional development of registered nurses Your Score: 48 My total score was closest to a:

4 7 9 12 49 50 The Bottom Line Evaluation of your provider unit helps you identify strengths, contributions, and areas of opportunity. Providing evidence of accomplishments of your

goals and related outcome measures helps you validate your worth to the organization and the nurses you serve. 51 Questions and Discussion 52 For Additional Information Pam Dickerson, PhD, RN-BC, FAAN [email protected] [email protected]

Zandra Ohri, MA, MS, RN [email protected] 53 References American Nurses Association & National Nursing Staff Development Organization (2010). Nursing Professional Development: Scope and Standards of Practice. Silver Spring, MD. Nursesbooks.org American Nurses Credentialing Center. (2013). 2013 ANCC Primary Accreditation Application Manual for Providers and Approvers. Revision 3.0. Silver Spring, MD. Author. Commission on Accreditation, ANCC. (2012). The Value of Accreditation for Continuing Nursing Education: Quality Education

Contributing to Quality Outcomes. Retrieved from: http://www.nursecredentialing.org/Accreditation/ResourcesServices/ Accreditation-WhitePaper2012.pdf Dickerson, P. (2012). Rocking the boat: challenges ahead for continuing education providers. Journal of Continuing Education in Nursing 43(10), 443-450. 54 References Kirkpatrick, D. (1984). Evaluating Training Programs: The Four Levels. San Francisco: Barrett-Koehler. Kirkpatricks Four Level Evaluation Model. (2012). Retrieved from: http://www.nwlink.com/~donclark/hrd/isd/kirkpatrick.html

Moore, D., Green, J., & Gallis, H. (2009). Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. Journal of Continuing Education in the Health Professions 29(1), 1-15. W.K. Kellogg Foundation. (2004). Logic Model Development Guide. Battle Creek, MI; Author. 55

Recently Viewed Presentations

  • European Exploration and Colonization Of Australia

    European Exploration and Colonization Of Australia

    European Exploration and Colonization Of Australia ... the Anglican and Methodist religions Most Australians are Christians and belong to the Catholic or Anglican Church Language Official language - English Brought by English prisoners and settlers Government limited number of immigrants...
  • Lesson 1 Classroom Expectations 1. Arrive on time.

    Lesson 1 Classroom Expectations 1. Arrive on time.

    Iago then persuades Cassio to speak to Desdemona so that she can convince her husband to reinstate Cassio. At the same time, Iago suggests to Othello that Cassio and Desdemona might be having an affair. Desdemona loses her handkerchief, which...
  • Les verbes pronominaux:

    Les verbes pronominaux:

    Les verbes pronominaux verbes réfléchis (reflexive verbs) verbes pronominaux idiomatiques verbes réciproques (reciprocal verbs) Il n'y a pas de différence dans la conjugaison de ces verbes. Tous les verbes pronominaux sont conjugués de la même manière.
  • How Science works : Graphs

    How Science works : Graphs

    This sequence of slides is designed to introduce, and explain, the idea of Graphs in practical work, as explained on pages 363-364 in New Physics for You, 2006 & 2011 editions or later.
  • The Great Start Readiness Program (GSRP)

    The Great Start Readiness Program (GSRP)

    Hazel Park-224 TOTALS-4651 Holly-185 Clarkston-151 Farmington-197 Brandon-118 Berkley-100 Southfield-327 Oxford-97 Royal Oak-105 Clawson-55 Madison-53 Pontiac-336 0 9.1633466135458175 0 18.125 13.445378151260504 0 0 0 0 19.512195121951219 0 4.2553191489361701 24.840764331210192 0 29.62962962962963 23.566878980891719 20.982142857142858 16 ...
  • ME321 - Kinematics and Dynamics of Machines Design Process Notes

    ME321 - Kinematics and Dynamics of Machines Design Process Notes

    ME321 Kinematics and Dynamics of Machines Steve Lambert Mechanical Engineering, U of Waterloo Kinematics and Dynamics Position Analysis Velocity Analysis Acceleration Analysis Force Analysis Dynamic (Force) Analysis Determine the reaction forces on pins, etc. as a consequence of a specified...
  • Estrous Synchronization (Ovulation Induction) and Ovsynch John Parrish

    Estrous Synchronization (Ovulation Induction) and Ovsynch John Parrish

    Insert CIDR with CIDIROL capsule Day 6 Remove CIDR Detect Estrus or Inseminate Fixed-time 50 hours From CIDIROL Inject PG Day 7 Beef Cow Day 9-11 Day 8 Inject 2mL CIDIROL 7 Days Triggers GnRH surge Estrogen Lutyalytic to day...
  • Gustar - Chandler Unified School District

    Gustar - Chandler Unified School District

    VerboGustar(To be pleasing to) Remember that the verb GUSTAR is not used like other verbs. You don't conjugate GUSTAR into each form like you normally do with most verbs. ... me/te/le/nos/les. According to the appropriate subject. STEP 3: