Because technology has changed the ways in which we use and manipulate data, there is growing recognition that today’s increasingly interdependent world values the skills of an informed citizenry capable of thinking critically. Now more than ever, students will need to learn to make decisions based on complex and interconnected issues, and that has shaped our expectations for what and how students ought to learn. The skills include the ability to navigate and analyze increasing amounts of information and to effectively communicate with others. As a result we are thinking less about students as passive recipients of information, and more about providing them with active, engaging learning experiences. To this end, faculty are seeking methods for providing active, reflective and collaborative exercises that often require alternative methods of assessing their students’ understanding.
How can we give students greater responsibility for their learning that measures ‘collaboration’, ‘creativity’ and ‘critical thinking?’ The classroom level assessment section offers ideas and rubrics for developing authentic criterion and performance-based assessment in situations where students are asked to investigate, collaborate, exchange, synthesize and share knowledge with others.
Course assessments include methods to assess your own courses, student evaluations of online teaching, and grading rubrics. In some instances, examples from actual courses are provided. The resources presented here are meant as a guide, but you should always check with the faculty and administrators within your own college before officially using any tools.
While course assessment is used to measure the effectiveness of a single course, program assessment might be used for a series of courses leading to a degree or certificate. The program section highlights resources for departments and organizations.
Clarity and consistency are the guiding principles for online course design. While Online Learning programs in our University do not mandate a uniform look and function in online course site presentation, students should expect minimal navigation adjustments as they move from one class and course site to the next within a department or program. Individual colleges often present guidelines and observation formats for online courses that can assist departments with formal assessment. Common structural elements from within the guidelines are often utilized on a case by case departmental level which serves as the basis upon which faculty teaching and student learning might be evaluated and weighted.
Colleges and departments that have instituted a faculty-based course assessment or peer observation program for its online sections often find that these learning communities can highlight exemplary pedagogical practices that focus on continuous improvement. The qualitative and quantitative data derived from peer observations can be used to optimize the online experience and derive targeted professional development more closely aligned to particular faculty development needs. Included with the section on formal assessment are resources that can be used to design and implement a faculty peer review process focused on developing instructional excellence in your department or organization.
At the institutional level, college administrators and faculty are asking broader questions about the advantages and benefits of hybrid/blended online learning for the institution, the instructor and the students to determine the effectiveness of online learning compares with that of face-to-face instruction.
Assessing hybrid courses can vary according to the discipline, percentage of the course delivered online, and other factors. Some CUNY professors, such as those in the Business department at New York City College of Technology, believe hybrid courses can and should be assessed the same way as traditional courses. They believe the course objectives are the same for both traditional and hybrid courses, so the assessment procedures should not differ. Other professors, such as those teaching Biological Sciences at City Tech, believe hybrid courses need to be assessed differently. For example, they argue that experiments conducted in online labs need a different procedure for assessment than those run in a face-to-face classroom. Here you will find a range of perspectives expressed in the presentations, lecture notes, and articles from various colleges.
Where to Start?
If you’re interested in learning more about assessment the AAHE (American Academics for Higher Education) relates that “assessment is not an end in itself but a vehicle for educational improvement” that begins with a “vision of the kinds of learning we most value for students and strive to help them achieve”. AAHE has established Principles of Good Practice for Assessing Student Learning. John Jay College, CUNY recommends Carnegie Mellon’s Guide to Assessment. This site offers a comprehensive guide to understanding the various types of assessment and how to implement assessment in teaching, learning, and at the program level. If you are interested in methods for incorporating assessment into your teaching York College recommends, Carnegie Mellon’s “Conduct Assessments of Learning and Teaching.”
Articles About Assessment
- November 17, 2012 The link takes you to the School of Professional Studies Online Design and Interaction Guidelines checklist SPSOnlineDesignandInteractionGuidelines09-01-14 SPS Online Peer Observation Form Lehman College Department ObservationForm-1
- May 22, 2013 Peter Nonacs, professor of Game Theory believes that ‘much of evolution and natural selection can be summarized in three short words: “Life is games.” ‘ The object of any game he knows, is to win—whether it’s “getting the best grade on a midterm, or successfully inculcating critical thinking into your students.’ Find out the tips and tricks he uses to get his students to learn, beginning with letting them ‘cheat’ to win. We all agree that a successful teacher is measured by the success of his or her students, and that students generally measure their success by their grade, earned or otherwise. Nunacs says that if testing “is really just measures of how the Education Game is proceeding” then both the professor and the students’ goals can “be maximized simultaneously when students are given ‘permission’ to call the shots in the test taking game, to dive right in and cheat- cliff notes, sharing guesses and all. Why I Let My Students Cheat on their Exam
- July 5, 2012 The following YouTube videos are the presentation of a course model by Dr. Wenying Huang-Stolte. They include information about the course, detailed assignment information, and the rubric for grading. Part 2 provides actual assignments. Online Assessment part 1: http://www.youtube.com/watch?v=gzYzFRFg5fE Online Assessment part 2: http://www.youtube.com/watch?v=EfRff2UwogQ Here is a shorter PowerPoint presentation in less detail: Assessment in a Hybrid Course and Samples
- September 10, 2012 Bronx Community College has an intensive 6-month online course development program using Blackboard as the learning management system. Materials provided to the faculty participants have been developed collaboratively by Dr. Howard Wach, Dr. Charles Alston, and Dr. Laura Broughton, with feedback from Dr. Giulia Guarnieri, Prof. Moronke Oshin-Martin, and Mr. Albert Robinson. The BCC Online Course Development Checklist provides guidance on the most important components of an online course.
- November 18, 2012 Queens College has a useful site for faculty who are interested in better understanding the process and purpose of student teaching evaluations. While the site includes information that is specific to Queens College Faculty, the site also contains links to articles from the Chronicle of Higher Education on “How to Read a Student Evaluation”, which faculty might find useful.
- September 10, 2012 Bronx Community College has an intensive 6-month online course development program using Blackboard as the learning management system. Materials provided to the faculty participants have been developed collaboratively by Dr. Howard Wach, Dr. Charles Alston, and Dr. Laura Broughton, with feedback from Dr. Giulia Guarnieri, Prof. Moronke Oshin-Martin, and Mr. Albert Robinson. The BCC Hybrid Course Site Review Evaluation Rubric is used by faculty mentors when advising faculty participants on the development of their courses.
- July 2, 2012 While this student evaluation is meant to be used in fully online courses and for the online CUNY BA, many of the questions apply to partially online courses. Moreover, questions that directly deal with the CUNY online BA can be reworded match individual courses. For example, in Section V, the 4th question asks, “If you had a chance to provide general academic advice to a student just beginning the CUNY Online Baccalaureate, what would your advice be?” This question could easily be reworded to read, If you had a chance to provide general academic advice to a student just beginning a partially online course, what would your advice be? FULLY ONLINE COURSE STUDENT EVALUATION INSTRUMENT I. The course… Taught me new concepts Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Taught me new skills Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Made good use of internet resources, books and other materials. Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Was intellectually challenging Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Made good use of available technology Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Made good use of assignments and exams to test knowledge/skills acquired Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Had the right balance of discussion and assignments Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA Will be useful for my current job, future career or future studies Strongly Agree –Agree-Neutral—Disagree–Strongly Disagree–NA ****************************************** II. Considering your experience in this course, please evaluate the features listed below with regard to their usefulness and importance in you learning of knowledge, concepts and skills. Reading assignments Not Important at All–Of Minor Importance–Of Average Importance–Of Above Average Importance–Very Important Feedback from and communication with instructor Not Important at All–Of Minor Importance–Of Average Importance–Of Above Average Importance–Very Important Discussions and other communication with other students Not Important at All–Of Minor Importance–Of Average Importance–Of Above Average Importance–Very Important Projects Not Important at All–Of Minor Importance–Of Average Importance–Of Above Average Importance–Very Important Syllabus and study guides Not Important at All–Of Minor Importance–Of Average Importance–Of Above Average Importance–Very Important *************************************************** Please rate the difficulty of this course based on your experience. Very easy:–Somewhat easy:–Average Difficulty: —Above Average Difficulty: –Very Difficult: On average, how many times did you log into the course site? 1-2 times per week 3-4 times per week 5-6 times per week 7-8 times per week2 9-10 times per week Other
- July 5, 2012 Susan Ko, who is in the Office of Faculty Development and Instructional Technology at SPS, made the following presentation on effective assessment. She includes types of learning activities that are appropriate for online learning, as well as rubrics and models of assessment that are widely used in distance learning. Keys to Effective Assessment in Online Learning
- July 6, 2012 The Rubric for Online Insruction from Cal State Chico provides a list of criteria to evaluate online instruction. The site also contains links to quality courses in Blackboard, instructional design tips, as well as the rubric in PDF format.
- July 6, 2012 Evolve: evaluate, review, and revise your online course From the School of Professional Studies Evaluate your course Once you have taught your course, you will be expected to review, evaluate, and ultimately to revise your course as the last step in your Course Development Process. For the purpose of assessing your course you should consider: What worked? What didn’t? why? What could be improved? How? Once you conclude the delivery phase you can use this information to review, evaluate, and document the revisions you want to make to your course in anticipation of the next time you teach it. Things to think about when ending an online course How will you end the course? Will you send a group “good-by” email or post something in your course? Will you send students their final grades via email? Is there an online mechanism for this built in to your course? Will the students get their grades from your institution via some other mechanism? What is the end date for your course? How will you deal with students who do not complete the course on time? Do you want to survey your students for feedback? Does your institution require and implement a course evaluation? Have you made/kept an copy of your course for yourself? Evaluate your course This is the last step in your online course development process. Once you conclude the teaching phase of your course, you should evaluate the course and your experience. Review any notes you made to yourself as you taught and review student feedback to assess the necessary improvements and revisions to the structure or activities in your course. You may want to think about: What worked? What didn’t? Why? What could be improved? How? Were your discussions successful? Were your assignments and other activities successful? Did you get through all the modules in the course? Did most students complete the course? How was the workload for you and for your students? Were you able to keep up? Was there anything missing? Were there any points in the course where you noticed that students did not do an activity, or did not understand the activity? You may want to ask a colleague or instructional designer to do a review of your course after it has concluded. If you conducted a culminating activity in your course as recommended, or a midterm feedback forum, review these student comments as part of the evaluation and revision planning process for your course. In addition, look at the types of questions your students had and where they had them. Activities, assignments, and areas in your course that did not go as expected or intended may indicate a need for revision. If there was any apparent confusion or a bunch of questions about the same thing, that most likely indicates that more instructions, clearer instructions, or details are necessary.
- July 6, 2012 Professor Lucas Bernard from New York City College of Technology took part in the CUNY Hybrid Initiative. Along with Professor Tim Reinig, he evaluated courses in the business department. The following is a summary of the findings: Status of The CUNY Hybrid Initiative Grant II
- July 11, 2012 Michigan State University’s Online Course Assessment Rubrics list contains links to assessment plans and rubrics from colleges offering major online programs. In addition to links to other college’s plans, rubrics from highly respected sources, including the latest plan from QUALITY MATTERS, are found here.
- July 11, 2012 The Online Journal of Distance Learning Administration is a peer-reviewed journal that primarily serves adminstrators and managers of online programs. Within this online journal, you can find a range of articles on assessment, covering everything from course to institutional.
- July 11, 2012 The Michigan Community College Association Virtual Learning Collaborative’s Program Assessment Plan provides a rubric that guides individuals implementing online programs. The plan covers what to expect during the early stages of evaluation and what to look for as a program matures. A successful program should meet most of the requirements articulated in the mature phase of the plan.
- July 5, 2012 Dr. Ellen Smiley of City College and SPS presented what is known at CUNY and in the United States about distance learning, including the demand for online learning and how well online learning stacks up against traditional face-to-face teaching. The following is from a presentation she made to the CUNY community on February 24th. Smiley Assessment Council 2 24 12 2