Formally Assessing Online Learning

Online courses have the same course objectives and learning outcomes as courses that are offered on campus. However, online courses are often different in their delivery method. Instructors may adapt materials and course information for delivery on the web. They may also supplement course materials with online resources and activities. For this reason, formal assessment for online and on-ground courses will be different.

Assessing a course involves an examination of the course design and organization, the course syllabus, the integration of online and face-to-face components, and other issues of instructional design. Similar to a face-to-face course, learning outcomes must also be measured. Here you’ll find templates, rubrics, presentations, and actual examples from model courses to help you plan your assessment process.

Linking Assessment and Online Learning

Online learning has been expanding rapidly in availability and popularity in U.S. colleges and universities in the past two decades, yet owing to the complicated nature of learning, linking online education to the assessment of student learning outcomes has been largely overlooked. According to the National Institute for Learning Outcomes Assessment (NILOA), “online education is multifaceted, sophisticated, and almost ubiquitous.  Assessment, meanwhile, has developed from the ad hoc use of available standardized tests like the GRE and the ACT to the deployment of a powerful array of instruments, portfolios, and rubrics.”

NILOA suggests that a wealth of data is available through learning management systems similar to Blackboard where “online learning management systems… harvest data not just on what students learn but also on students’ every learning activity.” With the potential to obtain a tracked and stored electronic record of everything a student does as they navigate an online course, Blackboard’s tools enable faculty to gain an understanding about student learning behaviors to improve instruction.

In this NILOA Occasional Paper, Matthew Prineas and Marie Cini argue persuasively, linking connections between online education and learning outcomes assessment, and illustrating that “the mediated settings provided by online education have the potential to significantly improve assessment and its capacity to improve teaching and learning”. While making sense of all these data will continue to be a challenge, data mining techniques and what Prineas and Cini term “data analytics” are already tools guiding improvement.

Faculty Training for Online Learning

Creating an online environment that encourages effective interaction and offers authentic learning activities can provide assessments that are naturally occurring phenomenon of online collaboration, as these two factors work together to help promote student engagement and alignment of objectives. One-to-one and one-to-many collaborations (discussion boards, blogs, wikis, journals, e-portfolios),  as well as synchronous interactions (chat, video conferencing, white boards) facilitate measurable demonstrations of student understanding.

To integrate assessment of student learning into workshops at your college or university system the resource entitled Effective Grading: A tool for learning and assessment enables faculty developers and faculty to delve past the concept of assessment and grades as the teacher-based assessment of isolated artifacts and discrete course grades by illustrating methods by which grading can be seen as a powerful tool of student learning and motivation.Authors Barbara Walvoord and Virginia Johnson Anderson (1998, 2009) describe grading as “a process by which a teacher assesses student learning through classroom tests and assignments, the context in which good teachers establish that process, and the dialogue that surrounds grades and determines their meaning to various audiences.” Grading in this context becomes a process of “matching the test or assignment to the learning goals of the course, establishing criteria and standards, helping students [to] acquire the skills and knowledge they need, assessing student learning over time, shaping student motivation, feeding back results so students can learn from their mistakes, communicating about students’ learning to the students and to other audiences, and using results to plan future teaching methods.” (Walvoord, Fassler, 1998)  [1]

Blackboard tools for Assessment     

An online course or content management system (CMS) like Blackboard allows faculty to engage students in Hybrid or Asynchronous online sections with multimedia content using formats like imported images, PowerPoint, podcast audio and video lectures and demonstrations. In addition, faculty can organize their students individually or in student groups to conduct asynchronous discussions via discussion boards, complete weekly blog or journal entries, and develop subject-specific wikis. Graded assignments allow faculty to administer quizzes and tests, and enable students to submit papers and assignments electronically so faculty can grade and provide other forms of feedback easily without fear of losing student papers. Item submissions are dated and organized by students as well as the number of attempts they’ve made.

Within a given online or hybrid course, an individual faculty member has access to a wealth of quantitative and qualitative data relating to student participation and engagement: number of words posted, total number of posts, average length of posts, and the text of every student comment or question from the first week’s discussion and throughout the course.

Additional kinds of data that can be gathered by faculty through CMS software and Blackboard include how frequently students log in to their courses; even the length of time they spend on given tasks such as group work and taking self-directed quizzes.

[1] (1998, 2009), Walvoord, B.E., Anderson, V.J., Effective Grading: A tool for learning and assessment. 1st Ed.

Links and Documents to Course assessment:

Articles About Formal Assessment

• Peer Observation

  November 17, 2012Maria Pagano
  The link takes you to the School of Professional Studies Online Design and Interaction Guidelines checklist SPSOnlineDesignandInteractionGuidelines09-01-14 SPS Online Peer Observation Form Lehman College Department ObservationForm-1

• Model of Assessment: AC230 Hybrid Course — YouTube videos, including class assignments and a rubric

  July 5, 2012Karen Lundstrem
  The following YouTube videos are the presentation of a course model by Dr. Wenying Huang-Stolte. They include information about the course, detailed assignment information, and the rubric for grading. Part 2 provides actual assignments. Online Assessment part 1: http://www.youtube.com/watch?v=gzYzFRFg5fE Online Assessment part 2: http://www.youtube.com/watch?v=EfRff2UwogQ Here is a shorter PowerPoint presentation in less detail: Assessment in a Hybrid Course and Samples

• BCC Online Course Development Checklist

  September 10, 2012Laura C Broughton
  Bronx Community College has an intensive 6-month online course development program using Blackboard as the learning management system.  Materials provided to the faculty participants have been developed collaboratively by Dr. Howard Wach, Dr. Charles Alston, and Dr. Laura Broughton, with feedback from Dr. Giulia Guarnieri, Prof. Moronke Oshin-Martin, and Mr. Albert Robinson. The BCC Online Course Development Checklist provides guidance on the most important components of an online course.

• How to Read a Teaching Evaluation

  November 18, 2012Maria Pagano
  Queens College has a useful site for faculty who are interested in better understanding the process and purpose of student teaching evaluations.  While the site includes information that is specific to Queens College Faculty, the site also contains links to articles  from the Chronicle of Higher Education on “How to Read a Student  Evaluation”, which faculty might find useful.

• BCC Hybrid Course Site Review Evaluation Rubric

  September 10, 2012Laura C Broughton
  Bronx Community College has an intensive 6-month online course development program using Blackboard as the learning management system.  Materials provided to the faculty participants have been developed collaboratively by Dr. Howard Wach, Dr. Charles Alston, and Dr. Laura Broughton, with feedback from Dr. Giulia Guarnieri, Prof. Moronke Oshin-Martin, and Mr. Albert Robinson. The BCC Hybrid Course Site Review Evaluation Rubric is used by faculty mentors when advising faculty participants on the development of their courses.

• Rubric for Online Instruction from Cal State Chico

  July 6, 2012Karen Lundstrem
  The Rubric for Online Insruction from Cal State Chico provides a list of criteria to evaluate online instruction. The site also contains links to quality courses in Blackboard, instructional design tips, as well as the rubric in PDF format.

• Online Course Assessment Rubrics from Michigan State University (and Quality Matters)

  July 11, 2012Karen Lundstrem
  Michigan State University’s Online Course Assessment Rubrics list contains links to assessment plans and rubrics from colleges offering major online programs. In addition to links to other college’s plans, rubrics from highly respected sources, including the latest plan from QUALITY MATTERS, are found here.

• Series of articles on online assessment

  July 11, 2012Karen Lundstrem
  The Online Journal of Distance Learning Administration is a peer-reviewed journal that primarily serves adminstrators and managers of online programs. Within this online journal, you can find a range of articles on assessment, covering everything from course to institutional.

• Michigan Community College Association Virtual Learning Collaborative Program Assessment Plan

  July 11, 2012Karen Lundstrem
  The Michigan Community College Association Virtual Learning Collaborative’s Program Assessment Plan provides a rubric that guides individuals implementing online programs. The plan covers what to expect during the early stages of evaluation and what to look for as a program matures. A successful program should meet most of the requirements articulated in the mature phase of the plan.

• Assessment of Hybrid and Online Courses by Professor Ellen Smiley

  July 5, 2012Karen Lundstrem
  Dr. Ellen Smiley of City College and SPS presented what is known at CUNY and in the United States about distance learning, including the demand for online learning and how well online learning stacks up against traditional face-to-face teaching. The following is from a presentation she made to the CUNY community on February 24th. Smiley Assessment Council 2 24 12 2