08/01/2015

Formative Assessments as Early Intervention Tools in Academic Environments

By Melanie Adams and Jennifer Blythe

At Florida Atlantic University (FAU), career counselors teach several different sections of a Career & Life Planning course that supports students with major and career indecision through identification of future career goals.

When the course was developed, a summative assessment model relying on a pre- and posttest evaluation was used to gauge student progress. The summative assessment, based on a career decision-making self-efficacy scale (CDMSES), measured students’ confidence in the areas of self-appraisal, occupational information, goal selection, planning, and problem solving, based on individual student responses. Most students’ scores tended to increase between the pre- and posttest, but this method failed to offer usable feedback during the 16-week course. More recently, we found a way to respond to student development needs in “real-time,” using formative assessments.

The simplest definition of formative assessment comes from James Popham (2008): “formative assessment is a planned process in which teachers or students use assessment-based evidence to monitor student learning and provide ongoing feedback” (p. 6). Educators employ formative assessment regularly in the K-12 school system. However, since many career counselors do not come from instructional backgrounds, this approach to evaluating programs and services may be unfamiliar and underutilized.

Process

Collaborating with a colleague that specializes in building and understanding assessment modules, we designed a formative assessment and implemented it in our Career & Life Planning courses during the 2014-2015 academic year. Each course was divided into two phases: self-knowledge (understanding one’s interests, values, and skills) and options-knowledge (understanding specific occupations, college majors or jobs). The formative assessments included five questions for self-knowledge and three for options-knowledge.

Questions were designed to directly relate to career concepts in the course material. For example, students should learn the different roles they play in their lifetime and how their career fits into that, so one question would read, “According to Donald Super and the Career Life Rainbow, what is the definition of career?

a) A set of work tasks that one usually performs on a specific job.

b) Specific parts we play in life that are defined by our relationships and the activities we pursue.

c) The sum total of the activities in all the roles one plays at any given time.

d) The number of hours per week devoted to a role.”

When building a formative assessment, instructors can determine whether or not they want to grade responses for accuracy or completion. We chose to grade on completion only, as this allowed students to answer freely without the pressure of individual answers affecting their grades in the class. When students selected wrong answers, the assessment would alert the instructor of a disconnect with the lesson at hand, but students would not lose points.

Whereas, the CDMSES pre and posttest focused on students’ perception and overall ability to execute behaviors, such as decision-making, goal selection, or career readiness, which are scored by an increase of confidence versus right or wrong. The assessment process allowed students to answer at least two questions incorrectly before being “flagged”. Once “flagged,” the instructor could determine an appropriate follow-up method to provide additional assistance. For example, our instructors would send e-mail alerts to students inviting them in for individual appointments. The mid-course timing of such an intervention was viewed as crucial, for if the student doesn’t understand self-knowledge, he/she will not be able to effectively apply it to the options-knowledge phase of the course. Based on this shared understanding, students were receptive to e-mail alerts and readily came in to discuss their difficulties with the content.

Implementation

FAU uses an assessment tool from the vendor Baseline that has a capability called the Student Response System. This allows students to answer questions in real-time and captures data for instructors. Students use smart technology to respond to questions in the classroom and view responses immediately. The instructor can then determine which individuals may need follow-up and/or which topics need to be explained further to the class at large. While this has been a great choice for us, there are many other systems that could be used to collect real-time student feedback.

As mentioned earlier, five formative assessment questions were administered in the first unit (self-knowledge) of each Career & Life Planning class. Students that completed fewer than three questions successfully would be offered one-on-one career counseling. During the second unit of the class (options-knowledge), three additional formative assessments were administered. If a student scored “ok” on at least two assessments, they would be viewed as grasping the concepts. However, if they scored unfavorably on more than two assessments, they would be viewed as struggling with the material. To date, all of the interventions happened before the options-knowledge phase, and the results of these later responses were instead used to gauge concept understanding for the entire group.

Results

Formative assessment results were recorded and analyzed weekly, and were uniquely helpful in identifying struggling students during the course of the semester. Students were later given a posttest to gauge improvements in self-efficacy and an end-of-course evaluation to gauge satisfaction. In fall 2014, 91% of students strongly agreed or agreed that the assessment tool was easy to use. In spring 2015, 87% of students strongly agreed or agreed that the formative assessment activity helped them better understand what they did/did not know. Based on pre- and posttest data, the formative assessments contributed to an overall increase in self-efficacy; this is likely because any inconsistencies or misunderstandings were detected early enough to correct and improve.

Conclusion

In a number of settings, formative assessment can help career course instructors identify concepts or ideas that students are struggling to understand, individually or collectively. Instructors can then adjust lessons and techniques, and offer targeted academic support. Adding formative assessment to career courses can also increase student engagement and retention, because students are being evaluated throughout the semester and instructors can better identify and remedy issues early on.

Reference

Popham, W. (2008). Transformative assessment. Alexandria, Virginia: Association for Supervision and Curriculum Development.

Melanie Adams, M.S. is a certified Global Career Development Facilitator and currently works as a Career Counselor in the Career Development Center at Florida Atlantic University (FAU). Melanie works closely with the University’s ACCESS (Academic and Career Enhancement for Student Success) program and teaches multiple sections of the Career & Life Planning course every semester for the ACCESS program, which is designed for at risk students with major and career indecision. Melanie recently presented this topic at the 2015 Global Career Development Conference in Denver, CO. She can be reached at madams39@fau.edu.

Jennifer Blythe, M.S., is the Director for Internships and Co-ops for the Career Development Center at Florida Atlantic University (FAU). In this role she works with Internship and Cooperative Education students and employers, manages FAU’s Cooperative Education/Internship Program, cultivates new employer partnerships, creates and implements program initiatives, assists students in career pursuits, and collaborates with internal and external constituents. Jennifer’s assessment experience includes learning outcomes and program evaluation. Jennifer can be reached at jblythe@fau.edu.