AAC&U VALUE Rubrics Pilot
This pilot project is intended to identify if common rigor in student learning assessment can be achieved using AAC&U rubrics. If so, scores reported on the Annual Progress Reports can be reliably combined to provide an indication of university-wide student achievement in written communication, oral communication, and critical thinking.
Who could be involved
The Office of Assessment would like to identify programs that currently use rubrics to assess Critical Thinking, Written Communication, and/or Oral Communication for program outcomes assessment. Please contact the Office of Assessment if your program would be willing to discuss the possibility of adapting the AAC&U rubrics for Critical Thinking, Written Communication, and/or Oral Communication to pilot use as assessment measures of student learning will be included.
1. Identify programs that use rubrics to assess one or more of the following:
2. Communicate with programs to:
- compare AAC&U Value Rubrics to currently used rubrics
- identify commonalities and editing options
- confirm agreement to implement
3. Collect student achievement data following the use of the rubrics.
4. Gather feedback from faculty to identify challenges, problems, and suggestions.
Faculty involved will provide feedback on ease of implementation, reliability of implementation across programs, and rubric validity. Scores from across all programs in the pilot will be combined to identify cumulative achievement scores. If reliability is confirmed, we hope to encourage greater implementation across programs of the AAC&U rubrics for assessment in all areas of student learning.
The process of reviewing the assessment measures implemented for written communication, oral communication, and/or critical thinking can lead to more effective student learning assessment and better data that could lead to program improvements.
Why AAC&U VALUE Rubrics?
These particular rubrics were selected for this pilot because they are the areas on which the Kansas Board of Regents is currently requesting data for levels of student achievement. National accountability expectations have also provided the option of reporting student achievement scores using the AAC&U rubrics if a reliable plan of implementation is confirmed. Internal measures such as the AAC&U rubrics would be more useful to colleges and programs than the national standardized tests like the Collegiate Learning Assessment (CLA) or the Collegiate Assessment of Academic Proficiency (CAAP) as the rubrics assess learning and application within the context of the major.
What are the VALUE Rubrics: In 2007, the Association of American College and Universities (AAC&U) launched Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics in fifteen specific areas of learning: civic engagement; creative thinking; critical thinking; ethical reasoning; foundations and skills for lifelong learning; information literacy; inquiry and analysis; integrative and applied learning; intercultural knowledge and competence; oral communication; problem solving; quantitative literacy; reading; teamwork; and written communication. (a sixteenth VALUE rubric focused on global learning was released in 2013). Having been tested for validity and reliability, they can be readily adapted to accommodate the language used to frame learning goals for individual programs. The VALUE rubrics have been embraced by all the regional accrediting bodies as one acceptable approach for institutions to use in assessing student learning. The rubrics represent an alternative to standardized testing, providing more robust and nuanced information on areas of strength and weakness in student learning and across a wider range of outcomes.
Indicators of Achievement: The labels represent levels of achievement moving from entry level moving toward more complex and sophisticated demonstrations of learning that result in baccalaureate credentialing. This provides an assessment measure for both formative and summative assessment of student learning for achievement and progress within a course, program, or across general education. In relation to the descriptors, the rubrics are meant to be adapted to reflect the content and expectations of a program or demographic. The criteria for each rubric represent the most commonly expressed dimensions of learning. On many campuses, the language has been modified to reflect local terminology. In some cases, dimensions or criteria have been added in order to represent particular aspects of how the specific learning outcome is manifested.
Rubric modification: The VALUE rubrics represent the most common characteristics or dimensions of student learning for the assessed student learning outcomes. The precise terminology used to describe learning expectations is likely to differ from program to program and should be adapted to accommodate the nuance of these variations. The main purpose of rubric modification is to increase faculty and student understanding of the state criteria so that the resulting assessments will accurately reflect actually learning as it is framed for a particular assessment activity. Additional performance criteria and associated levels can also be added.
Campus-level Calibration: Adoption and use of the VALUE rubrics on over 3,000 campuses have consistently demonstrated high levels of agreement among evaluators. Face-validity is already strong overall but will be confirmed for use in coursework assessment during our pilot of rubric integration. In addition, a national reliability study and several consortia of campuses have achieved acceptable levels of reliability in projects focused on one or more of the rubrics. Participants in the pilot will be individually involved with the Office of Assessment in a close reading of the rubric used to identify any ambiguities in phrasing or wording. Participants will be asked to review each part of the rubric carefully, identifying areas for discussion and raising specific questions. Discussions will indicate the extent in uniformity of scoring rigor as a measure of calibration. Further comparison of scoring reliability will be tested through inter-rater scoring following administration of the rubric. Group discussion will reflect on application commonality for each line of the rubric.
Focus group discussion: Following the use of a rubric in the assessment of student learning, the pilot group will be gathered for a focused group discussion on their experience of implementing the rubric(s), observed challenges, perception of rubric reliability, and suggestions for further implementation of the rubrics. We will also check for reliability through inter-rater reliability using examples of student work that you bring to the focus group.
Contact Fred Burrack at firstname.lastname@example.org or 785-532-3429.