Introduction

Building cogent, effective, and efficient credential programs takes deliberate and extensive planning. One must first define degree types (Associates, Baccalaureate, etc.), and how these evolve as a student progresses through these levels. The interplay between majors, minors, and tracks/concentrations, Honors, the Liberal, and General Education programs should be elucidated. Learning goals should be defined at the institutional, college/school, and departmental levels, and then used, in combination with discipline specific needs, to define learning outcomes for programs and courses. These include both curricular and extra-curricular expectations.

A curriculum of courses must be designed that addresses program learning outcomes within a progressive matrix of required, elective, pre and co-requisites. This complex design process uses tools such as Bloom’s Taxonomy, the Degree Qualification Profile, Tuning, and curriculum mapping to achieve optimal program design. This process starts by writing learning goals and outcomes that are then gathered into cogent programs that build students’ capabilities over time.

The Macro Level – Degrees, Majors, Liberal and General Education

Figure 1 illustrates how the learning expectations in the major (blue) increase as one progresses through the Associate, Baccalaureate, and Master’s degree levels. Concurrently, the liberal education learning expectations (yellow) decrease in an inverse relationship with major requirements. The liberal education expectations continue through the Baccalaureate and Master’s degrees as values, ethics, civic engagement, and lifelong learning, etc. should be a component of every degree program.

General Education learning expectations, shown by the dotted box, are the subset of the Liberal Education expectations that focus on skills that prepare the student for the courses in the major. The requirements of the Honors programs are shown as grey boxes that represent the expanded or additional learning expectations inherent in these programs. Honors programs will be addressed later in the article, along with a more thorough examination of the Liberal and General Education programs. Managing this complex interplay of learning expectations requires clarity, coordination, and planning.

Explicitly Defining Learning Expectations

“As a new Assistant Professor in my first academic position back in the 1990s, I knew nothing of learning outcomes or curricular planning. I was assigned courses to teach, studied the syllabi of faculty who had taught the courses before me, examined a selection of textbooks on the topic, and put my course content together. Occasionally, my mentor would chat with me about an issue with the preparation of students for his course, which was later in the sequence than mine, and I would adjust my content. I believed that I was quite good at what I did, but in retrospect, there were many ways in which I could have been better. While I had an intuitive sense of what I wanted to teach, I was not explicit about what I wanted the students to learn. I also know I repeated a lot of material unnecessarily because I was unsure what the students had learned prior to arriving in my courses – I did not have a strong sense of how my courses fit into the overall program. I know I would have really enjoyed conversing with my colleagues about what we wanted the students to learn, and how we might most effectively design our curriculum to meet our goals, but that never happened. Had it happened, the conversation would have been difficult because neither I nor my colleagues knew how to create clear statements about those learning expectations. To be honest, we were not even aware that such statements were necessary”⁽¹⁾Graham Glynn Ph.D. on his experience with course design as an assistant professor..

Most faculty know intuitively what they want the students to learn within their own courses, based on their knowledge of the field, courses that they have taken, textbooks that are the foundation for the course, and so on. Since teaching is all about communication, it is important that we have a concise way to convert this intuitive sense to explicit statements that enable the students to understand and work toward achievement of these expectations. It is also important for the faculty to have these statements for their own clarity of purpose, and to facilitate useful conversations among themselves about them, and how they contribute to the programs. “Developing these explicit statements for my own courses was one of the most eye-opening experiences of my teaching career – so much so that I believe having them is the most important characteristic of a good course.”⁽²⁾Graham Glynn Ph.D. on his experience with course design as an assistant professor.

These explicit statements come in two forms. Goals that are broad statements expressed as general aims or purposes of education. According to Linda Suskie “they describe how students will be different because of a learning experience. Effective goals refer to a destination rather than the path taken to get there – the ends rather than the means….[For example,] A faculty member’s real goal is not that students will write a term paper but that they will become effective writers”⁽³⁾Suskie, L. (2009). Assessing student learning: A common sense guide (2^nd ed.). San Francisco, CA: Jossey-Bass.. Student learning outcomes (SLOs) are statements that are more specific than goals, describing faculty intentions about what students should know, understand, and be able to do with their knowledge when they complete a course or graduate from a program. SLOs are what students take with them from a learning experience. Because SLOs are designed to be measurable, they provide mechanisms to help determine if the student learning goals have been accomplished.

SLOs should begin with an action verb that reflects the appropriate level of Bloom’s taxonomy that they are designed to achieve. This taxonomy defines a progression through increasingly demanding levels of intellectual skills, as shown in the diagram, and is a useful framework upon which to build learning expectations for courses at progressively higher levels (100, 200, etc.).

Creating Goals and Outcomes

Institutional goals and outcomes are usually developed by a special taskforce, approved by the faculty Senate, and at some institutions, may require a vote of the entire faculty body. Ideally, program-specific learning goals and outcomes are created by as many of the faculty from any given department as possible, working collaboratively to define and design each credential offered. It is important that this be a group process so that as many of the instructors as possible feel ownership of the program, and each has a clear understanding of the program’s SLOs and where they are aligned with the course-level student learning outcomes (CLOs) within its constituent courses. To further ensure the cohesiveness and clarity of the program, individual CLOs should also be the purview of the faculty body and not, as has traditionally been the case, individual instructors. A course must first and foremost serve the needs of the program(s) it supports, and not the interests of its instructor. However, individual instructors may feel constrained and rebel if the institution is too rigid with these expectations. A good rule of thumb is to use the 80/20 rule – i.e. at least 80% of the CLOs are defined by the faculty body and unchangeable by individual faculty, and the remainder be defined by each course section’s individual instructor. This has the added advantage of ensuring consistency across multiple sections of a course taught by different instructors.

Credential is deliberately underlined above because the learning expectations should be clearly defined for every certificate, minor, Associate degree, Baccalaureate degree, Master’s degree, Doctoral degree, and for any specialist tracks/concentration the students can take within these programs. The students taking each of these credentials have the right to have clearly defined learning expectations and understand how they are differentiated from each other. In addition, the faculty should have a clear understanding of the expectations.

Figure 2 illustrates the relationships between different levels of goals (shown in boxes) and outcomes (shown as ovals). Institutional goals can be broken down into curricular and extra-curricular. The curricular goals predominantly, but not necessarily exclusively, lead to the Liberal/General Education learning outcomes such as literacy and numeracy for all graduates of the institution. College/School learning goals incorporate, or are significantly aligned with institutional curricular goals, but often expand the goal set. For example, a College of Health Sciences may have a set of common learning goals (interdisciplinary team work for example), that students in other colleges would not have. Similarly, disciplinary/departmental goals incorporate or are aligned with the College/school goals. It is at the program level that these sets of nested or aligned goals are first converted to more specific and measurable SLOs. The exception in this hierarchy is that Institutional learning goals directly drive the learning outcomes for the Liberal / General Education program since these are expected of every graduate of the institution. Any individual course’s learning outcomes could serve more than one program. In Figure 2, Course A serves both Program 1 and Program 2. Course B serves both Program 2 and the Liberal/General Education program. This means that the learning outcomes in these courses must incorporate CLOs from two programs.

Institutional Learning Goals and Outcomes

The conversation that faculty need to have about defining the learning outcomes we want our students to achieve is aided by great work that is happening at the national level. Work such as the Degree Qualification Profile (DQP) project, being led by the Lumina Foundation, which has defined a broad set of ratcheted learning expectations at the Associate, Baccalaureate, and Master’s degree levels. The DQP is a great tool to help each of our institutions define or refine institutional learning goals and outcomes, especially the Liberal/General Education curriculum. In Text Box 1, a DQP executive summary derived from materials on the Lumina and NILOA web site is provided. This has been used this to brief deans and other stakeholders to encourage its adoption. Accreditation organization materials are another good source of institutional learning goals and outcomes, for example the Middle States Gen-Ed-Competency-Grid.

Another interesting approach to defining institutional learning goals is the conceptual framework called the T-Shaped Professional. This model adds competency in systems as a broad requirement for all students, both as a component of their major, and for at least one system outside the major. For example, a history student might study water management systems. This additional systems approach is intended to help students bring a unique perspective to their chosen field.

Liberal/General Education

One of the major organizations that advocates for a strong Liberal/General Education curriculum for all students is the American Association for Colleges and Universities (AAC&U). This organization has had a significant influence on thinking about the learning expectations we should have for all our students, especially through its five-day institutes on General Education and Assessment.

AAC&U defines Liberal Education as “a philosophy of education that empowers individuals with broad knowledge and transferable skills, and a strong sense of value, ethics, and civic engagement”. In Figure 1 above, General Education (Gen Ed) is defined as a subset (dotted purple line in) within the Liberal Education curriculum. The Gen Ed outcomes can be considered as those that all students need to fulfill to be successful in their major courses, critical reading and thinking, writing and communication skills, etc. It is therefore an advantage to have students achieve these outcomes in advance of as many major courses as possible. This should be balanced with engaging students in major courses as soon as is feasible to promote retention.  A sample set of Liberal Education learning outcomes,  developed using the DQP as their basis, is provided in the file sharing library.

The Transition to College and Student Success Outcomes

The Gardner Institute’s Foundations of Excellence (TM) self-study process guides examination of students’ transition into higher education and the factors affecting their success in their first year. Self-studies conducted at Stony Brook University and Mercy College identified serious student retention issues based on a lack of student knowledge about the institution and its expectations of them, and a lack of skills essential for student success. Far too often the academic community has either ignored these issues or handed them off to Student Affairs to deal with. While it may be valid for student affairs to take responsibility for providing information about, and helping the students to navigate the institution, the responsibility for developing the knowledge and skills needed to be an effective learner should fall squarely within Academic Affairs.

At Mercy College, a Transition to College taskforce (see this groups charge in Text Box 2), building on the findings of one of these self-studies, developed a comprehensive set of learning outcomes to address these deficiencies which is available in the file library, and defined opportunities for addressing them within the orientation process or within institutional learning goals and outcomes. Since these institutional learning outcomes must be addressed early in the curriculum to ensure student success and retention, it is recommended that they be incorporated into, and taught as part of the General Education curriculum, either within existing courses or in a special transition to college course. There are great resources available to help develop these goals and outcomes. For example, the Council for the Advancement of Standards in Higher Education (CAS) has produced a set of learning outcomes for many of the student affairs operational areas.

College/School and Program Learning Goals and Outcomes

Once the institution has defined the learning goals expected of all students, each college/school should develop specific learning goals that it would expect of its graduates in addition to, and aligned with, the institutional goals. This means that by achieving the college/school goal the student would essentially achieve the institutional goal. A College of Social & Behavioral Sciences, its departments and courses, for example, would have a very different set of goals and outcomes (sample set shared in the file library) than those developed by a School of Business.

Each department/program within each college/school should then develop its set of learning goals, incorporating or aligning them with institutional and college/school goals, and develop a set of program learning outcomes based on this combined set. There are numerous professional organizations, many of which represent specific disciplines, which have developed sets of learning goals and outcomes that can act as a starting point for this process. The Tuning Academy, a national faculty-led initiative to define specific learning outcomes for an ever-growing set of disciplines, is also a wonderful source of these outcomes. Each program specific student learning outcome (PLO) should be written at the highest Bloom’s level (see above) appropriate for the degree. For example, outcomes for a Master’s program may require a great deal of evaluation (level 6), at the Baccalaureate level they may mostly require analysis (level 4), whereas at the Associates level, comprehension (level 2) may be the main focus.

Honors Program Impact on Learning Outcomes

Honors programs usually attract the best and brightest students who are desirous of additional challenges, either in the form of greater depth or breadth of learning. These additional challenges are usually provided in special Honors sections of the General Education curriculum for an institution-wide program, or as Honors sections in major courses for a discipline-specific program. Therefore, from a learning outcomes perspective, honors courses should specify additional learning outcomes (increasing breadth) or should rewrite the outcomes of existing non-honors versions of the courses to reflect a higher Bloom’s level (increasing depth).

Extracurricular Learning Outcomes and their Interaction with Courses

It is usually understood when we talk about the “curriculum” that we mean the formal program of courses at an institution. There is some confusion about the meaning of co-curriculum and extra-curriculum. Co-curriculum in this article is defined as activities that re-enforce the curriculum. For example, tutoring is a co-curricular activity. A field trip to a museum to examine artifacts specifically relevant to the topics in a history course is another example. The co-curriculum therefore does not have its own set of goals and outcomes – it supports those developed for the curriculum. Extra-curricular activities are outside of, or in addition to the curriculum. As such, they represent opportunities to increase student learning in areas that may not necessarily be defined within the formal curriculum. Leadership opportunities in student clubs and organizations are a good example of this.

In a learning centered institution, we should be deliberate about defining student learning no matter where it occurs, and therefore extra-curricular learning outcomes should be specified. This is most commonly performed by Student Affairs. Once specified, there is an opportunity for Student Affairs and Academic Affairs to have a dialog about how their respective sets of learning outcomes can be mutually supported. For example, students given leadership opportunities through Student Affairs might be required to take a leadership course, so they are getting both the theory and practical experience. Extra-curricular learning outcomes are therefore shown as influencing CLOs in Figure 5 below.

Ramping Up Learning Expectations Through Course Sequences

It may be unreasonable to expect a student to be able to achieve learning outcomes written at the Bloom’s “analysis” level in a single course without any previous preparation. Additional courses may therefore need to be developed with learning outcomes written at lower Bloom’s levels, the comprehension and application levels for example, to progressively build the student’s capabilities. It is helpful to define three layers of progression. The literature suggests calling these layers Introducing, Reinforcing, and Mastery. These can be problematic because of the colloquial use of two of these terms. For example, nearly every faculty member who has some sort of writing assignment in their course will claim that they are reinforcing writing skills even though they have no CLO related to writing. Another issue is that mastery is generally considered as occurring after many years of practice in a field. The terms “Introducing, Broadening, and Fulfilling” are therefore recommended as the three categories into which the CLOs can be layered, as illustrated in the lower grid in Figure 3. CLOs do not need to be developed for every one of these levels – it is a judgment call based on the number of Bloom’s levels that need to be traversed between the students’ entry level and the top level assigned to the program. However, fulfilling is always required (we want the student to achieve the outcome). If a second level is needed it should be introductory (one can’t broaden an outcome if it hasn’t been introduced). Broadening is therefore only used if a progression through all three levels are needed.

Patricia Aceves, the director of the Faculty Center at Stony Brook, discusses the depth to which CLOs should be written and to what degree students can or should participate in this process. She also addresses the use of outcomes to guide course evaluations and how outcomes can be written to the different cognitive expectations of a course. http://tinyurl.com/yt-Aceves1

Ideally, but not always, certificates and lower level degrees should address a subset of the outcomes defined for a program so that the student can build on top of each. For example, a certificate could be part of an Associate’s program, which in turn is part of a Baccalaureate program. Specialist tracks/concentrations, such as an accounting degree with a focus on tax accounting, could be defined by additional learning outcomes. Alternatively, since all business students need some tax accounting, the specialist track/concentration could be defined by rewriting the base tax accounting learning outcomes at a higher Bloom’s level.

Putting It All Together – The Curriculum Map

It is much easier to construct a new program than to re-construct an existing one, as existing course sequences and ownership issues do not cloud the process. Faculty may feel threatened when asked to create a design for their own programs for the first time. An inquiry-based exercise based on the creation of a new or fictitious program, may help faculty understand the curriculum design and development process, as it bypasses these issues. Figure 3 and Figure 4 illustrate this process, whereas Text Box 3 provides a guide to this exercise. Figure 3 is essentially a curriculum map showing how the institutional, college/school, and program goals drive down to PLOs and CLOs. The figure also shows the CLOs grouped by color into proposed courses.

Click the figure to see an enlarged view.

When gathering sets of outcomes into courses (like colors) as shown in Figure 3, it is important to consider the sequence of learning outcomes and their Bloom’s Taxonomy level. In general, the higher the average Bloom’s level of the outcome set assigned to a course, the higher the course level designation should be i.e. 100 level, 200 level, etc. Once outcome sets (courses) are aggregated, assigned a level and assigned co and pre-requisites, they should be mapped to see if the sequence works. Figure 4 illustrates a course sequence map developed from the data in Figure 3. There are problems within this sequence map that have deliberately not been addressed to illustrate some of the thought processes which should be applied. For example, Course 5 which is designated at a 200 level has Course 4 at a 300 level as its pre-requisite. The level assigned to Course 5 or the set of learning outcomes assigned to it should therefore be reconsidered. There are also two instances (6 & 7, 2 & 3) where co-requisite courses are at different levels, which may be OK because they only differ by a single level, but this may need to be examined further. In addition, the two top-level courses (8 & 9) have no pre-requisites. Notice that Course 1 has no follow-on courses, which may be OK if this is deliberately the end of faculty expectations for this vein of the curriculum. If this is not the case it may be an issue. Given all these concerns, the assignment of learning outcome sets and levels to the courses in Figure 3 may need to be reconsidered and a new course sequence generated. This process may need to happen more than once until an acceptable curricular design is achieved.

A different approach to curriculum mapping may need to be taken for an existing program, especially if CLOs have not yet been developed or do not yet directly relate to the PLOs. Rather than redesigning the program from the ground up it may be useful to implement a phased approach. In Phase 1, creating a map as shown in Table 1 that correlates each program learning outcome with the course(s) in which it is introduced (I), broadened (B) and fulfilled (F) is suggested. Program outcomes are listed across the top of the table and existing courses are listed, ideally in course level sequence, down its left-hand side. A template for this activity is provided in the file sharing library.

Next, indicate with an * the courses in which it would be most appropriate to measure the student achievement of the program outcome. Since this measure of achievement is dependent on any prior introduction and broadening of the outcome, the assessment should provide adequate data upon which to make decisions about curriculum and course design. However, if problems are uncovered with the students’ ability to fulfill an outcome, additional measurement within the courses in which it is introduced or broadened may need to be conducted to uncover the source of, and potential solution to the problem. This mapping exercise is based on a general sense of where the program outcomes are addressed. A more explicit approach is to identify or create CLOs that directly correlate with the program outcome, which ensures that the program outcomes are explicitly being addressed in the courses to which they are mapped.

It is also useful to look at curriculum maps from a course centric perspective. Courses can simultaneously serve multiple programs, including more than one major, the Liberal/General Education and Honors programs, and even extracurricular learning outcomes as shown in Figure 5. The course learning outcomes can also serve other program outcomes at a variety of levels with a single course, potentially introducing, broadening, and fulfilling different program outcomes. The relationship between the programs and the courses is bidirectional, with program outcomes informing course outcomes and course assessment data in turn informing achievement rates of program outcomes. This can present challenges teaching the course as discussed in the following faculty interview.

Courses that serve multiple programs tend to have enrollees with a wide variation in background preparation and interest in the subject matter. In addition to teaching courses for majors, Scott Sutherland, a professor at Stony Brook University, teaches math courses that act as service courses for many departments within the University. There are unique challenges to teaching these courses since students are often only present to fulfill general education requirements and mathematics is not a specific interest for them. Scott discusses the pros and cons of integrating discipline-specific examples into his classes. http://tinyurl.com/yt-Sutherland

In reality, each instructor has strengths and weaknesses in content expertise, and passions about areas of the subject matter. Defining CLOs solely from program outcomes can be too restrictive and lead to a sense of excessive control. The 80/20 rule is therefore a good rule of thumb to use – i.e. that 80% of the course outcomes be defined by program needs and 20% be left to the discretion of the individual instructor in each course. This provides a good balance between serving the needs of the programs and providing opportunities for flexibility and individual creativity. Are there courses that do not serve any program and are therefore free from imposed program outcomes? Perhaps the least restricted would be elective courses within a program. However, the program faculty should have a sense as to why a course is eligible to be an elective in the program and therefore should have learning outcomes defined for it. Each program might therefore have core program outcomes, and a set of elective outcomes from which the students select a requisite number.

The Benefits of Well-Planned Curriculum

By engaging in the processes described in this article and institution should be able to:

• Create rigorous programs of study that build the students’ knowledge and intellectual skills in a progressive manner.
• Create efficient curricula with necessary but well-planned repetition in content, balanced with the need to introduce new material.
• Reduce the number of extraneous courses taken by students in its programs, and either create opportunities for additional courses in the major, or opportunities for broadening the students’ education through additional electives.

Bookmark0

Defining What, Where, and When We Want Students to Learn by Graham Glynn is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Defining What, Where, and When We Want Students to Learn by Graham Glynn is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Categories: Chief Academic Affairs Office Staff, Dean’s Office Staff (Deans, Executive Deans, Associate/Assistant Deans, etc.), Department Chair Office Staff (Chairs, Assistant Chairs, Program Directors, etc.), Featured, Office Director’s Staff (Director, Assoc./Assist. Director, Coordinator, etc.), President’s Office Staff, Vice President Student Affairs Staff (VP, Assoc./Assist. VP, etc.)

Tags: Bloom's Taxonomy, College/School Learning Goals, Course Learning Outcomes, Course Sequencing, Curriculum Design, Curriculum Mapping, Degree Qualification Profile, Extracurricular Learning Outcomes, General Education, Honors Program, Institutional Learning Goals, Liberal Education, Program Learning Goals, Program Learning Outcomes, Student Learning Outcomes, Student Transition