Assessment of Learning and
Evaluation Strategies
Eric Oestmann and Joanna Oestmann
INTRODUCTION
This chapter will examine the current literature involved in the assessment of learning and various evaluation strategies; discuss the relevant educational issues and teaching techniques with examples of each provided; identify common problem areas; and offer solutions to these problems accordingly.
LEARNING ASSESSMENT AND EVALUATION TOPICS
The assessment of learning and corresponding evaluation strategies and methods is a complex process. Learning styles/types must be considered first.
Learning Styles/Types
There are three primary modalities for learning: visual, auditory, and kinesthetic (Fig. 33-1). Each student may be dominant in one area, but all areas must be considered in the assessment of learning and evaluation of learning therein.
To assess learning style dominance, the following exercise may be used:
Learning Style Exercise 1
To better understand how you prefer to learn and process information, place a check in the appropriate space after each statement, then use the scoring key that follows to evaluate your responses. Use what you learn from your scores to better develop learning strategies that are best suited to your particular learning style. Respond to each statement as honestly as you can.
531
532 CHAPTER 33 • ASSESSMENT OF LEARNING AND EVALUATION STRATEGIES
Often
Sometimes Seldom
1. I can remember best about a subject by listening
to a lecture that includes information, explana-
tions, and discussion.
2. I prefer to see information written on a chalk-
board and supplemented by visual aids and as-
signed readings.
3. I like to write things down or to take notes for
visual review.
4. I prefer to use posters, models, or actual practice and other activities in class.
5. I require explanations of diagrams, graphs, or
visual directions.
6. I enjoy working with my hands or making things.
7. I am skillful with and enjoy developing and
making graphs and charts.
8. I can tell if sounds match when presented with
pairs of sounds.
9. I can remember best by writing things down
several times.
10. I can easily understand and follow directions on
a map.
11. I do best in academic subjects by listening to
lectures and tapes.
12. I play with coins or keys in my pocket.
13. I learn to spell better by repeating words out
loud than by writing the words on paper.
14. I can understand a news article better by reading
about it in the newspaper than by listening to a
report about it on the radio.
15. I chew gum, smoke, or snack while studying.
16. I think the best way to remember something is to
picture it in your head.
Learning Assessment and Evaluation Topics 533
17. I learn the spelling of words by “finger spelling”
them.
18. I would rather listen to a good lecture or speech
than read about the same material in a textbook.
19. I am good at working and solving jigsaw puzzles
and mazes.
20. I grip objects in my hands during learning
periods.
21. I prefer listening to the news on the radio rather
than reading about it in the newspaper.
22. I prefer obtaining information about an
interesting subject by reading about it.
23. I feel very comfortable touching others, hugging,
handshaking, etc.
24. I follow oral directions better than written
ones.
Figure 33-1 Learning styles 1.
Visual
Auditory
Kinesthetic
534 Chapter 33 • assessment of Learning and evaLuation strategies
Learning Style Exercise 1 Scoring Key
Place the point value on the line next to the corresponding item. Add the points in each column to obtain the preference score under each heading: Often, 5
points; Sometimes, 3 points; Seldom, 1 point. VPS is visual preference score, APS
is auditory preference score, and KPS is kinesthetic preference score.
Visual
Auditory
Kinesthetic
No. Points
No. Points
No. Points
2 ____
1 ____
4 ____
3 ____
5 ____
6 ____
7 ____
8 ____
9 ____
10 ____
11 ____
12 ____
14 ____
13 ____
15 ____
16 ____
18 ____
17 ____
19 ____
21 ____
20 ____
22 ____
24 ____
23 ____
VPS 5 ____
APS 5 ____
KPS 5 ____
If you are a visual learner, by all means be sure that you look at all study materials. Use charts, maps, filmstrips, notes, videos, and flash cards. Practice visualizing or picturing words and concepts in your head. Write out everything for frequent and quick visual review.
If you are an auditory learner, you may wish to use tapes. Tape lectures to help fill in gaps in your notes. But do listen and take notes—and review your notes frequently. Sit in the lecture hall or classroom where you can hear well.
After you have read something, summarize it and recite it aloud. Talk to other students about class material.
If you are a kinesthetic learner, trace words as you are saying them. Facts that must be learned should be written several times. Keep a supply of scratch paper on hand for this purpose. Taking and keeping lecture notes is very important.
Make study sheets. Associate class material with real world things or occurrences.
When appropriate, practice role playing.
Another learning classification system is based on the following categories: concrete experience (CE), reflective observation (RO), abstract conceptualization (AC), or active experimentation (AE) (Fig. 33-2). Again, each person may be Learning Assessment and Evaluation Topics 535
Figure 33-2 Learning styles 2.
Concrete
Reflective
Experience
Observation
Abstract
Active
Conceptualization
Experimentation
dominant in one area, but all must be considered in the assessment of learning and evaluation of learning therein.
To assess learning style dominance per CE, RO, AC, or AE, the following exercise may be used.
Learning Style Exercise 2
In the following assessment instrument, you are asked to complete 12 sentences that describe learning. Each has four endings. To respond to these sentences, consider some of the recent learning situations you have just written about. Then rank the endings for each sentence according to how well you think the ending describes the way you learned. Write a 4 next to the sentence ending that describes how you learn best, and so on down to 1 for the sentence ending that seems least like the way you learned. Be sure to rank all the endings for each sentence unit.
1. When I learn:
a. _____ I like to deal with my feelings.
b. _____ I like to think about ideas.
c. _____ I like to be doing things.
d. _____ I like to watch and listen.
536 Chapter 33 • assessment of Learning and evaLuation strategies 2. I learn best when:
a. ______ I listen and watch carefully.
b. ______ I rely on logical thinking.
c. ______ I trust my hunches and feelings.
d. ______ I work hard to get things done.
3. When I am learning:
a. ______ I tend to reason things out.
b. ______ I am responsible about things.
c. ______ I am quiet and reserved.
d. ______ I have strong feelings and reactions.
4. I learn by:
a. ______ Feeling
b. ______ Doing
c. ______ Watching
d. ______ Thinking
5. When I learn:
a. ______ I am open to new experiences.
b. ______ I look at all sides of the issues.
c. ______ I like to analyze things, break them down into their parts.
d. ______ I like to try things out.
6. When I am learning:
a. ______ I am an observing person.
b. ______ I am an active person.
c. ______ I am an intuitive person.
d. ______ I am a logical person.
7. I learn best from:
a. ______ Observation
b. ______ Personal relationships
c. ______ Rational theories
d. ______ A chance to try out and practice
8. When I learn:
a. ______ I like to see results from my work.
b. ______ I like ideas and theories.
c. ______ I take my time before acting.
d. ______ I feel personally involved in things.
9. I learn best when:
a. ______ I rely on my observations.
b. ______ I rely on my feelings.
c. ______ I can try things out for myself.
d. ______ I rely on my ideas.
Learning Assessment and Evaluation Topics 537
10. When I am learning:
a. _____ I am a reserved person.
b. _____ I am an accepting person.
c. _____ I am a responsible person.
d. _____ I am a rational person.
11. When I learn:
a. _____ I get involved.
b. _____ I like to observe.
c. _____ I evaluate things.
d. _____ I like to be active.
12. I learn best when:
a. _____ I analyze ideas.
b. _____ I am receptive and open minded.
c. _____ I am careful.
d. _____ I am practical.
Learning Style Exercise 2 Scoring Key
To compute your learning profile based on the four dimensions of the learning style inventory, add your scores for each of the items listed here.
LEARNING STYLE
DIMENSION
ITEMS
1
2
3
4
5
6
7
8
9
10
11
12
Concrete Experience
a
c
d
a
a
c
b
d
b
b
a
b
Total CE
Reflective Observation
d
a
c
c
b
a
a
c
a
a
b
c
Total RO
Abstract Conceptualization
b
b
a
d
c
d
c
b
d
d
c
a
Total AC
Active Experimentation
c
d
b
b
d
b
d
a
c
c
d
d
Total AE
538 Chapter 33 • assessment of Learning and evaLuation strategies The style with the most points indicates your primary learning style, next highest is your second style, and so on.
ANDRAGOGICAL VERSUS PEDAGOGICAL CONSIDERATIONS
Once information is known about the primary modes in which learning takes place, considerations for the teaching and evaluation of that learning revolves around a central argument of andragogical versus pedagogical techniques (Conner, 2003).
Andragogy, popularized by Malcolm Knowles (1973), is based on student-centered learning, also referred to as Socratic or learner-centered styles, that embrace experiential learning and social interaction from which to improve cognitive development. The andragogical teacher plays an interactive role, coaching the students, guiding them to epiphanies and applications, and keeping them on track. The facilitator’s experience and background cannot be emphasized enough.
Active and open participation by teachers and students drive the andragogical learning process based largely on Vygotsky’s socio-cultural cognitive theory.
Lev Vygotsky’s social-cultural cognitive theory (1987) is related to education effectiveness that states basically that the more social interaction, the better the learning outcomes. Therefore, “team” based intervention/groups would be theoretically more effective than those that are not. In application of Vygotsky’s theory, educational institutions have focused the majority of content delivery systems using various learning applications to create an atmosphere where students have maximal interaction with other students and instructors in order to optimize learning outcomes and cognitive development.
Oestmann and Oestmann (2006) completed a research study based on Vygotsky’s social-cultural cognitive theory related to online class size and learning outcomes. Discussion interactivity (i.e., the number of substantive discussion posts) and final course grade percentages between large online classes ( 20
students) and small online classes ( 10 students) were evaluated for significant differences and correlations. A retrospective comparison between five large and small online master’s level healthcare management courses delivered between 2004 and 2005 was evaluated. The results indicated significantly different substantive discussion posts in the large class cohort at 76.3 as compared with the small class cohort at 49.9. Similarly, the average final grade percentage in the large class cohort was 91.1% and significantly higher than the average final grade percentage in the small class cohort of 84.9%. The correlation between average number of substantive discussion posts and final grade percentages in the large class cohort was qualified as “strong” at 0.864 and significantly different from the small class cohort value qualified as “moderate” at 0.670. Consequently, Andragogical Versus Pedagogical Considerations 539
more learning occurred in the larger online classes as compared with the smaller online classes.
Anecdotally, traditional brick-and-mortar educational institutions have generally found smaller class sizes result in the most interaction between students and faculty, according to Parbudyal and William (2004). Achilles, Finn, and Bain (1998) reported in a traditional higher education classroom setting that small classes (13 to 17 students), compared with regular class sizes (23 to 26 students), provided higher student learning outcomes as determined by final grades, also citing anecdotal evidence of more student discussion interactivity in the smaller classes. Lou, Abrami, and Spence (1996) also reported increased student achievement in traditional higher education classroom settings favored small group learning sizes. In Abrami, Lou, Chambers, Poulsen, and Spence’s (2000) latest study involving class size and learning outcomes in traditional higher education classroom settings, the results confirmed previous research findings related to smaller class sizes and higher student learning outcomes. However, the authors stated that this is most likely because smaller class sizes allow for more social interaction among the instructor and student peers, which is not as prevalent in larger class sizes.
In regard to online versus face-to-face learning environments, several research studies have concluded there are no significant differences in learning outcomes.
Sims, Dobbs, and Hand (2002) reported that studies have demonstrated both positive and negative impacts in terms of effectiveness and achievement of outcomes between online and traditional higher education delivery modalities. This may directly reflect the fundamental differences in learning styles between the traditional face-to-face and virtual (online) classroom being structural: speaking and listening in the traditional classroom versus typing and reading in the online classroom. Moreover, studies by Parbudyal and William (2004); Driver (2002); and Dutton, Dutton, and Perry (2001) all reported that once courses are effectively managed by instructors, online education can be the same quality as, or even better than, the regular in-class method due in part to fostering an increase in participation by students.
Glahn and Gen (2002) summarized the “no significant difference” phenomena by stating, “Online teaching is not better than face-to-face teaching, nor is it worse. It is simply different” (p. 777). Both online and traditional methods of higher education delivery are unique and have their own advantages and disadvantages. However, the goal in both traditional and online educational delivery systems is to maximize the best features of teaching in order to promote optimal, active student-centered learning that is directly related to student learning.
Although the large versus small class size argument may never be unani-mously agreed upon in regard to optimizing learning, according to Knowles 540 Chapter 33 • assessment of Learning and evaLuation strategies (1973), five characteristics of adult learners maximize learning within the andragogical approach.
1. Self concept: As adults mature, they move from dependence to self direction.
2. Experience: As a person matures, he or she develops experience, which becomes a resource for learning.
3. Readiness to learn: Adults’ readiness to learn is related to a task- or problem-centered approach. They are motivated to learn what is useful to them.
4. Orientation to learning: Adults desire immediacy of application and prefer experiential learning.
5. Motivation: Adults have an internal motivation to learn (Smith, 2001).
Conversely, pedagogy is based on teacher-centered and teacher-directed learning in which teachers select the applicable topics and determine the structure of the learning process. The student is the vessel for receiving knowledge imparted by the teacher. Most often, this model of teaching/learning involves a lecture-based process (Conner, 2003). Despite the longstanding history of pedagogical teaching/learning, critics state the passivity of the student learning is the primary weakness of this approach.
COURSEROOM ASSESSMENT TECHNIQUES
Setting the stage for effective learning and assessment of that learning is therefore a complex process that involves, at a minimum, considerations for learning style; andragogical versus pedagogical teaching style; class size; class venue (online versus traditional face-to-face); and student age, experience, and other demographic characteristics before assessment of learning and the corresponding evaluation strategies can be implemented. Two primary resources were used for this section.
1. Angelo and Cross’ book entitled Classroom Assessment Techniques (1993) has been the gold standard by which educators have based learning assessments and evaluation for nearly two decades.
2. Diamond’s book entitled Designing and Assessing Courses and Curricula (1998).
The assessment of learning has become a focal point for accreditation bodies of higher education for obvious reasons. Not only is knowledge and application of educational materials important to validate and justify the educational curriculum and process, but in the cases of educational trades (e.g., health care, vocational jobs, etc.) learning outcomes are required to ensure competence of service provision, safety, and reduction of malpractice/litigation. Consequently, Courseroom Assessment Techniques 541
the teaching goals are a primary consideration and answers to the following questions are required:
1. What should the students learn?
2. How should the materials best be presented to achieve the learning?
3. How can we objectively measure the learning outcomes of students?
While course knowledge is traditionally measured by examination (multiple choice, true/false, short answer, essay), Bloom (1956) is credited with historical research that suggests the action verbs associated with six cognitive levels (e.g., knowledge, comprehension, application, analysis, synthesis, evaluation) for particular assignments are related with various progressive levels of knowledge. Moving from simple rote memory to evaluation and synthesis of the information demonstrates higher levels of thinking and course-related knowledge. Again, this is important for the teacher to understand because rote memorization can yield outstanding results on standardized testing and yet fail to demonstrate application of the course knowledge in the real world.
1. Bloom’s Level 1 Knowledge. At the knowledge level, the instructor’s questions require students to recognize or recall information and to remember facts, observations, and definitions that have been learned previously. Although there is substantial criticism among educators of factual or knowledge level questions, this level of question does have its place in the instructional spectrum. The overuse or abuse of this level of questioning is the main reason for criticism. It is important for instructors to realize that the learner must function at the knowledge level before being expected to perform at higher levels.
Memorization of information is also required in order to perform a variety of tasks. The meaning of words, correct spelling, multiplication facts, and rules of the road are examples of important information that must be committed to memory. (Examples: remembering, memorizing, recognizing, recalling identification, recalling information, who, what, when, where, how?, describing.) 2. Bloom’s Level 2 Comprehension. Questions at the comprehension level require students to interpret and translate information that is presented on charts, graphs, tables, as well as specific facts. It is important to realize that the student must have certain factual information in order to gain the understanding necessary to organize and arrange the material mentally.
For questions of this type, the student must demonstrate a level of understanding by rephrasing, describing, or making comparisons. (Examples: interpreting, translating from one medium to another, describing in one’s own words, retelling, organization and selection of facts and ideas.) 3. Bloom’s Level 3 Application. Questions at the application level require a student to apply previously learned information to solve a specific problem. At this level, it is not sufficient for the student to relate or even to paraphrase and interpret previously memorized information. Instead, 542 Chapter 33 • assessment of Learning and evaLuation strategies students must use the information to answer questions or solve problems.
For example, a student who has learned the definitions of latitude and longitude may be asked to locate a given point on a map. (Examples: problem solving applying information to produce some result; use of facts, rules and principles how is _______ an example of _______? how is _______ related to _______? why is _______ significant?) 4. Bloom’s Level 4 Analysis. Questions at this level of cognition require students to look for hidden meaning or inferences of acquired information.
As information is analyzed, one must reach sound conclusions or draw generalizations based on available information. Analysis questions ask students to identify motives, reasons, and/or causes of specific occurrences or events; reach certain conclusions, draw inferences, or generalizations based on given information; or identify evidence needed to support or refute conclusions, inferences, or generalizations. In many instances, there are no absolute answers to the analysis question, as several answers are plausible.
Furthermore, because it takes time to analyze these questions, they cannot be answered quickly or without careful thought. (Examples: subdividing something to show how it is put together; finding the underlying structure of a communication; identifying motives; separation of a whole into component parts; what are the parts or features of _______?; classify _______
according to _______ outline/diagram; how does _______ compare/contrast with _______?; what evidence can you list for _______?)
5. Bloom’s Level 5 Synthesis. Synthesis questions require higher order thinking processes. Students must make predictions, use creativity in developing original approaches, or solve problems that do not have single answers. Instructors can use synthesis questions to help develop and reinforce students’
creative abilities. These questions demand a substantial amount of information and a thorough understanding of many factors as students consider possible responses. (Examples: creating a unique, original product that may be in verbal form or may be a physical object; combination of ideas to form a new whole; what would you predict/infer from _______? what ideas can you add to _______? how would you create/design a new _______? what might happen if you combined _______? what solutions would you suggest for _______?) 6. Bloom’s Level 6 Evaluation. Evaluation questions require students to judge the meaning of an idea or to assess the plausibility of a solution. These actions require that a student possess substantial information and be able to establish criteria for making a judgment. (Examples: making value decisions about issues; resolving controversies or differences of opinion; development of opinions, judgments or decisions; do you agree that _______?
what do you think about _______? what is the most important _______?
place the following in order of priority; how would you decide about _______? what criteria would you use to assess _______?)
Courseroom Assessment Techniques 543
According to Cross and Cross (1993) and Diamond (1998), there are over 50
courseroom assessment techniques (CATs) that may be implemented to measure student learning. While there is no singular and universal CAT that can be applied for all classes and all situations, generally two or three CATs are both effective and efficient based on teaching goals and other variables previously identified.
Obviously both knowledge and skills (application) are important in many higher education classrooms. However, the CATs to achieve these results are again specifically designed for each course based on the individual goals. Common CATs related to this level of learning include background knowledge probe; focused listing; misconception/preconception check; empty outlines; memory matrix; minute paper; muddiest point.
How do we measure skill in analysis and critical and creative thinking? Moving to the higher levels of Bloom’s taxonomy, evaluation, analysis, and synthesis become important components of CATs implemented. As indicated previously, this is important for the teacher to understand as rote memorization can yield outstanding results on standardized testing and yet fail to demonstrate application of the courseroom knowledge in the real world. For example, a student can recite verbatim theory, theorists, indications, contraindications, and goals, but if they cannot apply the theory, or specific details of a particular case, the knowledge application is ineffective and inefficient. This has particular consequences in health care as ineffective and inefficient treatment provision may result in legal ramifications and productivity/cost inefficiencies. In addition, healthcare careers are unique in the fact that they deal with the complexity of the human condition.
It is rare that two patients in any profession present to a treating professional with exactly the same symptoms and conditions. Therefore, the ability to critically think, analyze, and act in a safe, effective, and efficient manner are imperative in the application of CATs, which commonly include categorizing grid, defining features matrix, pro and con grid, content, form and function outlines, analytic memos, one sentence summary, word journal, approximate analogies, concept maps, invented dialogues, and annotated portfolios.
How do we measure skill in problem solving? Perhaps one of the most important educationally related goals for educators is to impart the skill of problem solving for students. Regardless of the profession or educational category, problem solving is a universal skill required for almost every college educated person working in government or private industry. Problem solving CATs commonly include: problem recognition tasks, what’s the principle, documented problem solutions, audio and videotaped protocols, directed paraphrasing, applications cards, student-centered test questions, human tableau or class modeling; and paper or project prospectus.
How do we assess learner attitudes, values, and self awareness? As teachers and clinicians, it is easy to assume that we know best for our students. However, as times change and multiple generational cohorts represent our students, teachers 544 Chapter 33 • assessment of Learning and evaLuation strategies need to periodically evaluate learner attitudes, values, and self awareness. The CATs commonly associated with the evaluation of learner attitudes, values, and self awareness include classroom opinion polls, double-entry journals, profiles of admirable individuals, everyday ethical dilemmas, course-related self confidence surveys, focused autobiographical sketches, interest/knowledge/skills checklists, goal ranking and matching, and self assessment of ways of learning.
How do we assess student learning and study skills? Again, what sounds good in theory may not provide the best student learning and study skills given a particular area of education. However, the assessment by educators is another important component of effective and efficient teaching and learning. Common CATs related to assessing student learning and study skills include productive study time logs, punctuated lectures, process analysis, and diagnostic learning logs.
How do we assess learner reactions to educational instruction? In almost every higher education venue, educators are evaluated by their students. This is an accreditation requirement and common practice among higher education schools of all types. While often criticized as a forum for students to voice complaints about a particular instructor or instructional technique, this feedback is imperative to improving the effectiveness and efficiency of education. In that, it is imperative for the educators and administrators to decipher what is legitimate in terms of complaints and what is not. Obviously, the theory behind this practice is simple.
Students learn more when the teaching method/educational instruction is highly regarded. In addition to student satisfaction, many educators advocate cross comparison of these reports with student learning outcomes (e.g., final grades primarily). While other measures of student learning can be used, the premise of this practice is based on the following examples. If student satisfaction reports are high and final grades are high, two conclusions can be drawn. First, teaching and learning is both effective and efficient. Second, teaching is pleasant but learning is ineffective as grades are artificially inflated. This second conclusion requires verification of teaching/learning ineffectiveness as indicated on standardized national exams. The additional outcomes (student satisfaction reports low; final grades low) and any combination can also yield various conclusions as to teaching and learning efficacy. Per Angelo and Cross (1993), assessing learner reactions to educational instruction involves more than just simple satisfaction surveys. The following CATs related to assessing learner reactions to educational instruction include chain notes, email feedback, teacher-designed feedback forms, group instructional feedback technique, and classroom assessment quality circles.
How do we assess learner reactions to class activities, assignments, and materials? Each classroom assessment method needs to be chosen for a specific purpose and then evaluated for its success, or lack thereof. This is referred to as the classroom assessment project cycle which is similar in nature to continuous quality improvement (CQI) programs. Class activities often consist of individual or group Educational Issues: Grading Rubrics 545
projects. Sometimes personality differences may confound the learner reactions to group activities, whereas the logistics of crossing several time zones in online group projects may represent a legitimate reason to decrease or even eliminate group assignments, despite Vygotsky’s social learning theory tenets. Materials used in each course should be pertinent and timely in nature, which is the reason for course content revision as part of most educational programming. For example, if learner reactions to class activities, assignments, and materials are negative and learning outcomes are negative (e.g., poor final grades, poor final performance on clinical applications or national standardized testing), it would represent an immediate change in curriculum and activity/assignment for that particular course. In addition, the grading rubrics used in a particular course are also involved in the assessment of learner reactions to class activities, assignments, and materials. Perhaps there is an unequal distribution of grading weight for a particular assignment to the time required to complete the assignment. If, on the other hand, grading rubrics are too subjective (e.g., not quantitative and specific), poor learner reactions to class activities and assignments may also result with resulting grades being overly inflated or overly strict without substantial objective justification. CATs related to assessing learner reactions to class activities, assignments, and materials include recall, summarize, question, comment, connect (RSQC2); group work evaluations; reading rating sheets; assignment assessments; and exam evaluations.
Despite the plethora of CATs available to measure student learning, one must objectively quantitate in order to avoid grade inflation and subjective biases. In order to accomplish this task, grading rubrics become invaluable considerations.
EDUCATIONAL ISSUES: GRADING RUBRICS
As education has evolved over the years, fundamental flaws of subjective grading were replaced with more objective (quantitative) rubrics in the search for measuring learning outcomes of students. How often have you attempted to grade your students’ work only to find that the assessment criteria were vague and the performance behavior was overly subjective? Would you be able to justify the assessment or grade if you had to defend it? The rubric is an authentic assessment tool that is particularly useful in assessing criteria that are complex and subjective (Miller & Miller, 1999).
The advantages of using rubrics in assessment are that they: 1. Allow assessment to be more objective and consistent.
2. Focus the teacher to clarify his or her criteria in specific terms.
3. Clearly show the student how their work will be evaluated and what is expected.
546 Chapter 33 • assessment of Learning and evaLuation strategies 4. Promote student awareness of about the criteria to use in assessing peer performance.
5. Provide useful feedback regarding the effectiveness of the instruction.
6. Provide benchmarks against which to measure and document progress.
Rubrics can be created in a variety of forms and levels of complexity; however, they all contain common features that:
1. Focus on measuring a stated objective (i.e., performance, behavior, or quality) 2. Use a range to rate performance.
3. Contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met.
A number of options for evaluating papers exist. Evaluating a paper need not involve correcting every surface error and writing voluminous comments at the end.
1. Give separate grades for form and content.
2. Use performance grading: If students do the assignment, they get credit (or points). You make no value judgments about the quality of the work, merely decide what is an acceptable amount of work.
3. Use impression marking: Scan the paper and mark it based on your general impression of the paper’s effectiveness. Again, have a clear set of criteria in mind—or even written down—as you read.
4. Use portfolio evaluation: Rather than evaluating individual papers, evaluate a student’s entire output at the end of the course.
5. Evaluate based strictly on clearly defined criteria, which may be set out in the form of: (1) contracts (you create a contract which spells out how much work and/or what sort must be done to receive a particular grade—the student chooses what grade to work for); or (2) checksheets (you list the criteria for an acceptable piece of work and evaluate based on how many criteria are met).
6. Scales: rank a student’s work based on your criteria. Analytic and dichotomous are just two of a variety of scales; examples are included in Tables 33-1 and 33-2 (Miller & Miller, 1999).
Two grading rubrics that the authors have developed and modified over the year of teaching include a general grading rubric (i.e., 5-point grading rubric) and a formal paper grading form (Table 33-3).
Five-Point Grading Rubric Example
Grading Rubric 4, Exemplary competency. Regarding how well a student understands terms and concepts in context. Highest level of mastery (e.g., Educational Issues: Grading Rubrics 547
Table 33-1 Sample Analytical Scale
Low
High
General merit
Ideas
2
4
6
8
10
Organization
2
4
6
8
10
Wording
1
2
3
4
5
Mechanics
Spelling and punctuation
1
2
3
4
5
Grammar and usage
1
2
3
4
5
Format
2
4
6
8
10
Comprehension
Understanding of terms
2
4
6
8
10
Application of concepts
2
4
6
8
10
Total score:
synonymous use of a concept, introduction to a new concept, identification of misunderstood concept). Includes American Psychological Association (APA) formatting and referencing compliance few to no errors.
A 5 Clearly stands out as an excellent performer. Has unusually sharp insight into material and initiates thoughtful questions. Sees many sides of an issue. Articulates well and writes logically and clearly. Integrates ideas previously learned from this and other disciplines; anticipates next steps in progression of ideas. Example: A work should be of such a nature that it could be put on reserve for all students to review and emulate. The A student is, in fact, an example for others to follow.
Grading Rubric 3, Accomplished competency. Regarding how well a student understands terms and concepts in context. Mastery of terms and concepts in context. Develops argument and cites prior material in support. Includes APA formatting and referencing compliance with a few errors (1–5 per page).
B 5 Grasps subject matter at a level considered to be good to very good. Participates actively in class discussion. Writes well and complies with the majority of APA formatting and referencing rules. Accomplishes more than the minimum requirements. Produces high-quality work. Example: B work indicates a high 548 Chapter 33 • assessment of Learning and evaLuation strategies Table 33-2 Sample Dichotomous Scale
Yes
No
Content
___
___
Ideas are insightful.
___
___
Ideas are original.
___
___
Ideas are logical.
___
___
Ideas are clearly expressed.
Organization
___
___
There is a thesis.
___
___
Thesis is adequately developed.
___
___
Each paragraph is developed with concrete
and relevant details.
Mechanics
___
___
Many misspellings.
___
___
Awkward sentences.
quality of performance and is given in recognition for solid work; a B should be considered a high grade.
Grading Rubric 2, Developing competency. Regarding how well a student understands terms and concepts in context. Interpreting terms and concepts in context. Begins to develop argument beyond opinion. APA formatting and referencing is semicompliant with consistent errors noted throughout (6–10 errors per page).
C 5 Demonstrates a satisfactory comprehension of the subject matter. Accomplishes only the minimum requirements, and displays little or no initiative.
Communicates in writing at an acceptable level for a college student. Has an acceptable understanding of all basic concepts. Example: C work represents average work. A student receiving a C has met the requirements, including deadlines, of the course.
Grading Rubric 1, Beginning to meet competency. Regarding how well a student understands terms and concepts in context. Distinguishing terms or concepts in context. APA formatting and referencing is awry throughout with consistent errors noted throughout (10 per page).
Educational Issues: Grading Rubrics 549
D 5 Quality and quantity of work is below average and barely acceptable.
Example: D work is passing by a slim margin.
Grading Rubric 0, Failure to meet competency. Regarding how well a student understands terms and concepts in context. Failure to meet competency. APA formatting and referencing is literally nonexistent.
F 5 Quality and quantity of work is unacceptable. Academic credit is not earned for an F. Example: F work does not qualify the student to progress to a more advanced level of coursework.
Table 33-3 Formal Paper Grading Example
Content and Organization (70%)
Points Earned
Comments
All key elements of the assignment are cov-
ered in a substantive way.
• [Add assignment requirements as needed]
The content is comprehensive, accurate,
and/or persuasive.
The paper develops a central theme or idea,
directed toward the appropriate audience.
The paper links theory to relevant examples of
current experience and industry practice and
uses the vocabulary of the theory correctly.
Major points are stated clearly; are sup-
ported by specific details, examples, or
analysis; and are organized logically.
• [Add major points as needed]
The introduction provides sufficient back-
ground on the topic and previews major points.
The conclusion is logical, flows from the body
of the paper, and reviews the major points.
Paragraph transitions are present and logical
and maintain the flow throughout the paper.
(continues)
550 Chapter 33 • assessment of Learning and evaLuation strategies Table 33-3 Formal Paper Grading Example (continued)
Readability and Style (15%)
Points Earned
Comments
The tone is appropriate to the content and
assignment.
Sentences are complete, clear, and concise.
Sentences are well constructed, with consis-
tently strong, varied sentences.
Sentence transitions are present and maintain
the flow of thought.
Mechanics (15%)
Points Earned
Comments
The paper, including the title page, reference
page, tables, and appendices, follow APA
guidelines for format.
Citations of original works within the body
of the paper follow APA guidelines.
The paper is laid out with effective use of
headings, font styles, and white space.
Rules of grammar, usage, and punctuation
are followed.
Spelling is correct.
Total (100%)
Points Earned
Comments
Grading Feedback
In addition to the use of grading rubrics as noted previously, two common types of grade feedback include (1) formative and (2) summative (Fig. 33-3).
However, they are not mutually exclusive concepts (Weimer, 1987).
Educational Issues: Grading Rubrics 551
Figure 33-3 Summative and formative feedback continuum.
Summative
Formative
End-point
Words
Numerical
Emphasis on Feedback
Stakes are High
Useful for Generic Skills
Reliability essential
Student Learning
Source: Adapted from Weimer, M. G. (1987). Translating evaluation results into teaching improvements.
AAHE Bul etin (1) 4, 8–11.
Assessment feedback used to sum up a person’s achievement on a particular assessment is often called summative. Conversely, formative feedback is intended to inform students how to improve their learning. The emphasis in formative assessment is in encouraging more understanding in the students in relation to their strengths, weaknesses, and gaps in knowledge from which additional learning opportunities occur. In both instances, feedback should ascribe to the following parameters: 1. Always begin your comments by saying something positive about the writing. Writers need to know what parts are effective (clear, original, well worded, etc.) as well as what needs work.
2. Marking every error or covering a student’s paper with comments will not only drive you crazy, it may also overwhelm your student. Research has shown that many student writers ignore or react negatively to a large amount of written feedback, even if many comments are positive!
3. Correcting students’ errors or rewriting parts of their papers makes a lot of work for you, and does not necessarily help them. A better technique—easier for you and more thought-provoking for the writer—is indicating an error with a squiggly underline or putting a check in the margin next to the line in which the error occurs.
4. Criteria for evaluating essays may differ depending on whether or not the student has a chance to revise the writing. Some instructors expect less grammatical and mechanical accuracy on in-class essays.
5. It is reasonable to require even short answer in-class essays to be written in complete sentences rather than just lists of words or phrases; the act of putting ideas into sentences requires a higher level of thinking.
When responding to student writing to enhance learning, instructors should bear in mind two points: (1) comments should be brief enough to provide at-tainable goals for the students and (2) comments should emphasize how communication fails rather than labeling problems. Try to form your comments as questions, avoiding questions with simple yes and no responses. Begin the 552 Chapter 33 • assessment of Learning and evaLuation strategies questions with why, how, or what to generate most substantive thought from the student. Avoid imperatives such as “proofread next time”; these point out problems but do not help the student learn how to solve them.
Avoid labeling problems unless you give students a way to solve the problem.
For example, rather than writing “unclear,” write, “Did you intend to say _____
or _______” or explain what is awkward and why. You can also highlight passages in which the writer successfully avoided the problem. Praise work that will lead to improvement. For example, explain why something is good or appeals to you. This kind of evaluation can be much more difficult than the identification of problems (we arere just not used to doing it), but it is extremely important to the young writer to know what works well and why. Avoid doing the student’s work. Rewriting a passage or two for modeling purposes is fine, but only if you clarify the principle that you are modeling. You can place a check in the margin of the line in which the problem occurs and then ask the student to identify and correct it. This will work quite well if the student feels comfortable checking with you to determine what the check means. Write a careful endnote that (1) recognizes what you have legitimately liked about the paper, (2) identifies one or two problems and explains how and why they harm the quality of the paper, (3) sets a goal for the student to work toward in the next assignment, and (4) suggests specific strategies for reaching that goal.
Even when the feedback to students is not positive, it must always be constructive and respectful. We must be conscious of the manner in which feedback is presented, especially when pointing out the shortcomings of an assignment. As with all messages, it is the listener who dictates what is actually received, and facilitators need to thoughtfully consider their communication to students when addressing areas in need of improvement. Comments need to encourage performance improvement without discouraging students from trying. Lastly, grade feedback should exemplar the following characteristics:
1. Timeliness. Prompt feedback can serve as a valuable motivator for students at all levels of performance. It also serves to help students improve their performance on subsequent assignments. Faculty members must return student assignments in a timely manner (within 7 days of the assignment being submitted) with specific, objective feedback that will assist students in learning from the experience and improving future submissions.
2. Clarity. The evaluation is communicated in a form and style that can be understood easily by the student. Include your own narrative remarks on student papers that inform the student why his or her conclusions are sound and which statements need more substantiation. Add questions that prompt the student to further investigate the selected topic.
Teaching Techniques/Examples 553
3. Thoroughness. The evaluation covers all the areas of student performances as requested by the faculty member. Assignments that earn an A need feedback as much as assignments that earn a C. Feedback means informing students of what they have done well in addition to that which they need to improve.
4. Consistency. Expectations and guidelines established by faculty in the beginning are not repudiated later on in the course.
5. Equitability. Faculty follow through on a commitment to make discriminatory judgments. It is not advisable to cater to student interests by putting everyone in only two categories, such as A and A-. Likewise, it is not necessary to designate a large percentage of the course grade to the instructor’s subjective judgment, just for the sake of creating an artificial grade variance.
6. Professionalism. The facilitator is respectful in his or her communications with the student because the tone affects how students interpret and use that feedback. If facilitators use a supportive, encouraging tone when making suggestions for improvement, the feedback comes across as constructive rather than critical.
TEACHING TECHNIQUES/EXAMPLES
CATs and the teaching particulars based on the aforementioned considerations for effective and efficient assessment of learning are now posited by the authors. A baker’s dozen of learning assessment best practices that should be considered in addition to the standard techniques of written essays and examinations are based on the applicable teaching goals.
Assessing Skills in Problem Solving
1. Problem recognition. Giving the students examples of problems and looking at the depth, breadth, and clarity of their responses can give the instructor a lot of information about how well they are integrating the information.
Group discussions of their responses also will help them discover other ways of thinking, and help them understand other perspectives. Give the students a case study and have them work in small groups to identify the important issues. I find that this is a quick, effective technique to identify the patient issue and apply it to the learned information.
554 Chapter 33 • assessment of Learning and evaLuation strategies 2. Documented problem solutions. How can we get our students to adapt the problem-solving skills to real life situations? How do they synthesize the information being presented and apply it to real life?
3. Audio and videotaped protocols. Instructors could look at the video with the students and see if the student recognizes strengths and weakness, and then they can talk about ways to improve handing or interactions. This is very time consuming.
Assessing Skill in Applications and Performance
1. Directed paraphrasing. This is a very useful CAT. It is important to know how the student understands the information they are learning, and if they can put it in their own words. Additionally, it allows you to find out if the student can put the diagnosis/theory into words their patients, family members, physicians, and coworkers can understand. In a clinical setting, I would ask my students to paraphrase the theoretical perspective and approach they were using prior to going into a session. This provides the instructor with a quick feedback to the student’s knowledge base. Very useful and effective in the education of practitioner students.
2. Application cards. These are one of the easiest CATs to give. Students are just asked to write down one application for what they have just learned.
For example: In a kinesiology class, students could be asked how shortening a lever arm could be used to make an exercise easier or harder. Or, after discussing a treatment technique, students could be asked to describe how they might apply it in an outpatient setting.
3. Student generated test questions. Students are asked to generate the midterm or final exam. Their task would be to create 50 multiple-choice questions (4 responses: a, b, c, d) highlighting the correct answer with the page number in the text in which the material and answer was found. This lets us know what is important to the student and that they actually read the text and spent time critically thinking through the material and processed the information. The responsibility is on them to create the work from what they studied.
4. Human tableau or class modeling. The instructor has to be very skilled in teaching the subject matter to make it work well. The students create
“living” scenes of patient/management situations. Great for assessing application.
5. The categorizing grid. Provides the faculty with a picture of the students sorting ability. This technique is most useful in introductory courses. It Problem Areas and Solutions 555
helps students reorganize their learning to make new associations and contextual meaning and it (1) assesses students’ basic analytic and organizing skill; (2) reinforces effective organization of material/recall; and (3) moves from broad concept to specific subcategories with mastery.
6. Analytic memos. This is a simulation exercise that allows the students to organize their thoughts and present them in a concise manner, connecting all the concepts together with practice application.
7. One sentence summary. This tool assesses the student’s ability to summarize quickly. This is a powerful technique for helping students grasp the complex techniques and explain to others in nontechnical language.
8. Word journal. Summarizing a text or lesson in a single word and writing two paragraphs explaining why the student chose that word. This forces the students to critically and analytically read and assesses the student’s skill at defending or explaining. This skill will help students learn to write abstracts skillfully. This allows for active learning through reading. By choosing a word the faculty member gains insight to the student’s personal connections with the subject.
9. Concept maps. Drawings of diagrams showing mental connections that students make between major concepts presented in class. This technique allows the faculty member to see the web of relationships that students bring to the task at hand and then can easily identify gaps.
10. Invented dialogues. Students synthesize their knowledge of issues, personalities, and historical perspectives. Students might create dialogues using a series of primary quotes of get into the character and summarize how the conversation might have gone with the characters in real time. This technique forces the students to internalize and process course material in profound ways.
PROBLEM AREAS AND SOLUTIONS
Common problem areas in the assessment of learning can be illuminated through the process of self reflection. According to Nater and Gallimore (2006), the following questions should therefore be answered by each instructor as part of an ongoing process to improve the learning opportunities and minimize problem areas: 1. What are my best practices in teaching? What are the strengths I can build on? How will I do that?
2. How will my teaching practice change (or will it?) as the result of my self reflection?
3. How comfortable do I feel in creating and delivering online courses? How comfortable do I feel in mentoring or conducting directed studies? What do I need to do to increase my level of comfort with these activities?
556 Chapter 33 • assessment of Learning and evaLuation strategies 4. How comfortable do I feel with my ability to give learners constructive feedback on their work and in the courseroom? What do I need to do to increase my level of comfort with these activities?
5. What questions remain unanswered for me, and how will I get the information I need (i.e., find resources on the topic or topics, take a class, shadow another instructor, etc.)?
According to Cloud (2006), one of the central problems in the assessment of learning is based on the lack of student generational cohort considerations by faculty, which leads to ineffective instructor questioning. It is desirable to enhance the quality of student thinking through the questioning process. A questioning strategy leads the student from one level of thinking to higher levels by a careful series of questions. Probing questions should follow students’ responses and attempt to stimulate thinking about their answers. This type of questioning may be used to prompt students to think more effectively at the higher levels of analysis, synthesis, and evaluation. Leading questions guide the discussion to focus or refocus the discussion to where you want it to go. In addition to questions, instructors must also be prepared to react to student-initiated comments and questions.
Relevant student questions indicate a need for clarification or a desire to know more about the subject. Relevant comments suggest that students are actively thinking about the subject matter and relating it to past experiences. (This, too, is a great way to lead to good threaded discussions.) These questions and comments present unique opportunities that demand attention. Instructors should welcome such questions and comments and, when possible, should respond positively.
Another central problem in the assessment of learning, according to Palmer (1998), is based on faulty assumptions regarding the teaching process of critical thinking. One of the biggest challenges facing a faculty member is how to encourage students who are not using their critical thinking skills to reach their fullest potential. The reason for this lack of use could be cognitive, affective, or a mix of both areas. Noncritical thinkers are missing many great learning opportunities in their lives because of placing their focus on following, rather than generating, new ideas and possibilities. We all would probably agree that critical thinking is hard work, but the rewards are worth the effort.
Closely related to the concept of noncritical thinkers is the concept of weak sense critical thinkers (Peterson, 2001). Weak sense critical thinkers are more motivated to defend their beliefs or egocentricity. Egocentric thinking puts the person into a defensive mode. These students react to probing questions/challenges with a defensive,
“me against you” attitude. We can typically break down thinkers into three types.
1. Fair-minded thinkers. These individuals try to understand ideas and situations and attempt to be honest, empathetic, and fair. They are willing to put forth the effort necessary to think critically.
Problem Areas and Solutions 557
2. Naïve thinkers. These individuals usually do not know or do not care about the concepts of critical thinking. They tend to not think things through, and to go along with the decisions of others.
3. Selfish thinkers. These individuals are good at thinking, but not fair to others.
They tend to be manipulative and are not always intellectually honest.
There are a variety of methods for dealing with noncritical thinkers. In addressing both naïve and selfish thinkers, one should encourage them to do the following:
1. Be clear; ask them to state what they mean, explaining in different ways and giving examples.
2. Be accurate; ask them to justify or explain how they know their claims are true or how they might find out.
3. Be relevant; ask them to explain how their ideas are related to the topic at hand.
4. Be logical; ask them to explain how the ideas fit together and why they make sense. Also ask how they came to their conclusions.
5. Be fair; ask them to consider how their ideas/behaviors will make others feel or think.
6. Practice intellectual integrity.
7. Treat oneself and others with respect.
When asking noncritical thinkers to address problems (or assignments), Brookfield (1987) encourages faculty to do the following:
1. Consider the purpose/goal.
2. Restate the question in numerous ways.
3. Gather information.
4. Be aware of inferences.
5. Be aware of assumptions.
6. Clarify the ideas you use to understand the problem.
7. Understand their point of view.
8. Think through the implications/possibilities.
Lastly, learning is an individual and continuous process with a variety of motivating factors that also need to be considered by teachers (Nater & Gallimore, 2006). At least six factors serve as sources of motivation for adult learning: 1. Social relationships: to make new friends, to meet a need for associations and friendships.
2. External expectations: to comply with instructions from someone else; to fulfill the expectations or recommendations of someone with formal authority.
558 Chapter 33 • assessment of Learning and evaLuation strategies 3. Social welfare: to improve ability to serve humankind, prepare for service to the community, and improve ability to participate in community work.
4. Personal advancement: to achieve higher status in a job, secure professional advancement, and stay abreast of competitors.
5. Escape/stimulation: to relieve boredom, provide a break in the routine of home or work, and provide a contrast to other exacting details of life.
6. Cognitive interest: to learn for the sake of learning, seek knowledge for its own sake, and to satisfy an inquiring mind.
According to Nater and Gallimore (2006), the best way to motivate adult learners is simply to enhance their reasons for enrolling and decrease the barriers.
Instructors must learn why their students are enrolled (the motivators); they have to discover what is keeping them from learning. Then the instructors must plan their motivating strategies. Instructors can motivate students via several means.
1. Set a feeling or tone for the lesson. Instructors should try to establish a friendly, open atmosphere that shows the participants the instructor will help them learn.
2. Set an appropriate level of concern. The level of tension must be adjusted to meet the level of importance of the objective. If the material has a high level of importance, a higher level of tension/stress should be established in the class. However, people learn best under low to moderate stress; if the stress is too high, it becomes a barrier to learning.
3. Set an appropriate level of difficulty. The degree of difficulty should be set high enough to challenge participants but not so high that they become frustrated by information overload. The instruction should predict and reward participation, culminating in success.
In addition, students need specific knowledge of their learning results (feedback). Feedback must be specific, not general. Participants must also see a reward for learning. The reward does not necessarily have to be monetary; it can be simply a demonstration of benefits to be realized from learning the material.
Finally, the participant must be interested in the subject. Interest is directly related to reward. Adults must see the benefit of learning in order to motivate themselves to learn the subject.
Reinforcement is a very necessary part of the teaching/learning process; through it, instructors encourage correct modes of behavior and performance.
1. Positive reinforcement is normally used by instructors who are teaching participants new skills. As the name implies, positive reinforcement is good and reinforces good (or positive) behavior.
2. Negative reinforcement is normally used by instructors teaching a new skill or new information. It is useful in trying to change modes of behav-
Conclusion 559
ior. The result of negative reinforcement is extinction; that is, the instructor uses negative reinforcement until the bad behavior disappears, or it becomes extinct.
Students must also retain information from classes in order to benefit from the learning. The instructors’ jobs are not finished until they have assisted the learner in retaining the information. In order for participants to retain the information taught, they must see a meaning or purpose for that information. They must also understand and be able to interpret and apply the information. This understanding includes their ability to assign the correct degree of importance to the material.
The amount of retention will be directly affected by the degree of original learning. Simply stated, if the participants did not learn the material well initially, they will not retain it well either.
Transfer of learning ( transference) is the result of training. It is the ability to use the information taught in the course but in a new setting. As with reinforcement, there are two types of transfer: positive and negative.
1. Positive transference, like positive reinforcement, occurs when the participants use the behavior taught in the course.
2. Negative transference, again like negative reinforcement, occurs when the participants do not do what they are told not to do. This results in a positive (desired) outcome.
Transference is most likely to occur in the following situations: 1. Association: participants can associate the new information with something that they already know.
2. Similarity: the information is similar to material that participants already know; that is, it revisits a logical framework or pattern.
3. Degree of original learning: participants’ degree of original learning was high.
4. Critical attribute element: the information learned contains elements that are extremely beneficial (critical) on the job.
CONCLUSION
The following checklist is a useful tool for faculty to consider in the assessment of learning and evaluation strategies:
1. What are the primary teaching goals/objectives?
2. How do these teaching goals/objectives fit within the appropriate Bloom’s taxonomy (knowledge, comprehension, application, analysis, synthesis, evaluation)?
560 Chapter 33 • assessment of Learning and evaLuation strategies 3. What are the student’s predominant learning style(s): visual, auditory, and kinesthetic or concrete experience (CE), reflective observation (RO), abstract conceptualization (AC), or active experimentation (AE)?
4. What type of educational setting is involved (face-to-face, online, hybrid)?
5. What teaching philosophy is applicable given the student demographic characteristics (pedagogical, andragogical)?
6. What is the predominant teaching theory applicable with consideration of class size (group-based learning, Vygotsky, individual-based learning)?
7. What are the potential CATs available to assess learning based on numbers 1 to 6 considerations?
8. What type of grading rubric(s) need to be used/developed or modified in order to match the CAT used?
9. What type of feedback is most appropriate to maximize learning (formative, summative)?
10. What kind of problems may be involved with the chosen CAT and how will you best minimize these based on the techniques shared in this chapter?
Answer these questions, and you are well on your way to successful teaching and learning assessments!
REFERENCES
Abrami, P. C., Lou, Y., Chambers, B., Poulsen, C., & Spence, J. C. (2000). Why should we group students within-class for learning? Educational Research and Evaluation, 2, 158–179.
Achilles, C. M., Finn, J. D., & Bain, H. P. (1998). Using class size to reduce the equity gap. Educational Leadership, 55, 40–43.
Bloom, B. S. (1956). Taxonomy of educational objectives, Handbook I: The cognitive domain. New York: David McKay.
Brookfield, S. (1987). Developing critical thinkers. San Francisco: Jossey-Bass.
Cloud, H. (2006). Integrity: The courage to meet the demands of reality. New York: Harper Collins.
Conner, M. L. (2003). Andragogy and pedagogy. Retrieved December 20, 2008, from http://ageless-learner.com/intros/andragogy.html
Cross, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass.
Diamond, R. M. (1998). Designing and assessing courses and curricula: A practical guide. San Francisco: Jossey-Bass.
Driver, M. (2002). Investigating the benefits of Web-centric instruction for student learning: An exploratory study of an MBA course. Journal of Education for Business, 77(4), 236–245.
Dutton, J., Dutton, M., & Perry, J. (2001). Do online students perform as well as lecture students?
Journal of Engineering Education, 90(1), 131–141.
Glahn, R., & Gen, R. (2002). Progenies in education: The evolution of internet teaching. Community College Journal of Research and Practice, 26, 777–785.
References 561
Knowles, M. (1973). The adult learner: A neglected species. Houston: Gulf Publishing.
Lou, Y., Abrami, P. C., & Spence, J. C. (1996). Within-class grouping. Review of Educational Research, 66, 423–458.
Miller, W. R., & Miller, M. F. (1999). Handbook for col ege teaching. Atlanta: PineCrest Publications.
Nater, S., & Gallimore, R. (2006). They haven’t taught until they have learned. Morgantown, WV: Fitness Information Technology.
Oestmann, E., & Oestmann, J. (2006). Significant difference in learning outcomes and online class size. Journal of Online Educators, 2(1), 1–8.
Palmer, P. J. (1998). The courage to teach. San Francisco: Jossey-Bass.
Parbudyal, S., & William, P. (2004). Online education: Lessons for administrators and instructors.
College Student Journal, 38(2), 302–308.
Peterson, D. (2001). Critical thinking in the strong sense in the weak sense non-critical thinking.
Retrieved on December 29, 2007, from http://www.santarosa.edu/dpeterso/permanenthtml/Phil
.CT.lessonsP5.html
Sims, R., Dobbs, G., & Hand T. (2002). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135–148.
Smith, M. K. (2001). Andragogy: The history and current use of the term plus an annotated bibliography. Retrieved December 20, 2008, from http://www.infed.org/lifelonglearning/b-andra.htm Vygotsky, L. S. (1987). Thinking and speech: The col ected works of L.S. Vygotsky. New York: Plenum.
Weimer, M. G. (1987). Translating evaluation results into teaching improvements. AAHE Bulletin, (1) 4, 8–11.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
