Q&A        Answers to Frequently-Asked Questions

  
 
Learning Outcomes Management		  

 
  1.  What is Learning Outcomes Management  
  LOM is the systematic process of designing, measuring, monitoring and managing student learning-outcomes in a formal educational setting. Historically, the determination of effectiveness of higher education (possibly, even the education sector as a whole) relied on measures of inputs (incoming freshman class GPA or SAT scores; faculty qualifications and other credentials etc) and of processes (number and types of courses taught/ taken, breadth and depth of courses offered, rigor of the academic requirements for completion of a course, other requirements for graduation etc). More recently, the emphasis has shifted to assessment of outcomes. This doesn't mean that inputs and processes are unimportant; the shift signifies a general appetite among external stakeholders for more information, transparency and accountability.  
  EduMetry has developed a comprehensive, end-to-end solution to assist academic officers (provosts, deans, vice-presidents of academic affairs and department chairs) with the assessment of student learning. EduMetry's engagement model emphasizes working as an extension of the Office of the Provost or Dean or their Institutional Research departments. Academic administrators (or the committees they preside over) handle all of the strategic aspects of assessment, such as setting institutional mission and determining institution-wide learning-outcomes and later, in the process, "close the loop" by implementing changes.  
     
 
Design SLOs Develop rubrics Generate SLO data Analyze SLO data Recommend action
 
     
  2.  How is Learning Outcomes Management different from assessment?  
  One of EduMetry's tag-lines summarizes Learning-Outcomes Management (LOM) as a process to Measure, Monitor and Manage learning outcomes. Assessment of student learning is an integral part of Learning Outcomes Management, but the term Management connotes much more. Most articles and books emphasize that a holistic approach to Assessment starts with learning objectives, their associated learning outcomes and the assessment methods and instruments used to appraise learning. It is the integration of all of these elements into a systemic whole that underpins what we called Learning Outcomes Management.  In other words, Learning Outcomes Management is much more than Assessment; it entails a carefully thought-through framework for first designing, then measuring, followed by continual monitoring of what students have learned.  Out of this process come the insights about what to add, change, eliminate and so on, in order to improve learning outcomes.  To be sure, for us, LOM entails measurement (assessment), but our model places equal emphasis on monitoring (hence, our learning-dashboard) and management.  
  Here's a concrete illustration. Suppose you are the provost or dean of a large, urban college, which draws an equal proportion of adult-learners and students fresh out of high school. Anybody who has taught at such an institution knows, both intuitively and from experience, that these sets of learners come with different needs to the classroom. The younger learner is generally better prepared academically and displays a higher capacity to absorb theoretical material, whereas the adult-learner, though somewhat "academically rusty", comes with a wealth of experience that can be related to the material being taught. (They confirm what Richard David Bach wrote: Learning is finding out what you already know".) We venture to hypothesize that adult-learners show higher-levels of critical thinking than new high-school-leavers, because of experience and wisdom. Now, if the college in question had the size (urban colleges usually have the FTE numbers to do this) to empirically test this hypothesis with data on student learning-outcomes, then it would be possible to design curricula, pedagogy and testing formats all based on such data. It would also be possible to match instructors (given their diverse background and teaching-style to) with learner-needs. It is in this broader, strategic sense that we use the expression Learning-Outcomes Management; learning-data-driven recommendations have the power to fundamentally transform the way colleges educate their students.  
  3.  Why focus on learning?  Isn’t teaching what academic institutions are all about?  
  Traditionally, universities and colleges have measured institutional effectiveness by measuring teaching, not learning. Teaching is the activity most associated with educational institutions, but learning is the desired outcome. An outcomes orientation focuses attention on what students ought to have learned, thus rendering the enterprise of teaching more purposive, accountable and mission-centered. A teaching-centric perspective to academic administration emphasizes activity and is largely incidental to learning outcomes. In the language of systems thinking, the emphasis must shift from inputs (SAT scores) and transformation (contact-hours, courses, credits) to output (knowledge, skills, competencies). Increasingly, thanks to the internet, students are encouraged to learn on their own, drawing from a variety of sources (with guidance from the professor), much like they do in their daily lives. The pace of change around us means that educational institutions must see college as the supreme opportunity to teach students to learn how to learn. It is these meta-skills that will allow them to engage in continuous, life-long learning.  
  4. Why is the measurement of learning outcomes so important?  
  Over the past three decades or more, we've learned a lot from other areas of human endeavor about assessing outcomes and performance measurement. As the old adage goes, to improve a phenomenon, we need to first measure it, in all its inherent richness and complexity. Physical scientists have practiced that for a long time, but it is still evolving in the social sciences. The more elaborate the measurement, the greater the ability to make improvements, try out new wrinkles and assess how the changes work.  
  5. What's the difference between grading and assessment?  
  Grading entails evaluating an individual student's academic performance. The emphasis is on determining whether, and to what extent, each student has grasped conceptual material, has learned to apply theory to solve problems or create something and can extend a concept or make inferences etc. Viewed this way, the responsibility is to the individual student and to a class of students as a whole.  
On the other hand, assessment is the systematic appraisal of learning of a class or cohort of students enrolled in a program of study. It entails examining learning-outcomes data and making summative evaluation about whether, and the extent to which, a program of study (curriculum pedagogy, testing etc) meets its objectives by producing the learning intended.  In assessment, the evaluation is more fundamental and less about a particular body of knowledge. For instance, Chemistry and Psychology majors access very different bodies of knowledge, but would both be expected to master basic intellectual skills like analytical reasoning, critical thinking, written communication, sensitivity to cultural differences and the like. So, by its very nature, assessment tends to focus on developing intellectual skills and competencies. Finally, assessment has an element of external accountability to it in a way that grading doesn't. Taken together, the insights generated allow administrators to address the information needs of stakeholders (internal and external).  Employers, parents, governments and the media all have a legitimate interest in learning about the outcomes of the education system; consequently, the assessment process and the data it produces need to be transparent, accessible and amenable to comparison and bench-marking.
  6. How exactly would EduMetry liaise with us in carrying out its assessment work?  
  We work collaboratively with college administrators. Initial discussions will center on such preliminaries as alignment between institutional mission, degree-program objectives and student-learning outcomes. Thereafter, if we develop rubrics for you, we would validate (calibrate) with them pilot data, both of which will need your inputs and "sign-off". Then, the discussion shifts to crafting a representative, stratified  sample of student learning-artifacts. This, too, is a collaborative process, because institutions differ on the courses students take in the core curriculum versus area of specialization. Then, there's the actual logistics of electronically shipping us the sample of learning artifacts, which may require some coordination.  Once we have the sample of learning-artifacts, we can leave for an extended period to do the actual scoring (using rubrics) to generate the data, statistical analysis of the data and drawing up recommendations based on the findings. Finally, we will send the recommendations back to the administrator and, if need be, present the findings to your faculty.   
  7. What makes EduMetry confident that it can do high-quality assessment work from the outside?  
  We have invested time and energy in building our capabilities. First, there is research. We studied the requirements set forth by all of the regional and national and (some of the) specialized accreditation bodies in the area of learning assessment.  As might be expected, there are both commonalities and differences in their approaches. Our researchers have systematically mapped these similarities and differences. Second, we have invested in finding and training the people (all Masters- or PhD-qualified professionals who keep abreast of the latest thinking on assessment).  Third, we developed a detailed and standardized process for carrying-out assessment.  Far from being a drawback, our being an entity distinct from your institution allows us to bring so-called "best-practices" to our work. We are constantly cross-fertilizing our knowledge with good practices we are exposed to at client institutions, which then becomes part of our repertoire for the next client. In sum, we specialize in assessment.   
  8. It sounds like delegation of learning-assessment?  
  You can call it that or you can call it hiring a specialist. In the end, these three questions serve as the arbiter: (A) Is this activity the best use of faculty members' time? (B) Do we have all the expertise in-house to carry out assessment effectively, thoroughly and systematically, so that the resulting data and findings can be used to continually improve teaching-and-learning? (C) Even if faculty members had the time, inclination and the expertise, will an assessment exercise undertaken entirely by insiders have the objectivity and consistency to generate meaningful insights about what students have actually learned? Faculty are intellectuals who have earned their prized place in society as professional skeptics. While that characteristic serves the aim of knowledge-creation exceptionally well, it does not, unfortunately, serve the institution well in its need to manage the enterprise called teaching-and-learning, which, like all the other large-scale, organized human endeavors of our time, needs systems, processes, technology and at least a modicum of hierarchy. This last ingredient is anathema to faculty, which is why turning to specialists makes sense; we are paid to deliver results, not to contemplate the natural order of things!  
  9. How do you price your LOM service?  
  Some assessment vendors charge for assessment products (usually web-based software products) on a per-FTE basis.  EduMetry, on the other hand, recognizes that only some of the activities in the multi-step, assessment process require effort proportional to the size of the student body. More specifically, with reference to our LOM framework, Steps 1, 2, 4 and 5 do not vary with the size of the student body.  (For instance, the cost of designing a rubric to assess critical-thinking skills is the same whether the institutions has 500 students or 30,000 students, because the effort is the same.  Likewise, when you run a statistical analysis (Step 4), it makes no difference whether the sample size is 50 or 5,000.  By offering a fair and transparent pricing scheme, you can rest assured that you are truly paying for value and that it is, indeed, less expensive to leave the heavy-lifting to the experts.  
 
 
  Attend a webinar   Request a proposal *
 

   

           

       

  © EduMetry Inc 2005-07