Management-Oriented Evaluation Approaches (Ch. 5)
This is used to help management to make decisions.
They identified the decisions that have to be made. However, you have to know your goals before using this approach.
Steps that need to be made
1. Identify the level or scale to be served.
2. For each level to be served identity the decision situations to be served
3. Criteria (variables)
4. Collection-- Then you need to collect information about decision alternatives. consider source, instrument, sampling procedure, schedule.
5. Organize, analyze, and report
This model really seems political.
CIPP Model (based on management-oriented evaluation)
Decision evaluated approach and its goal is to help management make the correct decision.
CIPP Model: helps find the kind of decision that has to be made
Context-planning objectives
Input-structuring decisions
Process-implement decisions
Product-recycle decision
If you were to do a total evaluation you would use everything but you can just do part of it.
Has a good connection with stakeholders but this can be a drawback as well.
Wednesday, September 30, 2009
Monday, September 28, 2009
September 28
Objective-oriented evaluation
Tyerian Evaluation Approach
Founded by Ralph W. Tyler
Evaluation as a process of deterring if objectives were being met.
7 steps:
1. Establish goals
2. Classify goals
3. Define objectives in behavioral terms
4. Find situations in which achievement of objectives can be shown
5. Measurement
6. Collect data
7. Compare performance data with behaviorally stated objectives.
Because it is so goal oriented people can have a narrow view of value. People’s definition of goals can vary and cause some problems.
Metfessel & Michael’s Evaluation Paradigm
Stresses the involvement of stakeholders and would gather data in different ways.
Provus’s Discrepancy Evaluation Model
Viewed evaluation as a continuous information management process to assist decision making process.
Logic Model
Have a long term goal and then setting the objectives to reach that goal.
Goal Free Evaluation
They try to determine the goals that people have without being told what the goals are. This is good to find unintended goals.
The strength of this approach is that it is easy to do and pretty straightforward.
Experimental Approach
This can be unpractical and even unethical. Trying to control factors is impossible. Control is an illusion. It can also be unethical because you are denying treatments or methods to groups. It is also expensive.
Tyerian Evaluation Approach
Founded by Ralph W. Tyler
Evaluation as a process of deterring if objectives were being met.
7 steps:
1. Establish goals
2. Classify goals
3. Define objectives in behavioral terms
4. Find situations in which achievement of objectives can be shown
5. Measurement
6. Collect data
7. Compare performance data with behaviorally stated objectives.
Because it is so goal oriented people can have a narrow view of value. People’s definition of goals can vary and cause some problems.
Metfessel & Michael’s Evaluation Paradigm
Stresses the involvement of stakeholders and would gather data in different ways.
Provus’s Discrepancy Evaluation Model
Viewed evaluation as a continuous information management process to assist decision making process.
Logic Model
Have a long term goal and then setting the objectives to reach that goal.
Goal Free Evaluation
They try to determine the goals that people have without being told what the goals are. This is good to find unintended goals.
The strength of this approach is that it is easy to do and pretty straightforward.
Experimental Approach
This can be unpractical and even unethical. Trying to control factors is impossible. Control is an illusion. It can also be unethical because you are denying treatments or methods to groups. It is also expensive.
Friday, September 25, 2009
September 25
Psuedo Evaluations:
Public Relations Inspired - It is basically propaganda. The object is to create an evaluation specifically to make something look good or bad. It has little rigor and is not a good evaluation.
Politically Controlled - when the results are edited or withheld because of a desired perception. This happens all of the time. Unlike public relations inspired evaluation it can be a good evaluation but then it is edited as a way of misleading the public. An example of this.
Pandering - catering to the clients desire for predetermined evaluation conclusion. Really close to public relations inspired. This could be when an evaluator gives a positive evaluation in hopes of being used again.
Empowerment as Evaluation - You get someone else to do it but you take credit for it. When my cooperating teacher had me write my own letter of recommendation. Review publications will sell their top spot in a "top list."
Quasi Evaluations:
Accountability focus - limited scope/questions or simple objectives/criteria. Timeliness of results issues. Payment-by-results approaches. Preoccupation with specific outcomes. Only using test scores to evaluate the quality of the school would be an example of this.
Success Case - document successes, identify contextual factors that led to success.
Experimental Studies - Concerned with establishing cause and effect, often using only a narrow set of program factors. Naturalistic Evaluation is the opposite of this.
Management Information Systems - select limited set of variables as indicators of success. Again this is limiting the scope and time.
Cost Benefit Analysis - when are you just focused on the "bottom line"
Judicial Debate Approaches - role playing evaluators. Mock trials on the case of the evaluation.
Criticism and Connoisseurship - Experts make decision based on their experience and knowledge.
Psuedo Evaluations - consider accuracy, ethics/reporting, purpose and objective
Quisi - consider its completeness and focus.
Public Relations Inspired - It is basically propaganda. The object is to create an evaluation specifically to make something look good or bad. It has little rigor and is not a good evaluation.
Politically Controlled - when the results are edited or withheld because of a desired perception. This happens all of the time. Unlike public relations inspired evaluation it can be a good evaluation but then it is edited as a way of misleading the public. An example of this.
Pandering - catering to the clients desire for predetermined evaluation conclusion. Really close to public relations inspired. This could be when an evaluator gives a positive evaluation in hopes of being used again.
Empowerment as Evaluation - You get someone else to do it but you take credit for it. When my cooperating teacher had me write my own letter of recommendation. Review publications will sell their top spot in a "top list."
Quasi Evaluations:
Accountability focus - limited scope/questions or simple objectives/criteria. Timeliness of results issues. Payment-by-results approaches. Preoccupation with specific outcomes. Only using test scores to evaluate the quality of the school would be an example of this.
Success Case - document successes, identify contextual factors that led to success.
Experimental Studies - Concerned with establishing cause and effect, often using only a narrow set of program factors. Naturalistic Evaluation is the opposite of this.
Management Information Systems - select limited set of variables as indicators of success. Again this is limiting the scope and time.
Cost Benefit Analysis - when are you just focused on the "bottom line"
Judicial Debate Approaches - role playing evaluators. Mock trials on the case of the evaluation.
Criticism and Connoisseurship - Experts make decision based on their experience and knowledge.
Psuedo Evaluations - consider accuracy, ethics/reporting, purpose and objective
Quisi - consider its completeness and focus.
Wednesday, September 23, 2009
September 23
You need to look at where people are coming from to understand the way people evaluate. People's views of ontology and epistemology. In ontology there is a spectrum of beliefs with realists on one end who believe in a concrete truth and on the other are the Nomilists who think that there is no truth and everything is relative.
This was a really helpful way to understand these topics. As we do our presentations we should be thinking what ontology it came from. We should also be looking at Utilitarian (Greater Good) vs. Intuitionist-Pluralist (Individual Impact).
Monday, September 21, 2009
Monday, Sept 21
It seems to me that inductive and deductive logic seem pretty clear. However there is another form of logic called Emotional Logic. When we were talking about emotional logic I thought of Steven Colbet word "truthiness." This is how he describes it:
Truthiness is tearing apart our country, and I don't mean the argument over who came up with the word…
It used to be, everyone was entitled to their own opinion, but not their own facts. But that's not the case anymore. Facts matter not at all. Perception is everything. It's certainty. People love the President because he's certain of his choices as a leader, even if the facts that back him up don't seem to exist. It's the fact that he's certain that is very appealing to a certain section of the country. I really feel a dichotomy in the American populace. What is important? What you want to be true, or what is true?…
Truthiness is 'What I say is right, and [nothing] anyone else says could possibly be true.' It's not only that I feel it to be true, but that I feel it to be true. There's not only an emotional quality, but there's a selfish quality.
There also exists a type of hierarchy of "Ways of Knowing"
1. Personal Experience (empiricism)
2. Reasoning
3. Expert Opinion or Tradition
4. Statistics
5. Pragmatism
6. Science
7. Scholarship
8. Conscience and Revelation
Why don't people use evaluation?
I was thinking that one big reason why people avoid it is because of their fear of failure.
What did you learn from the reading?
Evaluation comes form government trying to cut out the pork.
Subscribe to:
Posts (Atom)