Jump to content

Contracting Officer Testing


rmeman2k

Recommended Posts

Colleagues,

Tthe Army Contracting Command is about to implement procedures to test candidates seeking a contracting officer?s warrant above the simplified acquisition threshold. Undertaking preliminary research on the topic I discovered that Air Force Mandatory Procedures (MP) 5301.603 dated Apr 2010 requires testing that mirrors the new Army Contracting Command testing procedures. The highlights being:

1) A four-hour time limited test open book/open computer examination to assess contracting knowledge and research ability.

2) Each test shall be computer-generated and will contain 50 randomly selected true/false and multiple-choice questions from the FAR, DFARS, and AFARS.

3) A candidate must score a minimum of 85% to pass the test and be eligible for a warrant.

4) At a minimum, every candidate and contracting officer will retake the test and complete board interviews every five (5 years) to maintain warrant eligibility.

My Army agency is requesting feedback from contracting officer?s on the floor about the new policy. The initial knee jerk reaction from some of my peers was resistance to the change but upon further review I am a acquisition professional sure of my talents and I work hard at staying current in my craft and take my obligations and responsibility seriously so why would I not want to reaffirm by test my Governments decision to trust me with a warrant? I would like to hear from my Air Force peers that have been under this policy and thoughts from the field in general.

Link to comment
Share on other sites

Guest Vern Edwards

I support written exams for CO appointment. But...

First, contracting officers buy and do many things. Some buy base support services while others buy major systems. Some mainly issue and definitize change orders. Some don't buy much of anything, but negotiate forward pricing rate agreements and final indirect cost rates. Some are PCOs, some are ACOs, and some are TCOs. While all should possess the same fundamental knowledge, any notion that one test is suitable for all is simply absurd, and the people who support such exams are either insincerely advocating pro forma testing or do not understand the contracting field.

Second, any test made up entirely of true or false and multiple choice questions is an insult to the intelligence of truly qualified people. Such questions are sufficient to test only the most basic knowledge and are mainly aids to efficiency in testing. Any test that does not include both short and long form essay questions is not respectable. CO candidates should be tested for the ability to reason, argue, and explain. Candidates should be tested for knowledge of basic mathematics. Ideally, they also should be tested for oral communication skill. So, as for the test described by rmeman2k, I ask: What's the point?

Third, lawyers don't have to retake the bar exam, so why should COs routinely have to retake the qualifying test? There is a reason for continuing education. Instead, a CO's continued qualification should be evaluated on the basis of performance.

Fourth, a score of 85 percent isn't saying much. Is it okay for a CO to be right only 85 percent of the time?

Finally, if the 1102 community has any sense of self-respect and if it wants respect from other functional personnel, it should not merely assent to such testing, it should demand it, and it should demand that the testing be challenging. Agencies are giving CO appointments to unqualified people because they need signatures. It is shameful and a sign of disrespect for the contracting function.

Sadly, many people are afraid of high standards. I doubt we will ever have serious testing of CO candidates.

Link to comment
Share on other sites

Second, any test made up entirely of true or false and multiple choice questions is an insult to the intelligence of truly qualified people. Such questions are sufficient to test only the most basic knowledge and are mainly aids to efficiency in testing.

I disagree with that statement as it pertains to multiple choice questions. Such questions can be used to assess more than just basic knowledge. For an example of multiple-choice questions at advanced levels of learning click here.

Link to comment
Share on other sites

Guest Vern Edwards

Don,

I think your examples prove my point. Those questions are high school. Here is my idea of an "advanced level" multiple choice question:

===============

In response to a request for proposals for 2,000 two-way radios to be manufactured to a government specification, Radio Corp. proposed to manufacture and deliver the radios for a unit price of $404.00 each. Each radio included a battery case, which the government?s specification required to be .064 inch thick. The cost of the battery case was included in the unit price for the radio. The battery case was not separately priced.

Radio Corp. had manufactured the radios for the government before. The last time, the government?s specification had required the battery case to be .051 inch thick. For this procurement, however, the government engineers had increased the thickness of the battery case to .064 inch thick. Radio Corp. planned to subcontract the manufacture of the battery case, as it had before. But in seeking quotes, Radio Corp.?s purchasing department failed to notice that the government had changed its specification, and solicited subcontractor quotes based on the old specification of .051 inch thick instead of the newly specified .064 inch thick. (This was an honest mistake.) The lowest quote Radio Corp. received from prospective subcontractors for a .051 inch thick case was $13.97 per case. When preparing its proposal, Radio Corp. assumed that it could bargain that price down to $12.48 per case, so it decremented the subcontractor quote to $12.48, which is the amount that it included in its proposed $404.00 unit price.

Radio Corp. won the contract at its proposed price of $404.00 per radio. The government did not require offerors to submit cost or pricing data or information other than cost or pricing data with their proposals. At the time of contract award neither the government nor Radio Corp?s proposal team was aware of Radio Corp.?s mistake about the case.

After contract award, but before Radio Corp. could award the subcontract for the cases, another prospective subcontractor called Radio Corp.?s purchasing department and asked to submit a quote. The purchasing department then detected its mistake about the thickness of the case and asked for a quote for cases .064 inch thick, instead of .051 inch. The new quote was $7.52 per case. Since the contract unit price included $12.48 for the case, the purchasing department?s mistake meant that Radio Corp. would make an additional profit of $4.96 per radio.

But before Radio Corp. ordered any cases, the government?s engineers discovered that an .064 inch thick case would not fit properly in the radio. The contract included FAR clause 52.243-1, Changes?Fixed-Price (AUG 1987), so the contracting officer issued a change order, changing the thickness of the case from .064 back to .051 inch thick. Only at this point did the government learn about the $13.97 quote that Radio Corp. had originally obtained for the .051 inch cases, about the $12.48 it had included in its proposal, and about the $7.52 quote for the .064 cases. The contracting officer asked Radio Corp. to get new quotes for the .051 cases. The lowest new quote Radio Corp. received was $11.63 per case.

Which of the following is the correct measure of the proper equitable adjustment under FAR 52.243-1, excluding adjustment to profit:

a. Reduction of $ 0.85 per unit.

b. Increase of $4.11 per unit.

c. Reduction of $2.34 per unit.

d. Increase of $4.45 per unit.

Explain the rationale for your answer in 500 words of less. Your choice must be correct and your rationale must support it.

===============

Vern

Link to comment
Share on other sites

Vern,

Really? If you think that those questions assess "basic knowledge", then we must differ on the definition of "knowledge".

Below is a sample multiple-choice question from the LSAT:

Exactly six trade representatives negotiate a treaty: Klosnik, Londi, Manley, Neri, Osata, Poirier. There are exactly six chairs evenly spaced around a circular table. The chairs are numbered 1 through 6, with successively numbered chairs next to each other and chair number 1 next to chair number 6. Each chair is occupied by exactly one of the representatives. The following conditions apply:

Poirier sits immediately next to Neri.

Londi sits immediately next to Manley, Neri, or both.

Klosnik does not sit immediately next to Manley.

If Osata sits immediately next to Poirier, Osata does not sit immediately next to Manley.

1. Which one of the following seating arrangements of the six representative in chairs 1 through 6 would NOT violate the stated conditions?

(A) Klosnik, Poirier, Neri, Manley, Osata, Londi

(B) Klosnik, Londi, Manley, Poirier, Neri, Osata

( C ) Klosnik, Londi, Manley, Osata, Poirier, Neri

(D) Klosnik, Osata, Poirier, Neri, Londi, Manley

(E) Klosnik, Neri, Londi, Osata, Manley, Poirier.

Do you think that this question assesses "basic knowledge"?

Link to comment
Share on other sites

Guest Vern Edwards

Don:

I added to my prior post after you posted your response. Sorry. As for the LSAT question, that is part of a test to decide who can go to law school, not who can defend a person up on a capital murder charge.

Vern

Link to comment
Share on other sites

Guest Vern Edwards
Vern,

Really? If you think that those questions assess "basic knowledge", then we must differ on the definition of "knowledge".

Below is a sample multiple-choice question from the LSAT:

Exactly six trade representatives negotiate a treaty: Klosnik, Londi, Manley, Neri, Osata, Poirier. There are exactly six chairs evenly spaced around a circular table. The chairs are numbered 1 through 6, with successively numbered chairs next to each other and chair number 1 next to chair number 6. Each chair is occupied by exactly one of the representatives. The following conditions apply:

Poirier sits immediately next to Neri.

Londi sits immediately next to Manley, Neri, or both.

Klosnik does not sit immediately next to Manley.

If Osata sits immediately next to Poirier, Osata does not sit immediately next to Manley.

1. Which one of the following seating arrangements of the six representative in chairs 1 through 6 would NOT violate the stated conditions?

(A) Klosnik, Poirier, Neri, Manley, Osata, Londi

(B) Klosnik, Londi, Manley, Poirier, Neri, Osata

( C ) Klosnik, Londi, Manley, Osata, Poirier, Neri

(D) Klosnik, Osata, Poirier, Neri, Londi, Manley

(E) Klosnik, Neri, Londi, Osata, Manley, Poirier.

Do you think that this question assesses "basic knowledge"?

I think the answer is (B) - Klosnik, Londi, Manley, Poirier, Neri, Osata.

1. Poirier sits immediately next to Neri.

2. Londi sits immediately next to Manley.

3. Klosnik does not sit immediately next to Manley.

4. Osata does not sit immediately next to Poirier or Manley.

If that's right, then how hard can it be if I can figure it out?

Link to comment
Share on other sites

My opinion - It is not reasonable to test a KO with a limited warrant who is performing SAP Commercial installation-level acquisitions on acquisition procedures for non-commercial R&D or Construction. As a KO I am in support of primarily SAP supplies and services. I have little to no experience in non-commercial R&D or Construction contracting and would not accept a warrant in support of either.

Link to comment
Share on other sites

Vern's Post #2 made 5 points. People have taken exception to one of them.

Let me add my endorsement to: "Fourth, a score of 85 percent isn't saying much. Is it okay for a CO to be right only 85 percent of the time?"

During SCUBA training I felt much the same way. After scoring 99 on a test, the 1% you missed can kill you.

Any system of testing has flaws, but also has the potential to measure knowledge and establish standards. Is putting an admittedly imperfect system in place better for the taxpayers than the status quo?

Link to comment
Share on other sites

What?s the answer to Vern?s question ? I say d) $4.45. I use the difference between the lowest quote before the change and the lowest quote after the change ? $4.11 ($11.63 -$7.52). Plus costs incurred to prepare a request for price adjustment proposal are allowable, per CAM 12.502, and since there is no other value above $4.11, the multiple choice answer of $4.45 must reflect the additive proposal costs.

I do think that Vern met to assign $6.45 to d) ($13.97 - $7.52) if so, then d) is incorrect and B) is correct if proposal costs are not meant to be part of this question.

In reality probably any answer is OK, because we can't see the contractor's equitable adjustment proposal. They may have asked for a price reduction - and I am sure that the Government would not have said no you should get more money.

DON's question LSAT question for would be jailhouse lawyers - Answer B

Link to comment
Share on other sites

Don:

I added to my prior post after you posted your response. Sorry. As for the LSAT question, that is part of a test to decide who can go to law school, not who can defend a person up on a capital murder charge.

Vern

Vern,

The difficulty of the question and the purpose of the test it appears on are beside the point. This is an example of a multiple-choice question that does not assess basic knowledge. One cannot "know" the correct order of the trade representatives and recall that fact from memory. One can only arrive at the answer through analytical reasoning.

The question that you wrote (minus the essay requirement) would be another example of a multiple-choice question that assesses more than basic knowledge.

Link to comment
Share on other sites

Guest Vern Edwards

Don,

A person could get the answer to my question right by guessing, which is why I would tack on the essay. I despise multiple choice tests and the institutions that use them. Nothing you have said or could say is going to change my opinion about multiple choice testing, which is summed up nicely by this:

By means of sample questions it is shown how the tests use ambiguity, and worse, as substitutes for worthwhile difficulty, how they favour quick-witted superficiality while penalizing depth, subtlety and creativity, how they encourage intellectual dishonesty and how they corrupt education.

Hoffman, "Multiple Choice Tests," 1967 Phys. Educ. 2, 247.

A person who gets a warrant because he or she could get a score of 85 on a 50-question multiple choice test gets no professional respect from me on that account.

We have hijacked this thread, so I am not going to discuss this further with you here. You can take this up with me privately if you wish.

Vern

Link to comment
Share on other sites

Don,

A person could get the answer to my question right by guessing, which is why I would tack on the essay. I despise multiple choice tests and the institutions that use them. Nothing you have said or could say is going to change my opinion about multiple choice testing, which is summed up nicely by this:

Hoffman, "Multiple Choice Tests," 1967 Phys. Educ. 2, 247.

A person who gets a warrant because he or she could get a score of 85 on a 50-question multiple choice test gets no professional respect from me on that account.

We have hijacked this thread, so I am not going to discuss this further with you here. You can take this up with me privately if you wish.

Vern

I had developed a lenghty explanation of how the National Council of Engineering Examiners tested for Engineer in Training certification and for Professional Engineer Registration. but I pushed the wron button on the BB and it erased it. Will try again tomorrow at office on the computer. Short version - relying only on multiple choice questions seems to be for the convenience of the testing organization and is not the best way to test knowledge and qualifications.

Link to comment
Share on other sites

Guest Vern Edwards
Short version - relying only on multiple choice questions seems to be for the convenience of the testing organization and is not the best way to test knowledge and qualifications.

That is right on the money.

Link to comment
Share on other sites

"1) Do you think that a test consisting solely of essay questions would be the best way to test knowledge and qualifications?"

It might or might not be. I had some college classes which used that method.

If memory serves me correctly (1977), the 8 hour National Council of Engineering Examiners standardized Professional Engineers' Examination at least partially consisted of engineering problems, in which we had to develop a solution. For the afternoon portion, we had to select 5 problems, worth 10 points, each. We had to show the full methodology used and the answer. No calculators were allowed. We used slide rules and logarithm books (The Chemical Rubber Company - "CRC-Handbook"). Other reference books and textbooks were allowed. Memorization of all engineering formulas, etc. wasn't necessary or even practical.

The morning session consisted of multi-part problems. I think we had to select 5 different problems to answer. Each had 10 questions. I don't remember if they built on each other.

And yes - even if we got the wrong answer, we could get partial credit. I remember a V-notch weir problem as part of a dam. I couldn't find the dad-gummed formula to calculate the flow capacity of such a weir, but I remembered that the flow was directly proportional to various factors, including a modifier. I admitted that I couldn't find the correct formula and said that I had developed the formula and included a "K-factor". I told them that and that my assumed K-factor was made-up. I remember getting something like 8 out of 10 points on that problem. I believe that we had to score at least 70 or 75 points to pass.

For example, other problems included steel design and reinforced concrete design. The complex formulas for the concrete design required numerous iterations to arrive at the optimum, most economical combination of size, concrete strength, rebar strength, rebar size, rebar spacing spacing, etc. One could get partial credit, even if we didn't have time to optimize the design elements.

There was also a State-specific exam, which included questions on state laws and regulations, including a section on handicapped accessible design standards. We had four years of engineering experience before taking the PE exam. With the subsequent advent of personal computers, design of reinforced concrete structural members and the involved math has greatly simplified the task!! If I could have even used a calculator or a programmable calculator, I would have enjoyed math, concrete design, structural design, back then. As it was, it was pure drudgery at times.

Now, the Engineer-in-Training exam was a prerequisite for the PE exam. We could take it in college. Unfortunately, I waited 5 years after college to take it (1976) and had to attend a semester of night refresher classes at the University of Wisconsin in Milwaukee - about 60 miles from our home at the time. It was also an 8 hour exam. The morning consisted of - yes - multiple choice questions. I dont remember how many there were but they covered all engineering disciplines. Each correct answer was worth 1/2 point with a maximum possible score of 50 points. Yes - we could guess - that was a weakness in the test. The afternoon session consisted of a selection of 5 problems with 10 answers. I think these were multiple choice but they all seemed to be related, so one couldn't just guess and expect to get the rest.

"2) What method was used when you got your warrant?"

There was no "test" for a Contracting Officer's Warrant in the 1990's, Army-wide. Individual Commands may have tested. I attended the DAWIA mandatory training for Levels I and II (later Level III) and had to have a college degree, including at least 25 semester hours in various business, law and other classes, as well as a number of years of related experience in various areas of acquisition - procurement - contracting - call it what you want. As an engineer, I obtained a KO warrant that was limited to ACO authority on construction contracts because I wasn't an 1102.

I described the test methods for the engineering registration process to show the difference between various testing methods. A 50 question multiple choice exam just doesn't seem like it would really be a useful measure of competency, knowledge and skill. The test method seems to be set up for expediency and the convenience of the test providers.

Plus, I do'nt think that periodic retesting would be very useful, either. A requirement for continuous learning and regular ethics and appropriations law training is more meaningful to me.

Link to comment
Share on other sites

Guest Vern Edwards

I spent this afternoon at the Air Force Judge Advocates Conference, where I served on a panel with Ty Hughes, the USAF Deputy General Counsel, Prof. Steve Schooner of The George Washington University Law School, and Rand Allen, Chair of the Government Contracts Practice at Wiley Rein, LLP. Rand is the attorney who represented Boeing before GAO in the tanker process. We were talking about challenges in Air Force contracting.

Before our panel started I listened to a presentation by USAF litigation attorneys in which a single theme came up again and again -- the importance of getting good documentation from contracting officers: statements of fact for protests and final decisions for claims. After our presentation, in which I discussed the difficulties USAF COs have producing good documentation, I was approached by several attorneys who complained that Air Force COs could not develop well-reasoned explanations for their decisions. I spoke with one who has written guidance for COs about how to prepare statements of fact for protests and who despaired about the inability of COs to write coherently. We have seen some evidence of this in posts at Wifcon.

This is why I want essay questions for CO exams. If you cannot explain your answers in well-written essays, you are not qualified to be a CO for acquisitions in excess of the simplified acquisition threshold.

The good news is that reasoning and writing can be learned by anyone who wants to do so.

Link to comment
Share on other sites

As an intern, I am very interested and open to the testing methods prescribed by Mr. Edwards. The current ?status quo? involves assembly-line style training, which appears to be focused on presentation and speed rather than substance. One can only refer to the sheer amount of badly planned online training as an example of what is wrong with government acquisition training today.

As you all know, DAU offers most basic CON courses such as 110, 111, and 112 online. From the onset, this would make sense to a time-conscious manager. These exams have no required essays and no presentations whatsoever. As a matter of fact, a student may finish the course without taking much away due to being allowed to ?fail? the exam 3 times before failing the class. By the way, the online exams are all untimed, open-book and seem to be focused on rote-learning methods rather than critical thinking.

I will dare to go one further than Mr. Edwards and offer some sort of teamwork section of an exam where you are assigned a certain task to perform with other students. Maybe this could even be combined with an oral/presentation section of the examination? Maybe I am being a bit naive, but teamwork seems to be almost as important as self-reliance in the world of government contracts. I am sure the veterans of contracting will rip the prevois statement apart though.

A lot of people seem to think that having an exam at the end of every class substitutes the need for having one final ?bar? examination. I can?t say that I have benefited from multiple choice exams at the end of a long class. From my experience, most people tend to take away more from a course that pushes critical thinking skills rather than memorization. Lectures with no form or clarity go in one ear and out the other when hands-on learning isn't invloved. That is why I have to read and explore on my own since I can't really trust the instructor to give me everything I need.

Mr. Edwards also mentioned about how lawyers are not required to take the bar exam after one has passed. That is true, but COs are required to complete X amount of continuous learning credit hours every so many years. I have taken one of these elective courses filled with experienced COs just looking for a blow-off class just to use as filler. These ?continuous education? modules serve as nothing more than a means for external vendors to line their pockets. Most of them are 2-3 days long and offer no examination at the end.

Link to comment
Share on other sites

Motorcity, detroit rock city, you stated, "Mr. Edwards also mentioned about how lawyers are not required to take the bar exam." For the most part attorneys are required to take a bar exam except for Wisconsin (And possibly other states). When you have a moment please explain your statement.

Link to comment
Share on other sites

Will the Army exam require a regulatory citation in support of each response?

If the new Army Contracting Command testing procedures completely mirrors the Air Force Mandatory Procedures (MP) 5301.603 dated Apr 2010, it should require a citation. Per MP 5301.603, 3.3.7., "A correct answer and citation of a valid reference, to the paragraph level (e.g. FAR 15.403-1(B)), as applicable, indicating where the candidate located the answer to each question must be provided to earn two points......." Therefore, the candidate is given an opportunity to provide rationale to support the answer, which is good. However, I still think essay questions should be included in the exam so that the candidate can demonstrate her/his thought process, logic, documentation skills, and even creativity.

As to documentation skills, when I started my very first 1102 job, my CO told me documentation is essential to contracting and I still religiously believe in that. If a sound business decision is orally stated with strong supporting information, but if it is not documented in file, the job is half-done. Speech skills are as necessary as writing skills, but any oral communications that resulted in important decisions shall be followed up by detailed documentation. You need to document: to justify your business judgment; to protect yourself and your agency by storing evidence; and to leave the history of the contract in the file for those who inherit the contract from you.

Link to comment
Share on other sites

Guest Vern Edwards

What I said was that lawyers are not required to retake the bar exam.

In California, in order to be an attorney you must pass a first-year law student exam and the bar exam, Here is a sample question from 2010's first-year exam. Students were given four hours to write an answer:

Adam accepted an invitation from his friend Dot to attend a baseball game. The seats

Dot had purchased were very good, a few rows up from the field, and just past first

base. Adam had recently immigrated to the United States and knew nothing about

baseball.

During the game, a player, Brad, hit a ground ball toward third base. The third baseman

fielded the ball and threw to the first baseman. Brad thought he was ?safe,? but the first

base umpire called him ?out.? Brad began to argue with the umpire, and in frustration,

Brad threw his batting helmet to the ground. The helmet, made of a strong, hard plastic

material, bounced on the ground and flew into the stands, striking Adam on the side of

the head, causing a serious injury. Adam never saw the helmet coming towards him

because he was looking around the stadium at the time rather than at the play on the

field. As he was unfamiliar with the game of baseball, he did not know that objects

occasionally fly into the stands during a baseball game.

What arguments would Adam make in support of a claim of negligence, what defense(s)

can reasonably be asserted, and who is likely to prevail in a lawsuit filed by:

1. Adam against Brad? Discuss.

2. Adam against Dot? Discuss.

Here is the guidance provided to students taking the exam:

Your answer should demonstrate your ability to analyze the facts in the question,

to tell the difference between material facts and immaterial facts, and to discern the

points of law and fact upon which the case turns. Your answer should show that you

know and understand the pertinent principles and theories of law, their qualifications

and limitations, and their relationships to each other.

Your answer should evidence your ability to apply the law to the given facts and

to reason in a logical, lawyer-like manner from the premises you adopt to a sound

conclusion. Do not merely show that you remember legal principles. Instead, try to

demonstrate your proficiency in using and applying them.

If your answer contains only a statement of your conclusions, you will receive

little credit. State fully the reasons that support your conclusions, and discuss all points

thoroughly.

Your answer should be complete, but you should not volunteer information or

discuss legal doctrines which are not pertinent to the solution of the problem.

You should answer the questions according to legal theories and principles of

general application.

Here is a sample question from the 2010 bar exam. Candidates must answer three such questions in one hour:

On April 1, Pat, a computer software consultant, entered into a written services contract

with Danco, Inc. to write four computer programs for use by Danco in controlling its

automated manufacturing machines. The contract provided that Danco would pay Pat

$25,000 on completion of the work and that the programs were to be delivered to Danco

no later than May 1. The contract stated, ?This is the complete and entire contract

between the parties, and no modification of this contract shall be valid unless it is in

writing and signed by both parties.?

Pat entered into the contract in anticipation that it would lead to significant work from

Danco in the future, and he consequently turned away opportunities to take on more

lucrative work.

On April 15, Pat called Chelsea, the President of Danco, who had executed the contract

on behalf of Danco, and told her, ?I?m having some problems with program number 3,

and I won?t have it ready to deliver to you until at least May 8 ? maybe closer to May 15.

Also, I have some doubt about whether I can even write program number 4 at all

because your computer hardware is nearly obsolete. But I?ll get programs numbers 1

and 2 to you by May 1.?

Chelsea said in response, ?I?m sorry to hear that. We really need all four programs. If

you can?t deliver until May 15, I guess I?ll have to live with that.?

On April 28, Pat called Chelsea and said, ?I?ve worked out the problems with programs

numbers 3 and 4. I?ll deliver them to you on May 12.?

Chelsea responded, ?I?ve been meaning to call you. I?m going to start looking around

for another consultant to do the work because I consider what you said in our April 15

telephone discussion to be a repudiation of our contract. My lawyer tells me that,

because of the language in the contract, nothing I said to you in that conversation

matters. You repudiated the contract, so we don?t owe you anything.?

Can Pat prevail in a suit against Danco for breach of contract, and, if so, what is the

measure of his damages? Discuss.

Here is the guidance given to persons taking the bar exam:

Your answer should demonstrate your ability to analyze the facts in question, to tell the

difference between material and immaterial facts, and to discern the points of law and

fact upon which the case turns. Your answer should show that you know and

understand the pertinent principles and theories of law, their qualifications and

limitations, and their relationships to each other.

Your answer should evidence your ability to apply law to the given facts and to

reason in a logical, lawyer-like manner from the premises you adopt to a sound

conclusion. Do not merely show that you remember legal principles. Instead, try to

demonstrate your proficiency in using and applying them.

If your answer contains only a statement of your conclusions, you will receive little

credit. State fully the reasons that support your conclusions, and discuss all points

thoroughly.

Your answer should be complete, but you should not volunteer information or discuss

legal doctrines which are not pertinent to the solution of the problem.

Unless a question expressly asks you to use California law, you should answer

according to legal theories and principles of general application.

Those are the kind of "long form" essay question that I would give to candidates for an unlimited warrant. However, my questions would be geared to contracting officer responsibilities, not those of lawyers. For example, candidates could be given a claim and the facts involved and be required to write a final decision.

The California bar hires graders. Here are the minimum qualifications:

The Committee of Bar Examiners solicits applications from attorneys to grade the California Bar Examination twice yearly. Minimum qualifications require that applicants:

Be active members in good standing of The State Bar of California;

Have been admitted to The State Bar of California not less than one year prior to the examination for which the grader is to be selected, and:

Have passed the California Bar Examination on the first or second attempt.

Graders are paid a token amount.

Motorcity:

You are right about DAU training. DAU has been given mission impossible -- to train almost 30,000 contract specialists, which is about the number of undergraduates at one of the California universities. Even to train just the interns would be a daunting task. Moreover, they don't have a real university faculty. Most of the instructors are contracting personnel who applied for the job and were hired. They teach mainly by rote to standard instructional materials, which are not very deep. The curriculum is very poor. Online learning could be improved dramatically. They might be able to contract with a university to develop the program. But it would cost a lot and I don't know if they have the budget to do it.

Your idea about a teamwork segment to a CO exam is intriguing, but probably impractical. The main reason for using standardized multiple-choice questions is that it is a practical way to test large numbers of people. You don't need experts to grade the tests. You can do it with automation. Practicality is one of the objections to essay questions for COs: If you must test a large number of people, who will grade the essays and how can you ensure consistent grading? However, the problem is clearly solvable, as shown by the California bar exam.

Link to comment
Share on other sites

I agree with Vern on most of his points but I have had experience with a multiple choice test system that did test comprehensive knowledge and reasoning abilities. That test was the Series 7 General Securities Representative Examination, an 8 hour 2 session test that had a higher failure rate than the bar exam in California when I took the test. The questions in that test required a lot of knowledge, logical reasoning, and also an ability to understand a complex written problem in order to answer the questions.

I used more scratch paper in passing that test than there were test pages and while the test did not test my ability to communicate orally, I could not have passed it without a very good knowledge of the subject AND an ability to interpret written problems sucessfully. Would a series of essay questions improve such a test? Perhaps, but I don't think the improvement would be significant given the complexity of the questions in that test. Also, how would the subjectivity of the grading official play into the grading of an essay question? Yes, in a perfect world that would not be in question. The trouble is we do not live in such a world, as I clearly understand facing some particularly subjective grades I have received in my doctoral program over the last year or so.

Link to comment
Share on other sites

Moreover, they don't have a real university faculty. Most of the instructors are contracting personnel who applied for the job and were hired. They teach mainly by rote to standard instructional materials, which are not very deep. The curriculum is very poor. Online learning could be improved dramatically. They might be able to contract with a university to develop the program. But it would cost a lot and I don't know if they have the budget to do it.

I have wondered about what you wrote above for a number of years now. I have applied for a number of positions at DAU, emphasizing the fact that I was a doctoral student and have experience teaching at a community college. That MAY have been a mistake, as I never even received an invitation for an interview. It appears that DAU does NOT WANT real university faculty, they seem to want rote instructors who may or may not have any experience in what they are teaching. Nothing against those individuals, but it seems to be the truth from my viewpoint.

Link to comment
Share on other sites

Guest Vern Edwards
I agree with Vern on most of his points but I have had experience with a multiple choice test system that did test comprehensive knowledge and reasoning abilities. That test was the Series 7 General Securities Representative Examination, an 8 hour 2 session test that had a higher failure rate than the bar exam in California when I took the test. The questions in that test required a lot of knowledge, logical reasoning, and also an ability to understand a complex written problem in order to answer the questions.

I used more scratch paper in passing that test than there were test pages and while the test did not test my ability to communicate orally, I could not have passed it without a very good knowledge of the subject AND an ability to interpret written problems sucessfully. Would a series of essay questions improve such a test? Perhaps, but I don't think the improvement would be significant given the complexity of the questions in that test. Also, how would the subjectivity of the grading official play into the grading of an essay question? Yes, in a perfect world that would not be in question. The trouble is we do not live in such a world, as I clearly understand facing some particularly subjective grades I have received in my doctoral program over the last year or so.

My main issue with multiple choice questions is that they do not require the candidate to demonstrate his or her ability to reason and explain. It is possible to construct a very difficult multiple choice test. But the answers will not show how the candidate arrived at his or her conclusion. COs must explain their decisions in writing and those documents are subject to court review for reasonableness. The most important questions faced by COs do not always have straightforward right or wrong answers.

Link to comment
Share on other sites

Guest carl r culham

I have no argument that selection of contract specialists and purchasing agents for warranting could be improved. In a general view everything can be improved.

The argument I have with essay questions is that their scoring is subjective and can be biased, as biased as many of the posts to WIFCON itself. I have through my own experiences seen this bias with regard to essay questions. A bias is an element in issuing warrants today under current application, testing and oral interview process used by the hundreds of agencies that issue warrants. Simply the essay tests along with oral interviews are a way for folks to apply their own bias as to who should and who should not get a warrant so be careful what you ask for unless you implicitly trust your upper level acquisition personnel. Essay questions and orals might in one persons mind be applied fairly but in another person?s mind bias will occur, no doubt about it.

Discussion of including essay questions to evaluate an individual for a warrant is easy. Developing and scoring essay questions to ensure they fairly evaluate ones knowledge is a whole other matter especially across the whole of the Federal government. Take a look even today there are different standards among the hundreds of agencies even with the implied cookie cutter approach of DAWIA and FAI regarding such things as FAC-C.

In noting my concerns I do not make the claim that I am an expert in the field, I simply offer that a detailed study of the value of essay questions and their application to determining who should and who should not get a warrant and the varied levels where the essay question standard should be applied needs a complete analysis and evaluation before applying across the board.

Example - Is the standard the same for the 1105 purchasing agent seeking a warrant at the SAT, and for the senior management level person who by position is simply given warrant authority (and for all the levels in between)?

My conclusion is that in an organization as large and varied as the Federal government the objective test should be give priority and not the subjective test implied by a standard of essay testing in part or in full.

In keeping with intent of posts made in WIFCON discussions I provide one reference: http://www.personal.psu.edu/faculty/s/r/sr.../essayexams.htm

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...