Test Questions in Civil Service Hiring Announcements

[WifWaf]

There should be another way to hire Contracting Officers besides the current "rate yourself" questionnaires on USAJobs.  If we instituted test questions instead, what would your best one be?

First, read this primer to my question.  The following problems identified by Vern Edwards on WIFCON in 2005 are still pending resolution in many contract shops:

Quote

Pressured by reformers to serve customers (who are always right), unequipped and unprepared to assist clients (who rely on professionals to decide what to do and to act on their behalf), forced by personnel reductions to take over clerical tasks, and demoralized by their reduction in status to administrative assistants to program personnel, too many contracting officers traded poor bureaucratic practices for poor but expeditious practices.

These were his proposed solutions back then:

Quote

The acquisition reform of the 1990s...was not the kind of reform that we most desperately need, which would focus on developing and maintaining a corps of professionally knowledgeable and competent contracting officers, people who could obtain best value by awarding well-written contracts expeditiously and administering them effectively, all the while maintaining their integrity, earning the respect of program officials and contractors, and capturing and keeping the public’s confidence and trust. That kind of reform would have been a lot harder to accomplish, but would have been immensely more effective and lasting.

See: https://www.wifcon.com/anal/br5.htm

I'll go first with a question.  "Where can we find one link to a SAM.gov Contract Opportunity demonstrating a recent well-written solicitation you issued?  Please provide the URL."

[C Culham]

20 hours ago, WifWaf said:

If we instituted test questions instead,

"Test" questions to me imply a right or wrong answer.   Yet the idea of questions that key in on experience makes sense.  How about this one - How many contracts have you been involved with in awarding and of that number how many have you personally been involved in negotiating.

But the question for you is what would be your question if it were a new hire?  What is your question be?

[formerfed]

I personally assign very little significance to written job responses.  While that might be a good way to screen applicants to a manageable number, there’s nothing like face-to-face interviews with challenging questions to get to the heart of the matter.  Then follow that up with reference checks.  Often the best sources of information is not the current supervisor but a prior one.  Sometimes a current supervisor won’t provide candid responses for either fear of losing a valuable worker or motivated to get rid of someone.  There are workers who are excellent at providing the right answers but are poor at actually doing the work.  

[joel hoffman]

I think that a face to face (if possible/practical) interview with promising applicants would be a more appropriate place to do something like what WifWaf proposes. For involved responses like the “test” here, I would inform the candidates when making the interview appointment, so they can be prepared think about and can prepare for the interview. 

[Vern Edwards]

I read an article by a person who had been personally interviewed by Jeff Bezos in the finally stages of a job application process. Bezos asked him one question:

What is your estimate of how many panes of glass there are in Seattle? 

Besos asked him to talk through his estimating process, and stood at a white board to record the interviewee's process and do the math.

Quote

I was momentarily terrified.

Then I paused to calm down, reminding myself to think about his motivation for asking me that question. He wants to see the way my mind works, I told myself. He wants to see me break down a complicated problem into small, manageable steps. I can do that.

I outlined how I would start with the number of people in Seattle, which I thankfully correctly guessed as around 1 million, just to make the math easier. Then I said that they would each have a home, a mode of transportation, and an office or school — all of which would have windows. So I suggested that we base the estimate on averages of those.

Then we did the math.

We got down into every possible scenario, group, anomaly and ways to account for these exceptions. It felt like I talked it through for hours while Bezos filled the whiteboard with numbers. I’m sure it actually took more like 10 minutes.

I remember feeling a thrill when he wrote down the final estimate. He circled it. “That looks about right,” he said.

The applicant successfully answered the question by showing Bezos a rational thought process.

A "test" question does not have to have a right or wrong answer. In fact, such questions don't show much imagination. A better purpose might be to learn how an applicant thinks, or whether they think at all. Another good technique is to present an applicant with a scenario and then ask, "What if any questions do you have?"

Voltaire thought it was better to judge a person by their questions than by their answers.

[WifWaf]

3 hours ago, C Culham said:

"Test" questions to me imply a right or wrong answer.   Yet the idea of questions that key in on experience makes sense.  How about this one - How many contracts have you been involved with in awarding and of that number how many have you personally been involved in negotiating.

But the question for you is what would be your question if it were a new hire?  What is your question be?

No tests in new hire job announcements.  Have to avoid the essay-writing contest.  I think you bring new hires in on the condition that probationary periods will by default lead to removal from Government service unless the newbie demonstrates a rational thought process sometime in the first year of hire.  Switch it from a guarantee to a test/trial period.  COCO schedules a Bezos-like meeting to assess the newbie at Progress Review of year 1 Appraisal Period.

[Vern Edwards]

It's all about cultivated curiosity.

[formerfed]

1 hour ago, WifWaf said:

No tests in new hire job announcements.  Have to avoid the essay-writing contest.  I think you bring new hires in on the condition that probationary periods will by default lead to removal from Government service unless the newbie demonstrates a rational thought process sometime in the first year of hire.  Switch it from a guarantee to a test/trial period.  COCO schedules a Bezos-like meeting to assess the newbie at Progress Review of year 1 Appraisal Period.

Good idea but that likely won’t work.  It was tried before in various forms and HR offices always killed it.  They said decisions to remove someone in the trial periods must be based in part against the OPM 1102 standards for the applicable grade.  The basis must be something more concrete.

[WifWaf]

2 hours ago, formerfed said:

Good idea but that likely won’t work.  It was tried before in various forms and HR offices always killed it.  They said decisions to remove someone in the trial periods must be based in part against the OPM 1102 standards for the applicable grade.  The basis must be something more concrete.

Dang.  Does anybody on this website know someone in HR at OPM? 😄

[Vern Edwards]

I think formerfed is basing his comments on old experience. I can tell you of agencies who have cut probationary employees based on disappointing performance. It all depends on the agency, the office, and relations with human resources.

I like your thinking, WifWaf.

[formerfed]

@Vern Edwards That’s correct.  I based my comments on outdated experience.  I see now things are much different

https://www.opm.gov/policy-data-oversight/hiring-information/practical-tips-for-supervisors-of-probationers/#:~:text=The probationary period is the,their fitness for continued employment.

@WifWaf I think you can you successfully do that.  As a word of caution, I would be clear with expectations for the new hire.  You should be good.

[WifWaf]

Thank you, all.

[RJ_Walther]

I liked "Think of a large purchase you've made in your personal life, like a house or a new car. Describe how you made that purchase?" as an interview question. The goal was to see what relevant techniques (market research, negotiation, making trade-offs, etc.) they could explain and apply without falling back to just referencing the FAR.