Jump to content
  • entries
    56
  • comments
    306
  • views
    355,485

Should Training Classes Be Required?


Don Mansfield

7,882 views

It’s time we rethink our approach to the training problem. Our traditional approach is to dictate a blueprint of training classes that must be followed in order to obtain prescribed levels of certification. To put it in acquisition terms, we’ve been using a design specification. What if we were to use a performance specification instead? What might that look like? Before answering these questions, we should identify what it is we are trying to achieve with training.

The purpose of training is to make the trainee proficient in performing one or more defined learning objectives by means of specialized instruction and practice. A learning objective consists of three parts—an action, a condition, and a standard. For example, someone new to federal contracting will likely receive training to select FAR provisions and clauses (action) given a set of facts about an acquisition and access to the FAR (condition) (the implied standard would be “correctly”). It follows that if an individual already behaves in the prescribed way under the prescribed conditions to the prescribed standard, then training would be unnecessary for that individual—they’ve already attained the learning objective.

Under the “design specification” training model that we currently use, there is an implied assumption that an individual cannot attain the requisite learning objectives without following the prescribed blueprint of training classes. Further, there is no method of demonstrating the attainment of the requisite learning objectives prior to the prescribed training classes. As a result, everyone must take the required training classes, regardless of individual necessity. Considering the resources involved in carrying out such a program, this is an expensive proposition.

While the implied assumption of the “design specification” training model may prove true in some cases, a more reasonable assumption would be that some individuals need to follow the prescribed training blueprint and some do not. Those that do not would include those that have already attained the requisite learning objectives by other means and those that could without following the prescribed training blueprint. Thus, the challenge would be to identify those that don’t need a particular training class before requiring their attendance at the training class.

What if we borrowed the thinking behind performance-based acquisition and applied it to the training problem (i.e., a “performance specification” training model)? That is, instead of dictating how the workforce is to attain requisite learning objectives, we specify the requisite learning objectives (performance outcomes) and method of assessment, and let the workforce decide how they are going to attain them. Some workforce members may choose a program of self-study, others may study in informal groups, some contracting offices may develop their own ongoing training programs, etc. Still others may choose to follow the existing blueprint of training classes. Regardless of how one attains the requisite learning objectives, all are held to the same standard using the same method of assessment.

For an illustration of how such a model might look, consider the profession of actuarial science. Beanactuary.org contains the following description:

Like other top-ranked professions (such as law and medicine), one must pass a set of examinations to achieve professional status as an actuary. Unlike other professions, in actuarial science you’ll have the opportunity to work as an actuary while completing the examination process—employers often allow study time during working hours, pay exam fees, provide internships, and even award raises for each exam passed. Though, to get the best start on a rewarding career, many soon-to-be actuaries begin taking exams while still in college. Of those that do, most achieve associateship in three to five years. All candidates acquire a core set of knowledge from required preliminary exams. The preliminary exams and Validation by Educational Experience requirements are the starting points for an actuarial career.

To attain an “Associate of the Society of Actuaries” (ASA) designation from the Society of Actuaries, one must pass exams in probability, financial mathematics, models for financial economics, models for life contingencies, and construction and evaluation of actuarial models. In addition, there is one required e-Learning course and a required one-day seminar in professionalism. After attaining the ASA designation, one typically pursues a “Fellow of the Society of Actuaries” (FSA) designation within one of six specialties: corporate finance and enterprise risk management, quantitative finance and investment, individual life and annuities, retirements benefits, group and health, and general insurance. To attain the designation, the FSA candidate must take 3-4 more exams unique to the specialty, complete four e-learning courses, and attend a three-day case-based fellowship admissions course that requires each candidate to deliver an oral presentation on a topic within the field. In case you weren’t keeping track, that’s a total of four days of required attendance in classrooms to achieve the highest designation in the field. In contrast, DoD contract specialists must attend 32 days of classroom training to attain the lowest level of certification.

What if to attain level 1 certification in contracting, one had to pass exams in, for example: acquisition planning, contracting methods, contract types, socioeconomic programs, and contract administration, and attend a one-day seminar on ethics? After level 1, contract specialists would choose a specialty in which they would pursue Level 2 certification. Specialties would be, for example, major system acquisition, research and development contracting, construction and A/E contracting, service contracting, IT acquisition, acquisition of commercial items, contract administration, etc. To attain Level 2 certification, contract specialists would have to pass a series of exams unique to that specialty. For example, to attain Level 2 certification in service contracting, there would be exams on specification of service requirements, source selection for services, pricing services, and service contract administration. There could also be a Level 2 admissions course where the candidate would have to submit and present a paper on a topic related to their specialty.

If nothing else, use of the performance specification training model would cost less than the design specification model currently in use. I would go as far as to say that, on the whole, the workforce would be at least as competent as it is now.

What’s your opinion? We’d like to know.

33 Comments


Recommended Comments



I need to think about your topic some more in regards to the training construct, but one thing I can say clearly and definitively - whatever portion of the DoD's DAU/training budget it takes would be better spent on providing the acquisition workforce access to quality publications/resources for continuous education (e.g. The Cibinic & Nash Report, Briefing Papers, etc.)

Link to comment

I believe the current method of assessment, or lack thereof, is a primary concern. I think separating certifications and standards into "specialties" is a worthy discussion.

DAWIA coursework and certification standards do not produce talented contracting professionals. What we need is a certification based on demonstrated knowledge and competence. We should not be focused on years of experience or courses taken, per se.

Reminds me, what is the status of the Aquisition Qualification Workforce Initiative (AQWI)?

Quote

Further, there is no method of demonstrating the attainment of the requisite learning objectives prior to the prescribed training classes.

I have taken computer-based training modules that offered a pre-test. Depending on how well you performed, the training lesson was modified to only address the areas you tested ineffectual in.

Link to comment

So if I understand the proposed solution correctly, if an 1102 were to move from an office that did installation support (like a military installation) using primarily Simplified Acquisition to an office with another agency (or perhaps the same one) and started working A/E actions, they would need to complete an additional Level II course.

How would this work in smaller offices where the 1102 has to do several types of contracting?  When I started out at a military installation I did Simplified Acquisition, Construction, Services and IT Acquisition all on any given day.  Under this scenario would I have had to be Level II qualified in all these areas?  I think having specialized certifications would create problems for the agencies and the workforce.

Although the current system is far from perfect, if it is going to be changed it will need to be flexible enough to facilitate all types of offices. With all its faults (and I admit there are many), the current system does provide some basic foundational contracting principles that can expand to most (if not all) types of contracting. The costs to administer a specialized program is something else that would have to be considered. With the trend of movement of 1102s that I've seen over the last few years, people would be constantly taking training/tests for new certifications. 

Intriguing idea, but I don't know how it would work from a practical standpoint.

 

Link to comment
Guest PepeTheFrog

Posted

this is pure fantasy, but you asked for opinions:

garbage in, garbage out: bring back a rigorous Civil Service Exam (this will help all Federal career fields, and it works for foreign service officers)

constipation: do everything possible to come closer to at-will employment for Federal employees (this will help all Federal career fields)

Federal agencies need to decide if they want to be (a) efficient, effective, and respected or (b) soup kitchens and vast, wasteful job programs

find somebody who was alive in the 1950s and 1960s and ask them how opinions of the quality of Federal employees (prestige, respect, assumptions of competence) has changed since those times

Link to comment
4 hours ago, Desparado said:

So if I understand the proposed solution correctly, if an 1102 were to move from an office that did installation support (like a military installation) using primarily Simplified Acquisition to an office with another agency (or perhaps the same one) and started working A/E actions, they would need to complete an additional Level II course.

Under the current model, the new office may require the employee take an A/E course that would include an exam at the end (like CON 243). Under the proposed model, the new office would require that the employee pass an exam similar to the one given in CON 243 instead. No required attendance in a classroom.

4 hours ago, Desparado said:

How would this work in smaller offices where the 1102 has to do several types of contracting?  When I started out at a military installation I did Simplified Acquisition, Construction, Services and IT Acquisition all on any given day.  Under this scenario would I have had to be Level II qualified in all these areas?  I think having specialized certifications would create problems for the agencies and the workforce.

"Installation contracting" could be a specialty all its own--like a multidisciplinary major.

4 hours ago, Desparado said:

Although the current system is far from perfect, if it is going to be changed it will need to be flexible enough to facilitate all types of offices. With all its faults (and I admit there are many), the current system does provide some basic foundational contracting principles that can expand to most (if not all) types of contracting. The costs to administer a specialized program is something else that would have to be considered. With the trend of movement of 1102s that I've seen over the last few years, people would be constantly taking training/tests for new certifications. 

Intriguing idea, but I don't know how it would work from a practical standpoint.

I do not see any additional costs for what I am proposing. I do see savings each time a student opts not to attend a class and prepares for the examination on their own, though.

Link to comment

I'm not sure I agree with your definition of "learning objective." From another source I read that a learning objective consists of

1. A description of what the student will be able to do

2. The conditions under which the student will perform the task.

3. The criteria for evaluating student performance.

That definition seems more practical and less theoretical than your definition.

Given the (revised) definition, it seems that any learning that is "one size fits all" is not meeting the second part of the definition -- i.e., the learning needs to address the conditions under which the student will perform the task. Thus, it seems we need to tailor the training.

I was also struck by your use of actuaries rather than accountants in your example. Once an accountant passes the CPA and other related tests they are good to go -- but need to document a certain amount of continuing professional education each year. (If memory serves it's 120 hours over 3 years with no more than 80 counting in any one year.) Again, though, we are talking about a rigorous examination that a certain amount of takers are expected to fail. But once over the hurdle, you are in the profession.

Link to comment

Don,

I like the idea of it, but how would you deal with that in today's workforce. I can only speak to my experience within DOD, but I doubt that it is different in the civilian world, given that some of my peers came from that world, I just have never worked in it so I can't speak to it directly. That being said... I find that each service, and often each MAJCOM within the service views things differently and therefore you can ask folks in each different group the same question and get different responses. A good example of this is what satisfies the requirement of the "determination" required by FAR 17.207(d). I've been at several agencies, and talked to and witnessed multiple KOs state that this determination means a D&F, as required by FAR 1.704, when pressed on the issue, as I learned early on in my career after reading one of Vern's posts on that very subject, the ultimate answer for why do you think that a formal D&F is required is... because that's how we do it here or that's what our policy office says. 

IMO no amount of training, or no style of training will be able to overcome the agency thats how we do it... especially if the application at the local level is incorrect because someone that goes to training and learns all these great things comes back is either discouraged because they hear "I don't care how DAU taught you... this is how we do it" or they start first at their agency, and then go to training, and can tell that there are differences in what is being taught, and get cross threaded because they cannot distinguish between these differences. 

I've always been curious... there seems to be quite a focus on the lack of training of today's workforce what's changed from how you learned contracting, and what would it take to go back to that model? 

Link to comment

It all starts at the hiring.  Post the right requirements, ask the right interview questions, and do a thorough background check.  Pre-hiring-test are not illegal; for example, the FDIC does a three-day interview process where the applicants have to do writing and editing tests, solve collaborative problems, and attend a formal dinner.

P.S.  I was once asked at a contracting job interview "with what comic book character do you identify the most?"; the other questions did not ask much about the actual contracting tasks either...

Link to comment
17 hours ago, here_2_help said:

Given the (revised) definition, it seems that any learning that is "one size fits all" is not meeting the second part of the definition -- i.e., the learning needs to address the conditions under which the student will perform the task. Thus, it seems we need to tailor the training.

If you're saying that it's necessary to tailor the conditions to the learner, I don't agree. In a learning objective, a condition is simply a description of the resources or tools the learner will need to complete the measurable or observable behavior. A condition is usually established with the word "Given" followed by such phrases as

  • a set of questions
  • a scenario
  • a series of problems
  • a visual representation of an object
  • a list of conditions
  • a choice between,,,
  • materials, or
  • a calculator.

Some instructional designers think of it in terms of the conditions of testing (or assessment), because they are trying to communicate the conditions under which the student will be asked to demonstrate mastery of the knowledge and/or skills learned. In my experience, the most important condition for students is whether the exam is open or closed book.

17 hours ago, here_2_help said:

I was also struck by your use of actuaries rather than accountants in your example. Once an accountant passes the CPA and other related tests they are good to go -- but need to document a certain amount of continuing professional education each year. (If memory serves it's 120 hours over 3 years with no more than 80 counting in any one year.) Again, though, we are talking about a rigorous examination that a certain amount of takers are expected to fail. But once over the hurdle, you are in the profession.

I'm more familiar with the actuary field. From what you described, it seems like the accounting field already uses the performance-based model--successful completion of required exams instead of attendance at required courses. Seems to work ok.

Link to comment
15 hours ago, uva383 said:

Don,

I like the idea of it, but how would you deal with that in today's workforce. I can only speak to my experience within DOD, but I doubt that it is different in the civilian world, given that some of my peers came from that world, I just have never worked in it so I can't speak to it directly. That being said... I find that each service, and often each MAJCOM within the service views things differently and therefore you can ask folks in each different group the same question and get different responses. A good example of this is what satisfies the requirement of the "determination" required by FAR 17.207(d). I've been at several agencies, and talked to and witnessed multiple KOs state that this determination means a D&F, as required by FAR 1.704, when pressed on the issue, as I learned early on in my career after reading one of Vern's posts on that very subject, the ultimate answer for why do you think that a formal D&F is required is... because that's how we do it here or that's what our policy office says. 

IMO no amount of training, or no style of training will be able to overcome the agency thats how we do it... especially if the application at the local level is incorrect because someone that goes to training and learns all these great things comes back is either discouraged because they hear "I don't care how DAU taught you... this is how we do it" or they start first at their agency, and then go to training, and can tell that there are differences in what is being taught, and get cross threaded because they cannot distinguish between these differences. 

I've always been curious... there seems to be quite a focus on the lack of training of today's workforce what's changed from how you learned contracting, and what would it take to go back to that model? 

Every professional has to deal with the tribal customs of the organization they work for. I am familiar with the type of problem you described and have experienced it myself. Recently I visited a tribe, let's call them the IDIQ tribe, where the shaman couldn't understand that a contract specifying a definite quantity didn't have a "ceiling". All she could see was evil spirits when I tried to explain. This is a different problem that may or may not be solved by what I'm proposing. 

I don't think training used to be good and now it is bad. The major difference between then and now is delivery--some training is now Web-based. I don't see a significant difference in quality.

Link to comment

Don,

The problem is that shaman might be the one selected to write the module that will grade everyone else's competence in fair opportunity considerations.  Or, to reference an earlier example, the dimwit who thinks the determination to include options has to be in a FAR Subpart 1.7 D&F format might (likely will) be selected for that module.  I fully support the idea, but I would likely be deeply troubled by the implementation details.

Link to comment
56 minutes ago, ji20874 said:

Don,

The problem is that shaman might be the one selected to write the module that will grade everyone else's competence in fair opportunity considerations.  Or, to reference an earlier example, the dimwit who thinks the determination to include options has to be in a FAR Subpart 1.7 D&F format might (likely will) be selected for that module.  I fully support the idea, but I would likely be deeply troubled by the implementation details.

I think that the exams should be centrally developed, administered, and controlled to mitigate cultural bias. I would keep it at the DAU level for DoD and FAI level for civilian agencies.

Link to comment

Don,

I agree that web based traing is ineffective for most modules as I have observed many of my peers just click through the screens to just get through the material or worse, just print the modules into .pdfs and do controlled searches for the answers to pass the exams at the end. 

I suppose my more basic question that I was trying to ask in my previous post is... Obviously something has changed from the way that most of my senior seasoned (15+ years of experience) 1102s learned contracting and my peers at the journeyman (5-15 years of experience and the  entry level folks (1-5 years). The seasoned folks I talk to often talk about how training was more individualized, that it was almost agency specific and although there was DAU or the AF service level training, when you got back to your agency you got the "tribal" position on the material you just learned. They (the seasoned folks) have said that the shift really occurred in the late 90s when the 1102 workforce was drastically cut, before those cuts, how was training handled and is it possible to go back to that format? I don't think learning at the college level has changed, although I do think that we have really struggle to teach the ability to think.

Would you agree that the methods of training today's workforce are different than it was 15-20 years ago, and are these methods ineffective and should we attempt to return to older formats? Or has the issue of an largely poorly trained workforce always been there and you just had more knowledgable individuals that could carry the weight and mask the issue(s)?

I can only speak anecdotally on the subject of the differences because I was trained in the current DAU format but based on the way that my more seasoned mentors talk about how they were brought up, I can see a difference between then and now.

I imagine that if we could get to a format where training is specialized almost from the very beginning based on types of procurements (systems, operations, services) rather than what feels like the lowest common denominator, that would help. And then maybe have the 1102 choose the format and then have to be tested and certified at different levels for the types of procurements, similar to what you were describing, that could help. 

Link to comment

Don,

There is tribalism and incompetence and small-world-view within DAU, too.  We would probably need an independent board of some sort to develop and update the test -- but who goes on that board?  Still, I like the idea of a test of professional knowledge.

 

uva,

I agree that the current crop of DAU web-based training is ineffective -- the in-person training is, too, to a large extent -- we send people to those courses to check boxes for certification rather than with any expectation of meaningful learning.  On this regard, DAU as a whole had been a total failure.*  But still, I believe web-based training can be effective if done right.

Certainly, we don't need one master test -- I much prefer small tests and smaller certifications.

*I mean no disrespect to past or current DAU staff -- just an honest observation in the hope of meaningful discussion.  

Link to comment

One other thought -- we cannot improve our professionalism as long as our executives care nothing about it.  When the SPEs and HCAs and people at their level start caring about professionalism in our career field, then they can be responsible for the oversight of the tests -- but we simply cannot leave the matter to DAU.  

Link to comment

The standardized tests issue has been debated for several decades and there are good arguments both in favor and against one size fits all tests.  In Federal permanent employees is too late after the person is hired into a different position to find out the person cannot pass a test because you are stuck with that employee for a long time.  For example, I had a coworker who could not pay her bills, lost the security clearance, could not touch a Government computer, and for over a year all she did was shred paper while her workload was distributed among the other workers in the office.  Management tried to get rid of her but could not, and a little over a year later she received a new clearance and back to business again.

On the other hand, tying promotions more to performance and test results than to time in grade has possibilities.  One problem with the promotion idea is that in Federal employment promotion usually means moving out to a different agency, and that takes us back to fixing the hiring system.  Passing the skills/knowledge test prior to interview, asking questions that are relevant to the position you are trying to fill, evaluating the applicant under pressure to assess the applicant emotional intelligence, and doing a thorough background (references) check should increase your chances of hiring the right person for the position.

Link to comment

Howdy, Don,

I think the best bang-for-buck reform is separating the certification and the training functions entirely.  Not my idea- I heard it from Arnold Kling, but he probably heard is from elsewhere too.  I'm attracted to is because it's simple, but I think colleges tend to hate it.  This is almost identical to what you propose except you would have to pick what side of the instruction/certification fence you wanted to be on and stay there.  The student would not have an option to hire you for both.

 

Link to comment
11 hours ago, uva383 said:

Would you agree that the methods of training today's workforce are different than it was 15-20 years ago, and are these methods ineffective and should we attempt to return to older formats? Or has the issue of an largely poorly trained workforce always been there and you just had more knowledgable individuals that could carry the weight and mask the issue(s)?

I went through DAU training ~20 years ago and the difference was that everything was in the classroom. I guess it was more individualized in the sense that I interacted with an instructor who was physically present instead of interacting with a Web-based training module. The move to distance learning at DAU began about 15 years ago. Some felt that the quality of the training suffered. Shay Assad thought DoD should go back to the classroom for introductory training and that's why we now have CON 090. I think that was an improvement, but to go back to my original analogy, all we did was modify the design specification. So, no, I don't think we should just return to older formats. I think we should let the employee choose the format and hold them accountable for the results.

Link to comment
On 8/2/2016 at 1:59 PM, Don Mansfield said:

I do not see any additional costs for what I am proposing. I do see savings each time a student opts not to attend a class and prepares for the examination on their own, though.

The additional costs that I was thinking of are for all these 1102s that switch specialties. Many agencies use the silo approach to contracting. Under this scenario when someone gets a promotion or transfer and moves from products to construction, or to services, or to some other specialty they would need to get another certification. If they don't test out, that means more dollars for training. I'm not saying that's a bad thing, but the extra cost is something that would need to be figured into the decision process.

Link to comment
10 hours ago, Don Mansfield said:

Jamaal,

You're still just tweaking the design specification.

Agreed, but if we acknowldege we have a crummy design specification (including conditions and standards) that generates lousy results, why not fix it. The design specification model is not to blame - the design specification designer and approver; or implementer are at fault. Define and implement a design specification that satisfies your requirement.

I guess my real question is - what problem are we trying to fix and what are the root causes of that problem. (Training problem).

I don't agree that there is an implied assumption that an individual cannot attain the requisite learning objectives without following the prescribed blueprint of training classes. Its not because they cannot attain the learning via other means ... but I dont think that is where you want this thread to go so I'll park it.

Link to comment

Don,

under your performance format, how would you account for the different learning styles for different material. Take a class like 170 or 270, very math intensive. Would a student that chose the web based instruction for material like contract law or acquisition planning be able to switch to classroom for a math based class? Or would they be locked into web based instruction for the entire certification?

Under this format how would you tie performance and successful or unsuccessful outcomes to the employee and the instructor? Would you recommend termination of an employee that cannot pass the classes? Also, given the current shortage of instruction in the classroom setting as demonstrated by reduced class offerings and long waitlists, how would your new format allow for scheduling issues or having to go outside of the DAU curriculum to obtain equivalency for some limited classes? 

Link to comment

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...