Thirty+ Years of Developing and Assessing COMPETENT PROJECT PRACTITIONERS using Project Based Learning- Notes from the Field and “Lessons Learned”
INTRODUCTION
This paper was originally presented at the 2nd annual University of Maryland Project Management Symposium in College Park, Maryland, the USA, in June 2015. It was republished as a “2nd Edition” in the PM World Journal, Vol. IV, Issue VII — July 2015 “Producing Competent Practitioners Using Project Based Learning” and is being updated to reflect changes in the underlying specifications and modifications based on “Lessons Learned” over the past 7 years.
For background information, the author is not an academic. He attended an agricultural trade school (Future Farmers of America- FFA) and, after graduation, joined the carpenters union, where he progressed through the construction trades from apprentice to journeyman to master carpenter and builder, as well as being a PADI SCUBA diving instructor for 35 years along with being a private pilot with 190 hours flying the Alaska Bush, and for the past 30 years has been developing and delivering Competency-Based training and Capacity Development to mid-career path professionals around the world in applied Asset, Portfolio, Program and Project Management.
His first exposure to “project-based learning” at the university level came while an undergraduate at Worcester Polytechnic Institute during the early 1970s. Back then, WPI was one of the “early adopters” of “Project-Based Learning” and is now recognized as being one of the global leaders in the field with their “Center for Project-Based Learning.”
Having grown up in a “project-based environment,” it was only natural that as his career evolved, he built around his life experiences in project management, and now as he approaches retirement age, he would like to share these experiences and “lessons learned” as both a practitioner as well as a recognized academic in the field with others. Sharing what he has learned along the way hopefully will make your job as a “project professional” a little bit easier and more likely to be “successful”?
This paper is designed to illustrate Step by Step how he and his team have developed and delivered COMPETENT PRACTITIONERS using Project Based Learning by applying the 7 Project-Based TEACHING PRACTICES to create a tested and PROVEN 7 Project-Based Design Elements[1]
This paper also challenges the underlying CONTENT, noting that since at least the 1996 PMBOK Guide, PMI advocated “those practices, tools, and techniques used on MOST projects MOST of the TIME,” meaning “average” practices? Then we wonder why, in 36 years the PMBOK GUIDE has been in circulation, we have yet to see any measurable improvements in project success rates?
For 10+ years, we have been compiling and publishing:
- Over 1300 pages of “best-tested and PROVEN PRACTICES;.”
- 700+ pages of tables, charts, graphs, and images;
- 200+ Excel and Word TEMPLATES;
- Accessible for VIEWING by ANYONE at NO COST;
- To DOWNLOAD requires paying a nominal membership fee (adjusted for PPP using Big Mac Index);
- Under Creative Commons Licence Attribution-Shar Alike (BY SA) v 4.0.
Our E-book is updated QUARTERLY and is open for ANYONE to CONTRIBUTE and for ANYONE to COMMENT or CRITIQUE. (NO “GATE-KEEPERS”) (Som Conditions/Constraints APPLY)
To access our FREE “PMO Handbook of “Best Tested and PROVEN PRACTICES”-
1) Click HERE- https://build-project-management-competency.com/
2) Click on the “Practice Standards” Tab
3) Select 1 or MORE of the 14 Units Tab:
1. Unit 1- Governance & Integration
2. Unit 2- Managing People
3. Unit 3- Managing the Business Case
4. Unit 4- Managing Scope
5. Unit 5- Managing Quality Assurance/Quality Control
6. Unit 6- Managing Risk & Opportunity
7. Unit 7- Managing Resources
8. Unit 8- Managing Contracts
9. Unit 9- Managing Planning and Scheduling
10. Unit 10- Managing Cost Estimating and Budgeting
11. Unit 11- Managing Progress
12. Unit 12- Managing Change
13. Unit 13- Managing Data
14. Unit 14- Managing Forensics
INTRODUCTION
This paper was originally presented at the 2nd annual University of Maryland Project Management Symposium in College Park, Maryland, the USA, in June 2015. It was republished as a “2nd Edition” in the PM World Journal, Vol. IV, Issue VII — July 2015 “Producing Competent Practitioners Using Project Based Learning” and is being updated to reflect changes in the underlying specifications and modifications based on “Lessons Learned” over the past 7 years.
For background information, the author is not an academic. He attended an agricultural trade school (Future Farmers of America- FFA) and, after graduation, joined the carpenters union, where he progressed through the construction trades from apprentice to journeyman to master carpenter and builder, as well as being a PADI SCUBA diving instructor for 35 years along with being a private pilot with 190 hours flying the Alaska Bush, and for the past 30 years has been developing and delivering Competency-Based training and Capacity Development to mid-career path professionals around the world in applied Asset, Portfolio, Program and Project Management.
His first exposure to “project-based learning” at the university level came while an undergraduate at Worcester Polytechnic Institute during the early 1970s. Back then, WPI was one of the “early adopters” of “Project-Based Learning” and is now recognized as being one of the global leaders in the field with their “Center for Project-Based Learning.”
Having grown up in a “project-based environment,” it was only natural that as his career evolved, he built around his life experiences in project management, and now as he approaches retirement age, he would like to share these experiences and “lessons learned” as both a practitioner as well as a recognized academic in the field with others. Sharing what he has learned along the way hopefully will make your job as a “project professional” a little bit easier and more likely to be “successful”?
This paper is designed to illustrate Step by Step how he and his team have developed and delivered COMPETENT PRACTITIONERS using Project Based Learning by applying the 7 Project-Based TEACHING PRACTICES to create a tested and PROVEN 7 Project-Based Design Elements[1]
This paper also challenges the underlying CONTENT, noting that since at least the 1996 PMBOK Guide, PMI advocated “those practices, tools, and techniques used on MOST projects MOST of the TIME,” meaning “average” practices? Then we wonder why, in 36 years the PMBOK GUIDE has been in circulation, we have yet to see any measurable improvements in project success rates?
For 10+ years, we have been compiling and publishing:
- Over 1300 pages of “best-tested and PROVEN PRACTICES;.”
- 700+ pages of tables, charts, graphs, and images;
- 200+ Excel and Word TEMPLATES;
- Accessible for VIEWING by ANYONE at NO COST;
- To DOWNLOAD requires paying a nominal membership fee (adjusted for PPP using Big Mac Index);
- Under Creative Commons Licence Attribution-Shar Alike (BY SA) v 4.0.
Our E-book is updated QUARTERLY and is open for ANYONE to CONTRIBUTE and for ANYONE to COMMENT or CRITIQUE. (NO “GATE-KEEPERS”) (Som Conditions/Constraints APPLY)
To access our FREE “PMO Handbook of “Best Tested and PROVEN PRACTICES”-
1) Click HERE- https://build-project-management-competency.com/
2) Click on the “Practice Standards” Tab
3) Select 1 or MORE of the 14 Units Tab:
1. Unit 1- Governance & Integration
2. Unit 2- Managing People
3. Unit 3- Managing the Business Case
4. Unit 4- Managing Scope
5. Unit 5- Managing Quality Assurance/Quality Control
6. Unit 6- Managing Risk & Opportunity
7. Unit 7- Managing Resources
8. Unit 8- Managing Contracts
9. Unit 9- Managing Planning and Scheduling
10. Unit 10- Managing Cost Estimating and Budgeting
11. Unit 11- Managing Progress
12. Unit 12- Managing Change
13. Unit 13- Managing Data
14. Unit 14- Managing Forensics
Before we get into the “How to Do It” instruction, this is an appropriate time to introduce the definition of COMPETENCY. Based on a restatement of the Merriam-Webster Online Dictionary, this is the definition of COMPETENCY used in this paper.[2]
As can be inferred from this picture, “competency” consists of multiple levels- the competency necessary to earn your first driver’s license when you turn 16 or 17, validating that you can drive the family sedan around town does NOT qualify you to drive a semi-trailer rig or a piece of heavy construction equipment or that obtaining your private pilot’s license qualifies you to fly a Boeing 747 or Airbus 380.
Implicit in this is that there are progressively more challenging” competency levels.” Also worth keeping in mind that “COMPETENCY” is a snapshot at a single point in time- that just because you got your driver’s license this morning does not mean you are unlikely to get in an accident this afternoon.
To paraphrase from the financial investment folks, being “COMPETENT” today is no guarantee of remaining COMPETENT into the future.” Interpreted literally, it means “competency” is not an end but a journey- that to become and remain competent means perpetually striving to improve or upgrade your “aptitude, attitude, skills, strengths and knowledge.”
Another fundamental truth underlying the concept of Competency development and assessment is that you cannot build and maintain COMPETENCY or develop CAPACITY by reading books, watching Youtube videos, or analyzing case studies. The only way to build, develop and maintain COMPETENCY is by DOING.
This is why pilots are required to maintain a CURRENT license. In order to maintain currency on a given aircraft type and fly it while carrying passengers, a pilot must have performed three takeoffs and landings within the previous 90 days[4].
[1] Buck Institute’s Gold Standards for Project Based Learning and Teaching https://www.pblworks.org/what-is-pbl
[2] Merriam-Webster Online Dictionary
[3] By Author from Course Materials
[4] Tom Benenson (May 13, 2021) “The Strategy of Staying Current“ https://www.flyingmag.com/strategy-staying-current/
[5] By Author from Course Training Materials
The “Big Picture”- The 6 Specifications or Standards to Design & Deliver Courses.
To realize the full value of Project-Based Learning and to develop a comprehensive and fully integrated program, we have learned that while PBL lies at the heart or the core, additional or supporting elements go into developing and delivering a consistently successful program.
While the paper is structured around the Buck Institute’s “Seven Project Based Teaching Practices” and their “Seven Project-Based Essential Design Elements,” the paper will explain how the ILO, Iowa State, NACE, and Kirkpatrick serve to bolster, augment or reinforce what Buck Institute advocates.
Step 1- DESIGN course content using the International Labor Organization’s (ILO) “Regional Model Competency Standards.”[2]
Having come up through the trades, I am especially partial to research and recommendations developed by the International Labor Organization in developing their “Regional Competency Standards.” This is especially important given that most of our clients are English as Second Language (ESL) Mid-Career Path practitioners, usually between the ages of 25 and 45.
To realize the full potential that Project Based Learning offers, we first must invest the time and effort to determine what skill sets are desirable. To accomplish this requires that the key stakeholders work together to identify what COMPETENCIES are desirable for any given job or degree.
Suppose we fail to take this essential first step when developing the 7 Elements required for Project-Based Learning. In that case, we risk missing many opportunities that will serve to reinforce the elements identified during this process.
If you look at PMI’s “PMP exam Content Outline,” you can see that PMI took the assessment down to this level of detail but did NOT take it down to the last two steps.
PMI is really weak on the “Performance Criteria” (Lads and Directs: is insufficient, and PMI is totally missing the RANG STATEMENTS. (See Figures 8 and 9 from Iowa State University)
[1] By Author from Course Materials
Lastly, PMI is missing any pr review or expert assessment as w normally see in many sports such as diving, Ic dancing, and figure skating.
Step 2- DESIGN course LEVELS using the IOWA STATE Revised Blooms Taxonomy[6]
This is another essential step PRIOR to designing a Project-Based Learning program. The program must be geared to the age and experience levels of the target audience. Going back to our definition of “Competency,” we know that there are multiple competency levels. Adopting the Iowa State, Revised Blooms Taxonomy provides us with a structured approach to designing our projects appropriate to the level of experience and the types of knowledge and how that knowledge can and should b applied given the number of years and types of experience.
As our clientele are all mid-career path professionals, ages 25 to 55, when we design the projects used in our PBL programs, to be successful, we must ensure that the projects chosen are appropriate. This same concept would apply to PBL at the Elementary, middle, High school, and college levels.
Using the example of obtaining your first driver’s license when you were 16 or 17, this is typically a two-part process consisting of a WRITTEN exam to validate your ability to REMEMBER and UNDERSTAND, followed by a PRACTICUM, where you sit next to the DMV officer who puts you through a series of STANDARDIZED TESTS (i.e., 3 point turn, starting on a hill, parallel parking, etc.)
And while that same process is followed to obtain a commercial driver’s license for a Semi-Trailer rig or heavy construction equipment, the written and practicum tests become more robust, demanding, and challenging.
[1] By Author from Course Materials
[2] International Labor Organization “Regional Model Competency Standards: Construction” https://www.ilo.org/wcmsp5/groups/public/---asia/---ro-bangkok/documents/publication/wcms_370561.pdf
[3] By Author from Course Materials
[4] By Author from Course Materials
[5] By Author from Course Materials
[6] A Model of Learning Objectives–based on A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives by Rex Heer, Center for Excellence in Learning and Teaching, Iowa State University https://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy-flash-version/
[7] By Author from Course Materials
[8] By Author from Course Materials
Step 3- DESIGN course CONTENT to develop the SKILLS that employers are SEEKING and VALUE[1]
This is an area that many in academia are missing, and the core objective must be to produce employable graduates. And that means an essential element of PBL must incorporate developing skills employers deem valuable.
As this has proven to be a contentious topic in many organizations, we have relied on the National Association of Colleges and Employers’ published research. (NACE) NACE has been around since 1956, and we have learned over the past 30 years that while the attributes change in weighting and rank order, the importance of the NACE research is that it helps us identify TRENDS that we can use to keep our projects current. As you can see from the red wording, we link or map each project in our Project Based Learning Portfolio to a specific attribute.
Under the heading of Design and Plan, this is an excellent time to explain that the Level of Effort on the part of the Mentor/Facilitator for a 90-day program ranges between a “Best Case” of 15 person-hours per student to a “Worst Case” of 40 hours per student but “Most Likely” you can budget 20 hours of Mentor/Facilitator time per student. For the 180-day course, the figures will be just about double- Best Case = 30 Mentor/Facilitator Hours per Student, “Worst Case” of 80 hours, and a most likely 40 Level of Effort Hours. You need to know this when calculating the fees to charge for these courses and what to pay your Mentor/Facilitator.
Step 4- The Seven Project-Based Teaching Practices[4]
Figure 12- Mapping Our Course Deliverables to PBL’s 7 Teaching Standards
Based on 30 years of experience developing and delivering competency development courses to mid-career path professionals, most of whom speak English as a Second Language (ESL), the author and his team have found the following standards have proven essential.
Based on 30 years of experience developing and delivering competency development courses to mid-career path professionals, most of whom speak English as a Second Language (ESL), the author and his team have found the following standards have proven essential.
1.1.1 DESIGN our course content using the ILO’s “Regional Model Competency Standard” https://is.gd/jb9GTl
1.1.2 DESIGN our courses using the IOWA STATE Revised Blooms Taxonomy- https://goo.gl/HNcuv8
1.1.3 DESIGN our courses to develop the SKILLS that employers are SEEKING and VALUE- https://goo.gl/U61YxY
1.1.4 DELIVER our courses in a way that FULFILLS all 7 of the Project-Based Learning (PBL) attributes- https://goo.gl/qKfS7u
1.1.5 MEASURE the EFFECTIVENESS of our training using all 4 Levels of Kirkpatrick- https://goo.gl/8YpZEA
And we do NOT subscribe to any single organization’s STANDARD (I.E., PMI, AACE, IPMA, et al.) but have compiled our own “Best Tested and PROVEN tools, techniques, and practices. However, we have adopted AACE’s RP-11R-88 as the content quality checklist.
Step 4.2- Build the Culture
Given that in much of the developing world, where the usual practice is for the teacher lectures for hours on end and the students dutifully take copious notes capturing every word the teacher or Professor says, we’ve found that “Project-Based Learning” often comes as quite a shock. So to realize full value from the Project-Based Learning Experience, we need to invest time upfront in establishing the objectives, standards, and the “ground rule” the team will be operating under.
Given that in much of the developing world, where the usual practice is for the teacher lectures for hours on end and the students dutifully take copious notes capturing every word the teacher or Professor says, we’ve found that “Project-Based Learning” often comes as quite a shock. So to realize full value from the Project-Based Learning Experience, we need to invest time upfront in establishing the objectives, standards, and the “ground rule” the team will be operating under.
- Vision
- Mission
- Values
- Team Performance Objectives/Metrics
- Definition of Success
- Expectations of Team Members
- Administrative procedures
- Rules of engagement
- Penalties/Sanctions/Rewards
It needs to be understood that not everyone does well in these types of courses, and procedures must be made to enable people to drop out or be removed by their team for anyone who cannot or will not perform as agreed.
Over the years, we have seen a dropout rate between 10% and 30%. For anyone planning on implementing project-based learning programs, you must recognize and accept that not everyone will succeed and that you have alternative programs in place OR are very selective in who you take into these programs.
Step 4.3- Manage Activities
Under the heading of “Manage Activities,” there are two crucial parts to this requirement.
The first is that these types of programs need to be carefully designed so that there is sufficient time allowed for “reasonably” motivated people to finish whatever projects have been assigned. And the second element is that project-based learning requires an extended period. In this case study, the author has two versions of the program, one is a 90-day or 13-week program, which is consistent with a full one-semester course, and there is the 2nd program of 180 days or six months that is consistent with a two semester-long “Capstone” type course. Obviously, the number and complexity of projects are not the same for each class.
From the perspective of the Professor (we prefer the term “Facilitator” or “Mentor”), there are two schedules. The first is the “big picture” schedule dictated by the course length or program. This is provided to the students at the outset.
Here is an example of an actual “Master Plan” provided by the Professor in his/her role as “Mentor” or “Facilitator” at the outset of the program. This provides the “big picture” and the critical milestone dates the participants must meet.
We will explore this in more detail later, but consistent with the 7 Best Design Elements of “Student Voice and Choice,” HOW they execute their projects is up to the students to decide. Explained another way, while the Mentor/Facilitator defines the critical milestone dates of the program, the students are free to work within those dates to “initiate, plan, execute, control, and close” their work.
This program was initially developed for use as a Blended Learning (combined face-to-face and distance learning mode) of 90 or 180 days. For the 180 days Blended Learning model, these are the projects that have been assigned.
Since COVID, this program has been MODIFIED to be pure distance learning mode ONLY, and while initially designed for classes of 25 people by adding or deleting project assignments or by changing the specifications and requirements for each of the project deliverables, this program can be modified for anywhere between 1 and 25 people. The maximum class size using this program was done for SKEMA School of Business, consisting of 190 graduate-level students, which to manage, was broken down into five classes of about 40 people each. However, this is not a recommended class size.
As explained in the introduction, having spent most of his life from high school onwards immersed in a project-based world, as the author evolved from being a practitioner to becoming an academic, the only logical approach was to develop courses designed to build COMPETENCY in global practitioners, most of whom speak English as a Second Language and to meet or at least support as many of the 8 Millennial Development Goals[7] as possible by developing the Organizational Capacity in the developing nations.
Step 4.4- Scaffold Student Learning
One of the significant challenges in developing and delivering project-based courses is because the Mentor/Facilitator has little or no clue what those joining the program know, the mentor/facilitator needs to recognize and accept that he/she is going to have to develop a comprehensive and deep understanding of the subject matter, as the strengths and weaknesses of the people in the program will not be discernible until after the program has started and the completed deliverables start to get submitted. Only when that happens will the Mentor/Facilitator be able to assess the REAL “strengths and weaknesses” of those in the program.
This reality also requires that the Mentor/Facilitator be prepared to offer remedial or supplemental training OR be willing and able to MODIFY the Program in the event that the majority of the participants in the program are not adequately prepared to be in it. This explains why the preselection of participants in these types of programs to ensure they have the necessary background and understanding is essential.
Step 4.5- Assess Student Learning
This is the most exciting aspect of these types of programs, but it requires Mentor/Facilitators who have as much “real-life” experience as possible in the subject they are teaching. Because these courses are APPLIED, more than THEORETICAL, the deliverables being submitted by the students must be assessed using well-defined metrics, rubrics, and technical specifications, typical of those likely to be found in the industry.
While this may be easy for most STEM subjects, if you start getting students from different backgrounds or majors, it becomes very challenging to establish a set of technical specifications that will be used to mark or grade their work results. Based on 30+ years of experience, we have developed detailed specifications for each project we assign. This includes TEMPLATES containing EXAMPLES of what a finished product looks like, instructional videos or references, and precisely what grading rubrics will be applied.
If there are going to be any disputes or complaints from the students about the program, it most likely will be due to ambiguity in the requirements or disagreements over the grading rubrics/
Step 4.6- Engage and Coach
Based on the author’s 30+ years of experience in the trades and being a PADI SCUBA diving instructor, he prefers the terms “Mentor and Facilitator” over “Engage and Coach.”
Done correctly, these programs require much more than just being a coach. Very often, as a mentor/facilitator, you are very much “hands-on,” showing them what has to be done and how to do it. Therefore, it is so important to select “professors” or “teachers” to serve as “Mentor/Facilitators” with actual field experience and not just pure academics. While the theory is essential to know, when it comes to developing COMPETENCY, the individual doing the mentoring/facilitating must also be recognized as competent in his/her field. Failing to have qualified practitioners teaching others guarantees that the program will eventually fail. Think about it pragmatically. If you were learning brain surgery, wouldn’t you want someone like Ben Carson mentoring/facilitating you?
Step 4.7- Design and Plan
This aspect relates to #1- Align to Standards and #3, Manage Activities.
As noted in #1, five elements go into the “successful” design:
1.2 DESIGN our course content using the ILO’s “Regional Model Competency Standards” https://is.gd/jb9GTl
1.3 DESIGN our courses using the IOWA STATE Revised Blooms Taxonomy- https://goo.gl/HNcuv8
1.4 DESIGN our courses to develop the SKILLS that employers are SEEKING and VALUE- https://goo.gl/U61YxY
1.5 DELIVER our courses in a way that FULFILLS all 7 of the Project-Based Teaching (PBL) attributes- https://goo.gl/qKfS7u
1.6 DELIVER our courses in a way that FULFILLS all 7 of the Project-Based Learning (PBL) attributes- https://goo.gl/qKfS7u
1.7 MEASURE the EFFECTIVENESS of our training using all 4 Levels of Kirkpatrick- https://goo.gl/8YpZEA
[1] National Association of Colleges and Employers (2018) “The Key Attributes Employers Seek on Students’ Resumes” https://www.naceweb.org/about-us/press/2017/the-key-attributes-employers-seek-on-students-resumes/
[2] By Author from Course Materials
[3] Buck Institute’s Gold Standards for Project Based Learning and Teaching https://www.pblworks.org/what-is-pbl
[4] Buck Institute’s Gold Standards for Project Based Learning and Teaching https://www.pblworks.org/what-is-pbl
[5] For an Example of an actual Team Governance Agreement, see Appendix 1
[6] By Author from Course Materials
[7] UN Millennial Development Goals (2020) https://www.un.org/millenniumgoals/
Step 5- The Seven Essential Project Design Elements[1]
Previously, we looked at the 7 Project Based Teaching Practices, and now we are going to explore the 7 Essential Project Design Elements, understanding that the course is used as a case study to develop COMPETENCY and DEVELOP CAPACITY in APPLIED Asset, Portfolio, Program (Operations) and Project Management. The examples shown here must be modified to fit your application, context, or subject matter.
Step 5.1- Sustained Inquiry
As explained in #3- Manage Activities and #7- Design and Plan, these courses do NOT lend themselves very well to programs less than 90 days or one semester long. To develop COMPETENCY, you need to have a sustained inquiry of 90 to 180 days or even longer. (i.e., for many trade apprenticeships as well as in engineering, architecture, and medicine, the “apprenticeships” or “internships” are anywhere between 4 to 7 years) Do NOT try to adapt this learning program to 2, 3, or even 5-day courses. It simply is not enough time to develop, much less assess or evaluate COMPETENCY in technical skills and the “soft” or people skills.
Step 5.2- Authenticity
This is the most challenging part of designing a Project-Based Learning course, and that is how to make it AUTHENTIC- that is, how do we make it “real,” “relevant,” and “meaningful” for each of the participants when we have no idea in advance what their career path objectives are or what their interests are?
Based on the fact that we are in the business of developing COMPETENCY in “Applied Asset, Portfolio, Programs (Operations) and Project Management, we can fulfill this requirement by offering multiple LEVELS of challenges based on Years of Experience. (See Figure 2) and we also provide the choice of obtaining CERTIFICATIONS from different professional societies as a REWARD for successfully completing the program. By offering these choices, we’ve designed the program to be CUSTOMIZABLE to meet the career path objectives of different people. While the core or underlying knowledge for each of these certifications is pretty much identical, it allows for considerable flexibility enabling the participants to customize the program for themselves.
AACE Family of credentials- CCP, CEP, PSP, EVP, DRMP http://web.aacei.org/certification
Construction Management Association Certified Construction Manager (CCM) https://www.cmaanet.org/certification/ccm
Guild of Project Controls COMPETENCY-based Credentials http://www.planningplanet.com/guild/certification
International Council of Systems Engineers (INCOSE)- https://www.incose.org/systems-engineering-certification
PMI Family of credentials- PMP, PfMP, PgMP, PMI-SP, PMI-RP, and PMI-AC http://www.pmi.org/certifications
One of the “Lessons Learned” is that there needs to be a robust “reward and recognition system” in place, given that the level of effort to succeed in these courses is so high. Typically, for the 90-day program, the participants (students) can expect to invest a “Best Case” of 45 person-hours, a “Worst Case” of 150 hours, and a “Most Likely” of 100 person-hours of effort. For the 180-day course, those values double- “Best Case” of 90 hours, “Worst Case” of 300 hours, and a “Most Likely” of 200 person-hours of effort.
Step 5.3- Student Voice and Choice
This attribute is another area that can represent a challenge for the Mentor/Facilitator to develop. How much “choice” are we willing and able to give each participant? How much flexibility can we offer?
As noted under 7.4.2 Authenticity, we give the participants a rather extensive choice of certifications they prepare for as their “REWARD” for completing the course.
Given this program is to build competency in Applied Asset, Portfolio, Program (Operations), and Project Management, we also give them considerable latitude in what software tools they can choose from but most importantly, under the heading of #3 Manage Activities and #7 Design and Plan, while the Mentor/Facilitator is responsible for the MASTER PLAN containing the program milestone dates, the participants are free to plan how they intend to PRODUCE the projects they have been assigned as part of the course curriculum.
Here is a link to the current class we are running for a major Indonesian National Oil company. https://fabdap2022.wordpress.com/
Step 5.4- Reflection
Given that we know from the NACE Research (See Figure 9 above) that “Written Communications” is one of the top 3 skills employers sought when we designed the course, we relied extensively on written communications. Thus two of the “projects” require written communication. We specify a WEEKLY blog of 250 to 500 words that are GRADED each week, and then we have a “big” project requiring a 2500 to 5000-word publishable quality paper to be written. Both the blog and the article have to be written to solve a “real” problem chosen by the participant, which is another example where we try to provide both #2- AUTHENTICITY and #3- STUDENT VOICE AND CHOICE. Once again, this requires that the Mentor/Facilitator has a broad range of experience in his/her field to ensure that he/she can provide guidance to the student in writing the blog or the paper and that he/she has sufficient knowledge of the topic to be able to grade the paper or blog.
Here is an example where a Blog posting was REJECTED by the Mentor/Facilitator, and the student was required to REFLECT on what he posted and make the appropriate corrections.
Step 5.5- Critique & Revision
The heading of Critique and Revision is closely related to the previous Reflection title. An individual student or team of students produces a DELIVERABLE as required by a project. The Mentor/Facilitator provides COMMENTS or SUGGESTIONS that the student (s) are expected to think about, and if the deliverable has been REJECTED, the next step in the process is to require them to make REVISIONS to the deliverable to bring it within the acceptable specifications. Of all the responsibilities of the Mentor/Facilitator, this is perhaps one of the most important. Not only do the responses have to be timely (the policy of this author is a response within a MAXIMUM of 24 hours and more often within less than 12 hours. While this may seem high, the author is based in Indonesia and services clients worldwide, so the different time zones must be considered. In reality, the response time averages between 4 to 8 hours.
Below are the MILESTONES for 2500 to 5000-word paper, and you can see clearly how the attributes of REFLECTION and CRITIQUE & REVISION have been designed into the process.
This process provides ample opportunity for Critique, Reflection, and Revision. This process also consumes the vast majority of the Mentor/Facilitators time with each student. This also explains why Mentor/Facilitators must be competent PRACTITIONERS and not pure academics, as the topics selected by each student tend to be very broad and diverse.
Step 5.6- Public Product
To meet or fulfill the “Public Product” requirement, the Weekly Blog Posting and the paper are designed to be published to allow the public to provide comments.
Below are some examples of the blogs being posted currently by our students from around the world as this paper is being written:
Team FAB n DAP, PRTAMINA Oil, Indonesia
https://fabdap2022.wordpress.com/
Danilo Arba, Milan Italy and Lima Peru- https://www.daniloarba.com/my-blog/
Mahar Arif, Pertamina Oil, Indonesia- https://arguniaace2020.wordpress.com/
Ilham Zazzali, PT. PATRA BADAK ARUN SOLUSI, Indonesia, https://ilhamzazzali.wordpress.com/
Team Octopus, Pertamina Oil, Jakarta, Indonesia https://2019octopus.wordpress.com/
Team Vaccine 2020, Pertamina University, Jakarta, Indonesia https://2020vaccineaace.wordpress.com/2020/
And here are some examples of the 2500 Word Papers published by graduates of this program:
Steve Patterson, Exxon-Mobil Singapore- https://pmworldlibrary.net/authors/stephen-j-c-paterson/
Piero Anticona, Deloitte, Lima Peru- https://pmworldlibrary.net/authors/piero-g-anticona/
Tijo Kurian, India and Libya- https://pmworldlibrary.net/authors/tijo-kurian/
Asma Al Fahda, Oman Oil- https://pmworldlibrary.net/wp-content/uploads/2014/11/pmwj28-nov2014-AlFadha-multidimensional-wbs-Featured-Paper2.pdf
Lita Liana, Shell Oil, Indonesia- https://pmworldlibrary.net/wp-content/uploads/2014/02/pmwj19-feb2014-liana-analytical-hierarchy-for-roi-oilandgas-projects-indonesia-FeaturedPaper.pdf
Yasmine Taybi, Morocco and France, SKEMA School of Business, https://pmworldjournal.com/article/is-earned-value-management-evm-consistent-with-sharia-law
Consistent with #2 Authenticity and #3 Student Voice and Choice, the individual student has selected these topics to solve a real problem facing them in their working environment and designed to help them.
Step 5.7- Challenging Problem or Question
As evidenced by the examples of the blog postings and selected papers shown under #6 Public Product and consistent with #2 Authenticity and #4 Student Voice and Choice, you can see by the diversity of the topics chosen that the students determine these topics to achieve or realize two primary objectives:
1) The topics are chosen that will help them either find a new job or get a promotion in their existing jobs by solving REAL PROBLEMS they are facing in their day-to-day job responsibilities or 2) fulfill or meet the requirements to earn a globally recognized professional certification or to fulfill degree requirements.
Explained another way, these are not theoretical case studies but practical, pragmatic solutions to real problems identified by the students that will enhance their professional image. The top of the Key Performance Indicators bing to
1) Maintain an SPI and CPI between 0.95 and 1.05 for 26 wks.
2) Pass their chosen AACE, Guild of Project Controls, INCOSE, CMAA, or PMI Certification on the first attempt
3) Generate a favorable Return on Training Investment (> 1.00)
“Return on Training Investment, we try to track and measure whether each student can document savings in excess of the cost of the class. Our best performing PMO case study[5] is Freeport Indonesia[6], which invested $300,000 to train ~50 people, documenting savings of 65 million USD over 4 years, generating an RoI of $65,000,000 / $300,000 = 217%.[7]
[1] Buck Institute for Education 7 Essential Project Design Elements (n.d.) https://www.pblworks.org/what-is-pbl
[2] By Author from Course Materials
[3] By Author showing typical Blog Posting Response for a REJECTED blog posting
[4] By Author from Course Materials
[5] PTMC Team 2014- https://build-project-management-competency.com/1-4-1-11-unit-11/
[6] Ridwan Wibiksana, PMP, CCE https://pmworldlibrary.net/wp-content/uploads/2013/01/PMWJ2-Sep2012-WIBIKSANA-EVM-Adapted-for-UndergroundMining-StudentPaper.pdf
[7] PTMC Tam- https://build-project-management-competency.com/ptmc-training-standards-and-specifications-individual/
6. MEASURE the EFFECTIVENESS of our training using all 4 Levels of Kirkpatrick[1]
Given that most course evaluations consist of “Smiley Face” reviews, the author and his team soon learned that using the traditional “Smiley Face” reviews was inadequate and inappropriate for evaluating Project Based Learning.
To remedy this, we researched and found that back in 1954, Dr. Don Kirkpatrick developed a 4 level assessment process based on the four steps he learned from Dr. Raymond Katzell to evaluate industrial supervisors’ training.[2]
Because the author and his team have taken a holistic systems approach, this 4 Level Kirkpatrick Model has proven to be the basis for measuring and assessing what Project Based Learning can deliver. This assessment is done at the COMPLETION of the 90-day or 180-day version of this program. Why? Because the participants can’t be able to assess how effectively the program was at addressing all four levels of Kirkpatrick until AFTER the Program has been completed.
6.1 Level 1- Measuring Reaction
This is done primarily through customer surveys sent to the participants with the request that they send the completed forms to the following MINIMUM people:
6.1.1) The person who SPONSORED them to take their training. This is usually their functional or line boss who put their name forward or otherwise approved them to take time off for the face-to-face portion of the program (if applicable)
6.1.2) Their H.R. or Learning Development Specialist is responsible for recording or documenting that they have taken and completed the training program.
6.1.3) The immediate boss or supervisor of the Mentor/Facilitator.
6.1.4) anyone else in either the client or the provider’s organization who is responsible for evaluating the effectiveness and efficacy of the training as well as the effectiveness and efficacy of the Mentor/Facilitator.
6.2 Level 2- Measuring Learning
This element of the Kirkpatrick scoring model is customizable based on the Cognitive Dimension (Blooms) and Knowledge Dimensions developed by the Iowa State Center of Excellence in Learning and Teaching, as shown in Figure 6. This provides real-time feedback to ensure that the course as designed is appropriate to the experience levels and job responsibilities of those attending.
6.3 Level 3- Measuring Behavior
Given the reality that everyone hates to be involved in team projects because of the “free rider” problem where some minority on the team do all or most of the work while some members (often the majority) do little or no work, the author and his team have developed a “360-degree” evaluation instrument where the team members can grade one another based on the top 20 attributes coming from the NACE research. The weight given to this score is not enough to fail any student but is sufficient so that if his/her team members all score him/her low that it will drop their grade from an “A” or 100% down to a grade of “D” or 65%. In the 15 or so years this has been in place, it has resulted in only one team out of more than 200 who have complained about the score given by their colleagues or teammates.
IDEALLY, the Training Provider should meet with the Sponsor and H.R. or the Learning Development Center 3 or 6 months AFTER the Program is done to see what changes in behaviors have been observed and institutionalized. Over the years, we have learned that unless what has been taught is reinforced by the organization on a day-to-day basis, the behaviors will quickly revert to “business as usual.”
6.4 Level 4- Measuring Results
There are two levels of “results” that we try to capture. The first level is easy to measure, and that is by looking at the blog postings and the papers to see if the sophistication of what they learned did or did not improve over the six months. Another way to measure results is to validate that they passed any or all certification exams identified in the team governance agreements.
The other objective which is harder to measure but is essential to the organization providing the funding is whether the investment in training resulted in savings equal to or preferably in excess of the program’s costs. This is known as “Return on Training Investment” or “RoTI,” and while not always quantifiable, we do have one case study from Freeport Indonesia where we trained 25 of their Underground Mining team and the changes they implemented, which are described in this paper, documented a savings of 65 million USD over four years. Ridwan Wibiksana (2012) “Earned Value Management: Adapted for use in Underground Mining Operations” https://pmworldlibrary.net/wp-content/uploads/2013/01/PMWJ2-Sep2012-WIBIKSANA-EVM-Adapted-for-UndergroundMining-StudentPaper.pdf
Step 7- Conclusions and “Lessons Learned”
Project-Based learning is, without question, one of the most potent and practical learning approaches, especially in today’s “VUCA” environment (volatility, uncertainty, complexity, and ambiguity,) HOWEVER unless treated as a holistic SYSTEM starting with defining what competencies you need or want to develop and then using that as the basis to determine what projects will best deliver the competencies you need or want.
Lessons Learned #1-
Begin with the end in mind. Do you want to develop a private pilot who can fly a Cessna 172 or a Commercial pilot qualified to fly a 747 or Airbus 380? Take the time and effort to design your PROJECTS to ensure they develop the APTITUDE + ATTITUDE + SKILLS + STRENGTH + KNOWLEDGE = COMPETENCIES Necessary for the job. BEWARE the laws of unintended consequences!!
Lessons Learned #2-
Don’t ignore the SOFT skills. Yes, technical skills are essential, but if a person cannot communicate effectively in writing and speaking or if they cannot function in a team environment, they are probably unlikely to succeed in most positions today. Contrary to PMI’s unsupported claims, Solving Problems is #1, followed by being able to work on a team. Written communications are #3.
Lessons Learned #3-
Budget more time and MONEY for your Mentor/Facilitator. These courses take 2–3 times the level of effort per student that a “traditional” class takes. Not only that but providing the level of support to keep the momentum going means the Mentor/Facilitator needs to be available or “on-call” 15 hours per day, seven days per week. This is NOT a 9–5 job.
Lessons Learned #4-
Pick Mentor/Facilitators with as much “real-life” experience as possible. These courses are very much APPLIED, not THEORETICAL, and if you don’t have Mentor/Facilitators with broad exposure to the working world, they will not be able to support the various problems that the students select.
Lessons Learned #5-
Ensure that your students are involved in the grading process for team-based assignments to avoid the “Free Rider” problem. This may not seem important, but if you are going to get complaints about project-based learning, it revolves around the fact that people HATE to do all the work and see people who did nothing getting an A for something they didn’t contribute to.
Lessons Learned #6-
Mentor/Facilitators need to be prepared to write Letters of Recommendation (LoR) or provide support to their students after the course is over to find jobs or apply for advanced degrees. PBL courses tend to produce more personal relationships between the student and the Mentor/Facilitator.
Lessons Learned #7-
Accept that not everyone is going to succeed. Like the Navy Seals or the Green Berets, some people will “wash out” of the program. Consistent with most grad school courses, you can expect between 10% to 33% not to make it to the end.
Lessons Learned #8-
Expect CONFLICT to occur within the teams. The Team Governance Agreement needs a conflict resolution PROCESS, and the school or organization needs to provide conflict resolution counseling and training as part of the course.
[1] Kirkpatrick Partners (n.d.) The New World Kirkpatrick Model https://www.kirkpatrickpartners.com/Our-Philosophy
[2] Kirkpatrick Partners (n.d.) The New World Kirkpatrick Model https://www.kirkpatrickpartners.com/Our-Philosophy
[3] By Author from Course Materials See Appendix 2 for more details
[4] By Author from Course Materials See Appendix 2 for more details
[5] By Author from Course Materials See Appendix 2 for more details
[6] By Author from Course Materials See Appendix 2 for more details
[7] By Author from Course Materials See Appendix 2 for more details
To see examples actual TEMPLATES referenced here visit this site:
https://pmworldlibrary.net/wp-content/uploads/2022/08/pmwj120-Aug2022-Giammalvo-30years-of-developing-and-assessing-competent-project-managers.pdf Scroll down to the end to the Appendices.
