-
Bahasa Indonesia
-
English
Oleh: Vincent Gaspersz,
Lean Six Sigma Master Black Belt & Certified Management System Lead Specialist
- American Production and Inventory Control Society (www.apics.org) Certified in Production and Inventory Management (CPIM), Certified Fellow in Production and Inventory Management (CFPIM) and Certified Supply Chain Professional (CSCP);
- American Society for Quality (www.asq.org) Certified Manager of Quality/Organizational Excellence (CMQ/OE), Certified Quality Engineer (CQE), Certified Quality Auditor (CQA), Certified Quality Improvement Associate (CQIA), and Certified Six Sigma Black Belt;
- International Quality Federation (www.iqf.org) Certified Six Sigma Master Black Belt (CSSMBB);
- Registration Accreditation Board (www.exemplarglobal.org) Certified Management System Auditor (CMSA), Certified Management System Practitioneer (CMSP), Certified Management System Specialist (CMSS), and Certified Management System Lead Specialist (CMSLS).
Paradigma baru dalam manajemen sumber daya manusia (Phillips and Phillips, 2015) adalah:
- Program-program atau solusi yang ditawarkan HARUS terkait dengan kebutuhan nyata dari bisnis dan industri.
- Penilaian berbasis efektivitas kinerja.
- Tujuan-tujuan spesifik untuk aplikasi dan harus berdampak signifikan pada bisnis dan industri.
- Ekspektasi hasil-hasil dikomunikasikan kepada seluruh karyawan (partisipan).
- Lingkungan dalam perusahaan disiapkan untuk mendukung implementasi sistem-sistem manajemen.
- Kemitraan (partnership) ditetapkan dengan manajer-manajer inti dan pelanggan.
- Pengukuran hasil-hasil kinerja dan analisis biaya/manfaat (cost/benefit analysis).
- Perencanaan dan pelaporan berfokus pada outcome.
Selanjutnya, jika kita membaca tentang penawaran program pelatihan eksekutif oleh Harvard Business School (HBS) berikut, yang berbiaya sangat mahal, maka kita akan bertanya apakah manfaatnya lebih besar dari biaya yang sangat mahal itu? Jika dinilai dalam rupiah dengan kurs US$ 1 = Rp. 13.350 berarti biaya pelatihan eksekutif AMP (Advanced Management Program) dari HBS itu sekitar Rp. 1.068.000.000 ditambah biaya pesawat dari Indonesia – USA – Indonesia dan ditambah biaya waktu yang hilang selama pelatihan (opportunity cost), maka total biaya bisa mencapai lebih dari Rp. 1,5 M atau mungkin mendekati Rp. 2M.
Advanced Management Program
Transforming Proven Leaders into Global Executives
10 SEP–26 OCT 2017 | $80,000
The program fee covers tuition, books, case materials, accommodations, and most meals.
Location HBS Campus
02 APR–17 MAY 2018 | $80,000
The program fee covers tuition, books, case materials, accommodations, and most meals.
Location HBS Campus
Program Contact
Ms. Kathryn Venne
Portfolio Director
Email: kvenne@hbs.edu
Telephone: 1-800-427-5577
(outside the U.S., +1-617-495-6555)
https://www.exed.hbs.edu/programs/amp/Pages/default.aspx
Bagaimana kita menilai apakah suatu training itu bermanfaat? Seorang profesional yang berpikiran maju tentu saja HARUS melihat dalam konteks Cost and Benefit, agar bisa secara adil menilai efektivitas (pencapaian tujuan) dari training dan efisiensi (pengeluaran biaya) untuk mencapai tujuan pelatihan itu.
Pada saat kita akan mengukur Cost and Benefit dari suatu pelatihan, maka ukuran level 5 yang dikembangkan oleh Jack Phillips pada tahun 1970 (Phillips, 2003) yang disebut Return On Training Investment (ROTI) seperti Bagan 1 terlampir di atas harus dilakukan.
Agar memudahkan penjelasan lebih lanjut, maka perlu dikemukakan bahwa terdapat lima tingkat pengukuran dalam menilai efektivitas pelatihan, yaitu: Level 1 – 4 yang dikembangkan oleh Donald Kirkpatrick pada tahun 1959 (Kirkpatrick and Kirkpatrick, 2005).
- Level 1: Reaction.
- Apakah partisipan (peserta pelatihan) menyukai training yang diterima oleh mereka?
- Level 2: Learning
- Apakah partisipan (peserta pelatihan) meyakini bahwa mereka telah belajar sesuatu dari program pelatihan yang diterima oleh mereka?
- Level 3: Behavior/Application
- Apakah partisipan (peserta pelatihan) mampu menerapkan ditempat kerja tentang apa yang mereka telah pelajari dalam program pelatihan?
- Level 4: Results/Business Impact
- Apakah pelatihan menunjukkan peningkatan dalam efisiensi, produktivitas, keuntungan, penurunan biaya-biaya, dan penurunan tingkat keluar-masuk karyawan?
- Level 5: ROI (Return On Investment)
- Apakah program pelatihan menunjukkan Retun On Investment (ROI) yang positif?
Tahap-tahap yang perlu dilakukan dalam menganalisis ROTI (Return On Training Investment) adalah:
- Perencanaan Evaluasi (Evaluation Planning)
- Pengumpulan Data (Data Collection)
- Analisis Data (Data Analysis)
- Pelaporan (Reporting)
Tahap 1: Perencanaan Evaluasi (Evaluation Planning)
Dalam tahap perencanaan evaluasi (Tahap 1) terdapat dua langkah utama berikut.
Langkah pertama adalah mengembangkan tujuan program pelatihan melalui menjawab beberapa pertanyaan berikut:
- Apa tujuan dari pelatihan?
Apa masalah yang sedang ingin diselesaikan? Misal: (1) Meningkatkan penjualan untuk tujuan pelatihan penjualan (sales training), (2) Menurunkan cacat produksi untuk tujuan pelatihan manajemen kualitas (quality management training), dll. Prinsip tujuan pelatihan adalah menjamin program-program pelatihan dilakukan tepat waktu, mengembangkan proses-proses yang lebih efisien, dan menjamin bahwa program-program pelatihan itu dilakukan dalam kendala anggaran yang terbatas.
Langkah kedua, mulai mengembangkan rencana evaluasi, menentukan data baseline, mengembangkan rencana pengumpulan data, rencana analisis dan evaluasi ROI (Return On Investment) dari program pelatihan itu.
Tahap 2: Pengumpulan Data (Data Collection)
Selama Tahap 2, dua jenis data akan dikumpulkan. Data jenis pertama adalah segera setelah program pelatihan selesai dilakukan, dan data jenis kedua adalah selama periode waktu tertentu (4 bulan, 6 bulan, atau 12 bulan) di mana individu peserta pelatihan telah menerapkan keterampilan mereka yang berdampak pada hasil-hasil bisnis. Data yang dikumpulkan dapat berupa data level 1 sampai level 4 dalam pengukuran efektivitas pelatihan, di mana level 1 dan level 2 dapat segera dikumpulkan setelah pelatihan dilakukan, sedangkan level 3 dan level 4 dapat dikumpulkan pada selang waktu tertentu (empat bulan, enam bulan, atau 12 bulan) selama masa aplikasi program pelatihan itu.
Tahap 3: Analisis Data (Data Analysis)
Selama Tahap 3, data yang dikumpulkan dari Tahap 2 diproses dan dianalisis tentang dampak positif dari program pelatihan yang dilakukan, menggunakan metode yang telah ditentukan dalam Tahap 1 di atas.
Selama Tahap 3, data dikonversikan ke dalam nilai moneter (uang) menggunakan rencana analisis ROI (Return On Investment) yang telah ditentukan dalam Tahap 1 di atas. Beberapa data yang bersifat “intangible or soft” seperti dampak dari program pelatihan akan menurunkan stress, meningkatkan kepuasan kerja, dll dapat dikonversikan ke dalam nilai moneter (uang) sebagai manfaat “intangible” atau “soft benefit”, jika memungkinkan. Apabila tidak memungkinkan untuk mengkonversikan data “soft benefit” ke dalam nilai moneter (uang), maka cukup disebutkan sebagai “soft benefit” dari program pelatihan itu.
Perhitungan ROI (Return On Investment) dari program pelatihan atau sering juga disebut sebagai ROTI (Return On Training Investment) dihitung menggunakan formula:
Training ROI or ROTI = (Net Training Program Benefits / Training Program Costs) x 100%
Komponen biaya investasi pelatihan yang perlu dipertimbangkan untuk mengukur Training ROI or ROTI adalah:
- Biaya mendesain dan mengembangkan program pelatihan.
- Biaya-biaya yang terkait dengan semua material pelatihan yang diberikan kepada peserta pelatihan.
- Biaya untuk fasilitator atau instruktur.
- Biaya-biaya yang terkait dengan fasilitas pelatihan.
- Biaya-biaya perjalanan, penginapan, makan-minum, dan lain-lain.
- Gaji ditambah berbagai manfaat yang diterima oleh karyawan yang mengikuti pelatihan.
- Biaya-biaya administrasi dan overhead dari Departemen Pelatihan, yang perlu dialokasikan dengan cara yang tepat.
- Biaya-biaya lain-lain.
Sebagai contoh sederhana, misalkan program pelatihan (setelah dimurnikan dari faktor-faktor lain yang berpengaruh) memiliki Benefits sebesar Rp. 350.000.000. Jika diasumsikan bahwa Training Program Costs adalah sebesar Rp. 125.000.000, maka Net Training Program Benefits = Rp. 350.000.000 – Rp. 125.000.000 = Rp. 225.000.000.
Sehingga,
Training ROI or ROTI = (Rp. 225.000.000 / Rp. 125.000.000) x 100% = 180%.
Training ROI or ROTI = 180%; dapat diinterpretasikan bahwa setiap Rp. 1 yang diinvestasikan dalam program pelatihan akan memberikan manfaat bersih (net benefits) sebesar Rp. 1.80.
Tahap 4: Pelaporan (Reporting)
Selama Tahap 4, hasil-hasil dari Training ROI or ROTI akan dilaporkan kepada manajemen organisasi yang berkepentingan. Laporan harus terperinci, mulai dari Tahap 1 sampai Tahap 4.
Contoh Sederhana Perhitungan Training ROI or ROTI (Return On Training Investment)
Misalkan sebuah perusahaan ingin mengurangi kehilangan produksi dari 20 orang tenaga produksi mereka. Perusahaan kemudian mengirim 20 tenaga produksi itu untuk mengikuti pelatihan dalam perusahaan berbasis keterampilan produksi (in-house production skills-based training) selama lima hari.
Beberapa informasi finansial telah diperhitungkan sebagai berikut:
- Perusahaan menggunakan instruktur internal yang pada saat ini bergaji (termasuk manfaat-manfaat lain) sebesar $52,000 per tahun. Sehingga biaya instruktur yang memberikan pelatihan selama 5 hari diperhitungkan sebesar $1,000 (= $52,000 / 52 minggu).
- Biaya administrasi, ruangan, makan-minum, material, dan fasilitas pelatihan lainnya diperhitungkan sebesar $100 per orang, sehingga biaya untuk 20 orang adalah: $100 x 20 orang = $2,000.
- Setiap pekerja produksi menerima upah sebesar $800 per minggu, sehingga biaya ketika 20 orang ini mengikuti pelatihan selama 5 hari diperhitungkan sebesar: $800 x 20 orang = $16,000.
- Biaya kehilangan kesempatan untuk menciptakan keuntungan (opportunity cost) sebagai akibat mengikuti pelatihan 5 hari harus diperhitungkan. Misalkan asumsi bahwa pekerja produksi selama ini memberikan keuntungan sekitar 10% dari hasil kerja mereka. Dengan demikian opportunity cost dari program pelatihan 5 hari adalah: 10% x $16,000 = $1,600.
Catatan: nilai $16,000 adalah total upah dari 20 orang pekerja produksi.
- Biaya total untuk pelatihan selama 5 hari diperhitungkan sebesar $20,600 (= $1,000 + $2,000 + $16,000 + $1,600).
- Asumsikan bahwa ke-20 orang tenaga produksi itu setelah memperoleh pelatihan berbasis keterampilan produksi akan mampu meningkatkan produksi sebesar 10% dan mengurangi cacat sebesar 20%. Dengan demikian diperhitungkan bahwa penghematan yang diterima selama setahun adalah $80,000.
Berdasarkan informasi sederhana di atas, maka secara kasar dapat dihitung nilai ROTI sebesar:
ROTI = [(Benefit – Cost) / Cost] x 100%
= [Net Benefits / Costs] x 100%
= [($80,000 – $20,600)/$20,600] x 100%
= 288%.
Kesimpulan: setiap $1 yang diinvestasikan dalam program pelatihan akan memberikan manfaat bersih (net benefits) sebesar $2.88.
Contoh lain perhitungan Return On Training Investment (ROTI) ditunjukkan dalam Tabel 1 terlampir.
Dari Tabel 1 dapat dihitung:
ROTI = [(Benefit – Cost) / Cost] x 100%
= [($1,077,321 – $554,165) / $554,165] x 100%
= ($523,156 / $554,165) x 100%
= 94%.
Payback Period = Biaya / Manfaat bulanan
= ($554,165) / ($1,077,321 / 12)
= $554,165 / $89,776.75 = 6.2 bulan.
Salam SUCCESS.
Referensi:
- Jack J. Phillips, 2003. Return on Investment in Training and Performance Improvement Programs, 2nd ed., Butterworth Heinemann, New York, 398 pages.
- Jack J. Phillips and Patricia Pullim Phillips, 2015. High-Impact Human Capital Strategy: Addressing the 12 Major Challenges Today’s Organizations Face. American Management Association, Amacom, New York, 316 pages.
- Jussi Ilari Kantola, Tibor Barath, Salman Nazir, Terence Andre, 2017., Advances in Human Factors, Business Management, Training and Education., Springer International Publishing Switzerland, 1.227 pages.
- Kirkpatrick, D.L., and Kirkpatrick, J.D. 2005. Transferring Learning to Behavior. Berrett-Koehler Publishers, California., 220 pages.
- Mulder, M., 2017. Competence-based Vocational and Professional Education: Bridging the Worlds of Work and Education, Springer International Publishing Switzerland, 1.144 pages.
- Preston, J. 2017, Competence Based Education and training (CBET) and the End of Human Learning—The Existential Threat of Competency, Palgrave Macmillan, London, 123 pages.
- Westcott-Abudi, Gina. Using Return on Investment to Evaluate Project Management Training.
http://www.ginaabudi.com/articles/using-return-on-investment-to-evaluate-projectmanagement-training/ - http://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
The Use of ROI (Return On Investment) or ROTI (Return On Training Investment) to Evaluate the Effectiveness and Efficiency of Training Program
By: Vincent Gaspersz,
Lean Six Sigma Master Black Belt & Certified Management System Lead Specialist
- American Production and Inventory Control Society (www.apics.org) Certified in Production and Inventory Management (CPIM), Certified Fellow in Production and Inventory Management (CFPIM) and Certified Supply Chain Professional (CSCP);
- American Society for Quality (www.asq.org) Certified Manager of Quality/Organizational Excellence (CMQ/OE), Certified Quality Engineer (CQE), Certified Quality Auditor (CQA), Certified Quality Improvement Associate (CQIA), and Certified Six Sigma Black Belt;
- International Quality Federation (www.iqf.org) Certified Six Sigma Master Black Belt (CSSMBB);
- Registration Accreditation Board (www.exemplarglobal.org) Certified Management System Auditor (CMSA), Certified Management System Practitioneer (CMSP), Certified Management System Specialist (CMSS), and Certified Management System Lead Specialist (CMSLS).
The new paradigm in human resource management (Phillips and Phillips, 2015) includes:
- The programs or solutions offered MUST be related to the real needs of the business and industry.
- Assessment is based on performance effectiveness.
- Specific objectives for the application and they should have a significant impact on business and industry.
- Expected results are communicated to all employees (participants).
- The company’s internal environment is prepared to support the implementation of management systems.
- Partnership is established with core managers and customers.
- Measurements of performance results and cost/benefit analysis.
- Planning and reporting focus on the outcomes.
Furthermore, if we read about the offer of the executive training program by Harvard Business School (HBS) below, which is very expensive, then we will ask whether the benefit is greater than that very expensive cost? If valued in rupiah at the exchange rate of US$1 = Rp.13,350, it means that the cost of that AMP (Advanced Management Program) executive training from HBS would be around Rp. 1,068,000,000 plus the cost of airfare from Indonesia-USA-Indonesia and plus the cost of time lost during the training (opportunity cost); then the total costs could reach more than Rp.1.5 billions or perhaps close to Rp.2 billions.
Advanced Management Program
Transforming Proven Leaders into Global Executives
10 SEP–26 OCT 2017 | $80,000
The program fee covers tuition, books, case materials, accommodations, and most meals.
Location HBS Campus
02 APR–17 MAY 2018 | $80,000
The program fee covers tuition, books, case materials, accommodations, and most meals.
Location HBS Campus
Program Contact
Ms. Kathryn Venne
Portfolio Director
Email: kvenne@hbs.edu
Telephone: 1-800-427-5577
(outside the U.S., +1-617-495-6555)
https://www.exed.hbs.edu/programs/amp/Pages/default.aspx
How do we assess whether a training is useful? A forward-thinking professional of course MUST look in the context of Cost and Benefit, in order to fairly assess the effectiveness (goal achievement) of the training and the efficiency (expenditures) to achieve that training objective.
When we are going to measure Cost and Benefit from a training, the level 5 measurement developed by Jack Phillips in 1970 (Phillips, 2003) that is called Return On Training Investment (ROTI), like on the attached Chart 1 above, should be done.
In order to facilitate further explanation, it should be pointed out that there are five levels of measurement in assessing the effectiveness of the training, namely: Level 1 – 4 that was developed by Donald Kirkpatrick in 1959 (Kirkpatrick and Kirkpatrick, 2005).
- Level 1: Reaction.
- Do the participants (trainees) like the training received?
- Level 2: Learning
- Do the participants (participants) believe that they have learned something from the training program received?
- Level 3: Behavior/Application
- Are the participants (trainees) able to apply at workplace about what they have learned in the training program?
- Level 4: Results/Business Impact
- Does the training show improvement in efficiency, productivity, profits, decreased costs, and decreased employee turnover?
- Level 5: ROI (Return On Investment)
- Does the training program show positive Retun On Investment (ROI)?
Steps that need to be done in analyzing ROTI (Return On Training Investment) are:
- Evaluation Planning
- Data Collection
- Data Analysis
- Reporting
Step 1: Evaluation Planning
In the planning evaluation stage (Step 1), there are two following main steps.
The first step is to develop the objective of the training program by answering the following question:
- What is the purpose of the training?
What is the problem you are trying to solve? For example: (1)Increasing sales for the sales training purpose, (2)Reducing production defects for the quality management training purpose, etc. The principle of training objective is to ensure that training programs are conducted on time, to develop more efficient processes, and to ensure that those training programs are conducted within the limited budget constraints.
The second step, start developing an evaluation plan, determining baseline data, and developing data collection plan, plan analysis and evaluation of ROI (Return On Investment) of the training program.
Step 2: Data Collection
During Step 2, two types of data will be collected. The first type of data is collected as soon as the training program is completed, and the second type of data is collected over a period of time (4 months, 6 months, or 12 months) in which individual trainees have applied their skills that impact the business results. The data collected can be level 1 to level 4 data in the measurement of training effectiveness, where level 1 and level 2 data can be collected immediately after the training, while level 3 and level 4 data can be collected at certain intervals (four months, six months, or 12 months) during the life of that training program’s application.
Step 3: Data Analysis
During Step 3, data collected Step 2 are processed and analyzed on the positive impact of the training program, using the methods specified in the Step 1 above.
During Step 3, data are converted into monetary value (money) using the ROI (Return On Investment) analysis plan that has been specified in Step 1 above. Some “intangible or soft” data, such as the impact of training programs will reduce stress, increase job satisfaction, etc., can be converted into monetary value (money) as “intangibles or soft” benefits, if possible. If it is not possible to convert “soft benefit” data into monetary value (money), then simply mentioned it as “soft benefit” of that training program.
The calculation of ROI (Return On Investment) of the training program, or often also referred to as ROTI (Return On Training Investment), is calculated using the formula:
Training ROI or ROTI = (Net Training Program Benefits / Training Program Costs) x 100%
The components of the training’s investment costs that need to be considered to measure Training ROI or ROTI are:
- Costs of designing and developing training program.
- Costs associated with all training materials provided to trainees.
- Costs for facilitator or instructor.
- Costs associated with the training facility.
- Costs of travelling, lodging, meals, etc.
- Salaries plus various benefits received by employees who attended the training.
- Administrative and overhead costs of the Training Department, which need to be appropriately allocated.
- Other costs.
As a simple example, suppose the training program (after being refined from other influential factors) has Benefits of Rp.350,000,000. If it is assumed that Training Program Costs are Rp.125,000,000, then Net Training Program Benefits = Rp.350,000,000 – Rp.125,000,000 = Rp.225,000,000.
Thus,
Training ROI or ROTI = (Rp.225,000,000/Rp.125,000,000) x 100% = 180%.
Training ROI or ROTI = 180%; can be interpreted to that for every Rp. 1 invested in the training program, it will provide net benefits of Rp. 1.80.
Step 4: Reporting
During Step 4, the results of the Training ROI or ROTI will be reported to the management of the organization concerned. The report should be detailed, starting from Step 1 to Step 4.
Simple Example of Training ROI or ROTI (Return On Training Investment) Calculation
Suppose a company wants to reduce the production losses from 20 of their production personnel. The company then sends those 20 production personnel to participate in a five-day in-house production skills-based training.
Some financial information have been calculated as follows:
- The company uses an internal instructor who has current salary (including other benefits) of $52,000 per year. Therefore, the instructor’s training fee for 5 days is calculated to be $ 1,000 (= $52,000/52 weeks).
- The costs for administration, room, meals-drinks, materials and other training facilities are calculated at $100 per person; so that the costs for 20 people are: $100 x 20 persons = $2,000.
- Each production worker receives a salary of $800 per week, so the costs when these 20 people are trained for 5 days are calculated as: $800 x 20 people = $16,000.
- The cost of losing the opportunity to create profits (opportunity cost) as a result of following the 5-day training should be taken into account. Suppose the assumption is that the production workers have so far created profits of about 10% of their work. Thus, the opportunity costs of the 5 day training program are: 10% x $16,000 = $1,600.
Note: $16,000 is the total wages of 20 production workers.
- The total training costs for 5 days are calculated to be $20,600 (= $1,000 + $2,000 + $16,000 + $1,600).
- Assume that those 20 production personnel, after receiving production skills-based training, will be able to increase production by 10% and to reduce defects by 20%. Thus, it is calculated that the savings received during the year to be $80,000.
Based on the above simple information, then the value of ROTI can be roughly calculated to be:
ROTI = [(Benefit – Cost) / Cost] x 100%
= [Net Benefits / Costs] x 100%
= [($80,000 – $20,600)/$20,600] x 100%
= 288%.
Conclusion: for every $1 invested in the training program will provide net benefit of $2.88.
Another example of Return On Training Investment (ROTI) calculation is shown in the attached Table 1 below.
From Table 1, it can be calculated that:
ROTI = [(Benefit – Cost) / Cost] x 100%
= [($1,077,321 – $554,165) / $554,165] x 100%
= ($523,156/$554,165) x 100%
= 94%.
Payback Period = Costs/Monthly Benefits
= ($554,165) / ($1,077,321 / 12)
= $554,165 / $89,776.75 = 6.2 months.
Best Regards for SUCCESS.
References:
- Jack J. Phillips, 2003. Return on Investment in Training and Performance Improvement Programs, 2nd ed., Butterworth Heinemann, New York, 398 pages.
- Jack J. Phillips and Patricia Pullim Phillips, 2015. High-Impact Human Capital Strategy: Addressing the 12 Major Challenges Today’s Organizations Face. American Management Association, Amacom, New York, 316 pages.
- Jussi Ilari Kantola, Tibor Barath, Salman Nazir, Terence Andre, 2017., Advances in Human Factors, Business Management, Training and Education., Springer International Publishing Switzerland, 1.227 pages.
- Kirkpatrick, D.L., and Kirkpatrick, J.D. 2005. Transferring Learning to Behavior. Berrett-Koehler Publishers, California., 220 pages.
- Mulder, M., 2017. Competence-based Vocational and Professional Education: Bridging the Worlds of Work and Education, Springer International Publishing Switzerland, 1.144 pages.
- Preston, J. 2017, Competence Based Education and training (CBET) and the End of Human Learning—The Existential Threat of Competency, Palgrave Macmillan, London, 123 pages.
- Westcott-Abudi, Gina. Using Return on Investment to Evaluate Project Management Training.
http://www.ginaabudi.com/articles/using-return-on-investment-to-evaluate-projectmanagement-training/ - http://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
