1 This definition covers the process of using the Work Based Learning (WBL) Success Rates Master File to produce the new measures success rates that will appear in Provider Reports.
Purpose
2 Learner outcome data used by the LSC as part of Performance Review and the data used by the Adult Learning Inspectorate (ALI) during inspections focuses on success rates extracted from the individualised student record (ISR) and individualised learner record (ILR).
3 The new measures of success for WBL are being introduced in order to provide a more consistent, reliable way of measuring success in WBL provision. They are also more closely aligned with the (Further Education) FE success rate methodology.
4 The new measures for WBL consist of two separate methods for calculating success rate:
The overall success rate counts all achievements and is therefore most similar to the current measure.
The timely success rate only counts achievements that occur either before or no later than a month from their planned end date.
5 We anticipate that as providers become more accurate in their predictions of course durations the two measures will converge to some extent as the timely measure is lower than the overall measure in the majority of cases.
Relevant Collections
6 The method is run on data collected in the most recent freeze for each year, allowing for success rates to be calculated up until the most recent month.
Source Data
7 The method makes use of the following datasets in order to produce the required outputs:
Derived Variables and Output Datasets
8 The methodology produces the following derived variable(s)
Field Name | Label | Dataset |
frm_succrate | The framework success rate. This is the proportion of leavers who achieved the framework, represented as a percentage. | WBL Success Rates Provider Reports |
nvq_succrate | The NVQ success rate. This is the proportion of leavers who achieved the NVQ, represented as a percentage. | WBL Success Rates Provider Reports |
frm_nvq_succ | The framework or NVQ success rate. This is the proportion of leavers who achieved either the framework or the NVQ, represented as a percentage. | WBL Success Rates Provider Reports |
timely_frm_succrate | The timely framework success rate. This is the proportion of leavers who achieved the framework within a month of their planned end date, represented as a percentage. | WBL Success Rates Provider Reports |
timely_nvq_succrate | The timely NVQ success rate. This is the proportion of leavers who achieved the NVQ within a month of their planned end date, represented as a percentage. | WBL Success Rates Provider Reports |
timely_frm_nvq_succrate | The timely framework or NVQ success rate. This is the proportion of leavers who achieved the framework or the NVQ within a month of their planned end date, represented as a percentage. | WBL Success Rates Provider Reports |
9 The method produces the following datasets
Detailed definition
10 This methodology follows on from preceding methodologies, which are carried out in the following order:
11 Any non-achieving learners who withdrew from their course within 6 weeks from the start date are removed from the calculation. This is to give similar allowances to WBL success rates as are given for FE success rates, where learners who withdraw before 1 November of each academic year are not counted in success rates.
12 Transfers to different courses at the same provider are also now excluded from success rate calculations. The outcome of the course to which the learner is being transferred will count for success rate purposes instead.
Overall Success Rates
13 Any learner who did not have an actual end date as of the latest freeze in the data is marked as ‘continuing in learning’ and excluded from the calculation. The exception to this is learners who were in an earlier year but still have had no end date recorded; these learners are considered to have gone missing and count as a withdrawal from learning without an achievement.
14 Learners are grouped into years based on their hybrid end year, which is either the year their learning was planned to end in or the year they actually ended in if they have gone beyond their planned end year.
15 Learners are only counted if their provider returned ILR data in their hybrid end year.
16 For any particular group, framework success rates are calculated to be (sum of framework achievements) / (sum of leavers) * 100.
17 For any particular group, NVQ success rates are calculated to be (sum of NVQ achievements) / (sum of leavers) * 100.
18 The overall ‘Framework or NVQ Success Rate’ can then be calculated for apprenticeship programme types. This is calculated by adding the individual framework success rate to the NVQ success rate.
Timely Success Rates
19 Any learner who did not have an actual end date as of the latest freeze in the data is marked as ‘continuing in learning’. These learners still count towards the timely success rate calculation. The exception to this is learners who were in an earlier year but still have had no end date recorded; these learners are considered to have gone missing and also count as a withdrawal from learning without an achievement.
20 Achievements are only counted if they occur within 31 days of the planned end date originally set for the aim. Any achievement after this date is not counted and will appear as a non-achievement in the success rate calculation.
21 Learners are grouped into years based on their expected end year, which is the year that the planned end date falls within.
22 Learners are only counted if their provider returned ILR data in their planned end year.
23 For any particular group, framework success rates are calculated to be (sum of framework achievements) / (sum of leavers + continuing learners) * 100.
24 For any particular group, NVQ success rates are calculated to be (sum of NVQ achievements) / (sum of leavers + continuing learners) * 100.
25 The timely ‘Framework or NVQ Success Rate’ can then be calculated for apprenticeship programme types. This is calculated by adding the individual framework success rate to the NVQ success rate.
Actual and Expected End Year Comparison Table
26 In the provider reports there is also a table showing when learners left compared to when they were expected to leave. This is primarily to aid understanding of how cohorts of leavers are defined in both of the new measures.
27 These tables are only calculated for apprenticeship courses. Therefore, the first step is to remove any learners that are not on apprenticeship courses or withdrew within the first 6 weeks of the apprenticeship.
28 For the group being examined, the data is counted so that planned end years form the rows of the table with one column for every actual end year. One table is calculated for each of three groups of learners:
All leavers within the master file.
All framework achievers within the master file.
All framework or NVQ achievers within the master file.
29 Extra columns are provided at the right side of the comparison table showing the number of leavers/achievers in each year so that people using the figures can see how the cohorts have been derived from the table.
Sample Code
30 The following sample code is available
