FAQs for Employer Responsive QSRs & Employer Responsive MLP
FAQs
1.
When will reports be published? (ER) 2.
Why can I not open my zip file? 3.
Which of my submitted ILR files have you used to produce this report? 4.
Why do I not have either a QSR or a MLP report ? 5.
Some types of provision seem to have been omitted from my report – Why is this? (ER TTG) 6.
My 2008/09 TTG timely figures have changed from the final 2008/09 QSR report when comparing to the latest 2009/10 TTG QSR report. Why? (ER TTG) 7.
Why are there two different measures of success – ‘overall’ and ‘timely’ on my report?
8.
Why have the ‘timely’ success rate criteria been changed? (ER APP) 9.
What is a ‘hybrid year’? 10.
What is a 'timely' leaver? 11.
What is an 'overall' leaver?
12.
My historical success rates for previous years (2007/8, 2006/7 etc) are being reported under a different Unique Identifier from the figures in the Period 6 report – has my data been changed? 13.
My organisation has merged with / been taken over by another organisation recently. Where and how are success rates for the new / old organisation reported?
14.
My historical success rates for previous years (2007/8, 2006/7 etc) are different from the figures in last year’s report – has my data been changed? (ER TTG) 15.
My historical success rates for previous years (2007/8, 2006/7 etc) have changed from the figures in last year’s report – has my data been changed? (ER APP)
16.
My report includes some success rates of ‘0%’ and some that are blank. I’m confused. 17.
My report seems shorter/smaller than last year’s - has any data been omitted? 18.
My QSR report seems to be incorrect compared to the data set attached to it. The learners are appearing in the wrong Sector Subject Area. What’s going on? 19.
Why has the analysis of Sector Subject Areas changed? (TTG) 20.
Why has the analysis of Sector Subject Areas changed? (TTG) 21.
Why does my report no longer have an ‘England’-level summary? 22.
We are trying to match up our QSR success rates data with our own internal routines, but we’re finding some issues. We believe the main reason for this is our region mapping is clearly not the same as that used to prepare our ER QSR results. 23.
Why do learners with the same aim (A09) appear twice? (TTG)
24.
Why is the SME flexibilities element of the Train to Gain reports missing? (ER TTG) 25.
Which elements of the SME Flexibilities of Train To Gain have been excluded from my success rate report? (ER TTG) 26.
Which elements of the Joint Investment Programme (JIP) learning aims of Train To Gain have been excluded from my success rate report? (ER TTG) 27.
Why is there now an analysis of success by Gender? 28.
Why is there now an analysis of success by Ethnicity? 29.
This learner appears multiple times on the QSR however they have different Ethnicity codes, why is this? (ER TTG) 30.
Why is there now an analysis of success by Learners with Learning Difficulties and / or Disabilities? 31.
Why is there now an analysis of success by age-band? (ER APP) 32.
How is the success rate calculated?
33.
How is the achievement rate calculated?
34.
How is the retention rate calculated?
35.
Will my funding be affected/what will be the impact on my funding? (ER MLP) 36.
At what level is MLP applied (ER MLP) 37.
How can I find out more about QSR & MLP policy and application (ER MLP)
38.
Does the MLP calculation use the 'Overall' success rate or the 'Timely' success rate? (ER MLP) 39.
My MLP zip file contains two pdf reports - one for Overall that's colour-coded, a second one for Timely that's not. What's going on? (ER MLP) 40.
Some areas of provision appear in the main (Overall) MLP report but have been omitted from the supplementary (Timely) report - or vice versa. Why is this? (ER MLP) 41.
Why do I have a section for a local authority on MLP but not on QSR? (ER MLP) 42.
When was the 2009/10 Employer Responsive Methodology for Period 15 available? (ER QSR/ ER MLP) 43.
When will the 2010/11 Employer Responsive Methodology be available? (ER MLP)
1. When will reports be published? (ER)
Answer
ER success reports are published to the following schedule:
Period 6:
Freeze: 17th February 2011
QSR Publication: 25th March 2011
Period 9:
Freeze: 19th May 2011
QSR Publication: 26th May 2011
Period 12:
Freeze: 17th August 2011
QSR Publication: 25th August 2011
Period 15:
Freeze: 22nd December 2011
QSR Publication: 12th January 2012
MLP Publication: 12th January 2012
Background:
The Period 15 QSR & MLP reports for the 2010/11 are to be published on 12 January 2012. In order to calculate the Timely measure of success for 2010/11, which is included in these reports, data for periods 1 - 3 is required from providers delivering adult skills provision. For FE Colleges this data will be provided in the R04 return. Return R04 has a return date of 06/12/11 and a freeze data of 22/12/11.
Back to top
2. Why can I not open my zip file?
Answer
Providers who encounter this problem should ask their IT team for advice as they will need to have an appropriate zip tool installed for them to be able to open the file. The Data Service can provide a link to free software http://www.winzip.com/trialpay.htm
Back to top
3. Which of my submitted ILR files have you used to produce this report?
Answer
Should you need to know which ILR submission files were used in the calculation of the QSRs please contact our Service Desk on 0870 2670001 or email servicedesk@thedataservice.org.uk .
Background
Although the name of the source ILR submission file used in the production of the report was present on the period 6 reports it has been removed from the period 9 and subsequent reports. Data for the success rates calculation can come from a number of ILR submission files and also from Provider Online. For previous reporting periods this file name was incomplete and occasionally misleading. As it cannot be guaranteed that complete and correct submission information can be shown, the ILR submission file name has been removed from the footer.
4. Why do I not have either a QSR or a MLP report ?
Answer
This is usually because there are no completed learning aims to report on in the current year, or in any of the three previous years.
Back to top
5. Some types of provision seem to have been omitted from my report – Why is this? (ER TTG)
Answer
The full list of exclusions can be found in the Employer Responsive Success Rates methodologies for 2009/10, which can be found on the Data Service website via the statistics link. For 2009/10 the Train to Gain reports exclude Flexibilities level 2 and level 3 units from, and include Flexibilities thin level 2 and thin level 3 qualifications in, the data reported.
Background
The Flexibilities exclusion has been refined in Period 12 of 2009/10 – some Skills for Life & Foundation Learning Tier aims were being wrongly excluded. These are now correctly included in the reports.
Back to top
6. My 2008/09 TTG timely figures have changed from the final 2008/09 QSR report when comparing to the latest 2009/10 TTG QSR report. Why? (ER TTG )
Answer
Should you need to know which ILR submission files were used in the calculation of the QSRs please contact our Service Desk on 0870 267 0001 or email servicedesk@thedataservice.org.uk .
Back to top
7. Why are there two different measures of success – ‘overall’ and ‘timely’ on my report?
Answer
This is because – The “overall” success rate measures the proportion of learning aims achieved against the overall number of learning aims completed. The overall success measure is based on the hybrid end year. The hybrid end year is the later of either the actual learning end year or the planned learning end year.
The “timely” success rate only measures learning aims achieved no later than 90 days after their planned end date (NB in the case of Apprenticeships, this has changed from 31 days). This is expressed as a percentage of the number of learning aims expected to be completed in the year.
In order to report timely success rates more accurately at each period, reports for period 6 show the success rate for Period 3 with data for the following 90 days included - i.e. outcomes for Periods 4, 5 and 6 of 2010/11. This harmonises the methodology with that used for Apprenticeship provision.
Background
Overall and timely measures were introduced for Train to Gain in 2009/10. The Apprenticeships timely limit changed to 90 days from 2009/10 onwards.
Back to top
8. Why have the ‘timely’ success rate criteria been changed? (ER APP)
Answer
In the case of Apprenticeships the timely success rate now measures learning aims achieved no later than 90 days after their planned end date, rather than 31 days as in previous years. This harmonises with the methodology used in the Train To Gain provision.
Background
Overall and timely measures were introduced for Train to Gain in 2009/10. The Apprenticeships timely limit changed to 90 days from 2009/10 onwards.
Back to top
9. What is a ‘hybrid year’?
Answer
The hybrid end year is the later of either the actual learning end year or the planned learning end year.
Back to top
10. What is a 'timely' leaver?
Answer
A timely leaver is an aim that has been completed with an actual end date within 90 days of the expected end date of the aim, and the completion status indicates that the learner has completed the learning activities leading to the aim.
Back to top
11. What is an 'overall' leaver?
Answer
An overall leaver is an aim that has been completed with an actual end date and the completion status indicates that the learner has completed the learning aims leading to the aim. It is not necessary for the actual end date to be within 90 days of the expected end date.
Back to top
12. My historical success rates for previous years (2007/8, 2006/7 etc) are being reported under a different Unique Identifier from the figures in the Period 6 report – has my data been changed?
Answer
No. From Period 6 2010/11 a fix has been applied to reflect mergers that took place between Learner Responsive providers in 2009/10 and which had already been applied to the LR QSR reports.
If you are one of these providers, your success rates are now reported under the new UPIN and name of the post-merger institution, reflecting the position as at the start of this academic year (1st August 2010).
Background
This change reflects mergers between FE colleges that also provide Apprenticeship or Train to Gain provision, and that took place before 1st August 2009 and harmonises the approach with that of LR reporting. If you are an FE institution that has merged after 1st August 2010, your LR and ER QSRs will be updated to reflect this change when the Period 6 2011/12 ER QSRs and the 2011/12 LR QSRs are produced.
Back to top
13. My organisation has merged with / been taken over by another organisation recently. Where and how are success rates for the new / old organisation reported?
Answer
From Period 6 2010/11 a fix has been applied to reflect mergers that took place between Learner Responsive providers in 2009/10 and which had already been applied to the LR QSR reports. If you are one of these providers, your success rates are now reported under the new UPIN and name of the post-merger institution, reflecting the position as at the start of this academic year (1st August 2010).
Background
This change reflects mergers between FE colleges that also provide Apprenticeship or Train to Gain provision, and that took place before 1st August 2009 and harmonises the approach with that of LR reporting. If you are an FE institution that has merged since 1st August 2010, your LR and ER QSRs will be updated to reflect this change when the Period 6 2011/12 ER QSRs and the 2011/12 LR QSRs are produced.
Back to top
14. My historical success rates for previous years (2007/8, 2006/7 etc) are different from the figures in last year’s report – has my data been changed? (ER TTG)
Answer
No. Success rates for all years on the report are calculated using the Demand-Led Funding (DLF) methodology and overall and timely methodology measures. DLF methodology only has true currency since 2008/09 and overall and timely measures only have true currency for 2009/10. Therefore the reports do not show a true measure of success for years prior to 2009/10. For a consistent record to assess historic performance, use QSR reports for previous years on Provider Gateway.
Back to top
15. My historical success rates for previous years (2007/8, 2006/7 etc) have changed from the figures in last year’s report – has my data been changed? (ER APP)
Answer
Overall success rates should be the same. Timely success rates may have varied due to the change from a 31-day to a 90-day definition of timeliness. To view historic performance, QSR reports for previous years can be found on the Provider Gateway.
Back to top
16. My report includes some success rates of ‘0%’ and some that are blank. I’m confused.
Answer
You are now receiving a QSR where you have any leavers in either the current year or any of the three previous years. In some cases this will mean that one or more of the previous years (or sometimes the current year) do not have any leavers at all. In this case, the rate shows as a blank. In other cases, you will have a cohort of learners who have left without achieving. In this case, the success rate will be zero.
Back to top
17. My report seems shorter/smaller than last year’s - has any data been omitted?
Answer
No. But duplicate data has been suppressed. Particularly where Sector Subject Area hierarchies would lead to the repetition of the same data at each level of aggregation. (See below).
Back to top
18. My QSR report seems to be incorrect compared to the data set attached to it. The learners are appearing in the wrong Sector Subject Area. What’s going on?
Answer
The current standard method for mapping Apprenticeship success rates onto the Qualifications and Curriculum Development Agency (QCDA) ‘Sector Subject Areas’ at Tier 1 and Tier 2 was agreed by both the LSC (before the Machinery of Government transition on 1 April 2010) and Ofsted, working with the Sector Skills Councils and other representative bodies. The Skills Funding Agency and Young People's Learning Agency (YPLA) support this previous agreement and will continue to use it until it is appropriate to undertake a further review.
This action only addresses current frameworks - new frameworks will be allocated to a Single Subject Area by Sector Skills Councils and Sector Skills bodies, as they are finalised and ready to be funded. The purpose of introducing the change is to resolve any confusion amongst providers, and conflicting messages about performance based on the different approaches currently used by the Sills Funding Agency, Young People’s Learning Agency and Ofsted.
Please note - The method for calculating 'Overall and Timely'* success rates will not change, but the distribution of individual
frameworks to Tier 1 and Tier 2 will be different in some instances.
Background
Impact of the change will only be seen when the success rate results are reported at Sector Subject Area. Results at provider level will not be affected. Providers who deliver frameworks which may have previously occurred in more than one Sector Subject Area will see differences to the success rates previously reported. The implications of these changes are currently being examined for the application of Minimum Levels of Performance to Apprenticeship provision for the forthcoming commissioning year.
Due to the change in the ILR, the previous method of mapping is no longer appropriate for 2008/09 data. Therefore, a new approach was implemented with the release of 2008/09 Period 9 QSR reports, and will apply to all future reports. Providers will need to consider the effect of this on Sector Subject Area focused sections of their self-assessment reports and quality improvement plans. The data is held in the LSC_FRAMEWORK_AVS table in the LAD downloads. You need to go to search LAD downloads, open All CSV Zip file and this holds the table highlighted above.
Back to top
19. Why has the analysis of Sector Subject Areas changed? (TTG)
Answer
An analysis of Tier 2 Sector Subject Areas has been added to these reports where appropriate data exists. The Sector Subject Area (SSA) section of the detailed reports has been changed to reduce duplication of data if there is only one qualification type the summary section will not be shown or if there is only SSA T2 for an SSA only the results for the SSA T2 will be shown.
Back to top
20. Why has the analysis of Sector Subject Areas changed? (TTG)
Answer
The Sector Subject Area (SSA) section of the detailed reports has been changed to reduce duplication of data. If data for an SSA are recorded in only one Sector Framework Code (SFC) then the SSA Tier 1 and SSA Tier 2 summaries will not be shown. Similarly if in an SSA there are only data for one SSA Tier 2 then the SSA Tier 1 summary will not be shown. From Period 9 2009/10, totals for Apprenticeship Types have been added at Sector Subject Area Tier 1, Tier 2 and Framework level.
Back to top
21. Why does my report no longer have an ‘England’-level summary?
Answer
This has been re-named to the ‘All Regions’ summary.
Back to top
22. We are trying to match up our QSR success rates data with our own internal routines, but we’re finding some issues. We believe the main reason for this is our region mapping is clearly not the same as that used to prepare our ER QSR results. Could you therefore point me in the direction of where I can pick up the definitive data lookups for the delivery region and local authority based on the delivery postcodes of the learners?
Answer
The Qualification Success Rates reporting system has been changed in Period 12 of 2009/10 to use the Office Of National Statistics (ONS) National Statistics Postcode Directory (NSPD). This is used to map learner home post-codes and workplace post-codes onto Local Authorities and Regions. This database is available from The Office of National Statistics
or via:
- Customer Contact Centre
- Office for National Statistics
Room 1.101
Government Buildings
Cardiff Road
Newport
South Wales, NP10 8XG
Background
This brings ER reporting in line with the practice of the SFR in respect of this data source. Our checks on the likely impact of this change suggest that fewer learner & provider postcodes will go unrecognised. We are not aware of any examples of a learner or provider postcode changing from one recognised region or local authority to another as a result of this change.
Back to top
23. Why do learners with the same aim (A09) appear twice? (TTG)
Answer
This is usually because a different start date (A27) has been recorded for the learner than the start date recorded in a previous year’s final return. The methodology matches records returned on L01, L03, A09 and A27 and will use the most recently recorded start date. If this start date is different to a previously recorded start date, it will cause a mismatch across L01, L03, A09 and A27, and result in duplication in the A09 field. Provided that this information is corrected in the final return, and the correct start date for the learner is recorded, the duplication will not appear in the end of year reports.
Back to top
24. Why is the SME flexibilities element of the Train to Gain reports missing? (ER TTG)
Answer
This issue was resolved with the release of the Period 12 of 2009/10 reports on 20 August 2010. A table of the aims included in the SME flexibilities definition is available on the Data Service website:
Back to top
25. Which elements of the SME Flexibilities of Train To Gain have been excluded from my success rate report? (ER TTG)
Answer
Level 2 Units and Level 3 Units are now excluded from Train to Gain success rates. The units were issued on a trial basis and therefore are now excluded from success rate calculations as they will not be an ongoing part of Train to Gain and are still subject to wider trials.
Background
On 2nd January 2009 Flexibilities were introduced within Train to Gain which included access to a restricted list of QCF units and thin qualifications for use with Small to Medium Sized Enterprises (SMEs). This flexibility was to support SMEs in the recession whilst awaiting the wider expansion of QCF qualifications. Guidance Note 3, issued on 3rd March 2010, gave notice that access to QCF units was to end on 31st July 2010.
Back to top
26. Which elements of the Joint Investment Programme (JIP) learning aims of Train To Gain have been excluded from my success rate report? (ER TTG)
Answer
A range of qualifications can be agreed with employers within a JIP including Apprenticeships and other employer focused qualifications within the QCF. Within the other employer focused qualifications, certain agreed delivery patterns for units of QCF qualifications are available, however these will be excluded from success rate calculations. Inclusion of units within success rate calculations may skew the overall success rate results for a provider due to their relative short duration and low delivery content. The Skills Funding Agency is running pilot studies into the development of unit delivery and appropriate success rate methodology for that type of provision.
Background
This funding is directed specifically at those qualifications and units of qualifications identified as solutions to specific skills needs. Successful applicants will play a leading role when working with approved Skills Funding Agency colleges and training organisations to develop an employer/provider partnership which delivers the specific sectoral provision which includes to long-term capability and a capacity with committed investment.
Back to top
27. Why is there now an analysis of success by Gender?
Answer
Sub-totals by gender have been added to the summary report. These are now broken down by Apprenticeship type in the Apprenticeships reports and by Level / provision type in the Train to Gain reports.
Back to top
28. Why is there now an analysis of success by Ethnicity?
Answer
Sub-totals by ethnicity have been added to the summary report. These are now broken down by Apprenticeship type in the Apprenticeships reports and by Level / provision type in the Train to Gain reports. If there are no learners for an ethnic type, then that ethnicity is omitted.
Back to top
29. This learner appears multiple times on the QSR however they have different Ethnicity codes, why is this? (ER TTG)
Answer
This is usually because the ethnicity information submitted by the learner to the provider for recording is different to the ethnicity recorded in a previous year’s return.
The process identifies the latest recorded L01, L03, A09, A27 combination from all final files submitted since 2006/07. So if the learner has more than one A09 record and these are from different years, the reports will show the learner with the ethnicity from these years which could be different.
Back to top
30. Why is there now an analysis of success by Learners with Learning Difficulties and / or Disabilities?
Answer
Sub-totals have been added to the summary report showing success rates for Learners with and without Learning Difficulties and / or Disabilities. If there are no learners for a status, then the status is omitted.
Background
The analysis shows this information where it is confirmed by the learners themselves. They may have identified themselves as having or not having learning difficulties. Where no information is recorded or available, the learner is reported by default as not having learning difficulties or disabilities. This is a high-level analysis only. There are no plans at present to report on types of disability or learning difficulty.
Back to top
31. Why is there now an analysis of success by age-band? (ER APP)
Answer
Sub-totals by age band have been added to the summary report. Each geographical area now has results shown as 16-18, 19-24, 25+ and All Ages. If there are no learners for an age-band, then the age-band is omitted. If there are only learners in one age band, then the All Ages section is omitted.
Back to top
32. How is the success rate calculated?
Answer
In simple terms success rate is the number of aims achieved divided by the number expected to have been completed.
Back to top
33. How is the achievement rate calculated?
Answer
In simple terms achievement rate is the number of aims achieved divided by the number completed.
Back to top
34. How is the retention rate calculated?
Answer
In simple terms retention rate is the number of aims completed divided by the number started.
Back to top
35. Will my funding be affected/what will be the impact on my funding? (ER MLP)
Answer
The Data Service cannot comment on this but in the first instance you should raise your query through the Service Desk: 0870 2670001
or email servicedesk@thedataservice.org.uk . Your query will then be directed to the appropriate area.
Back to top
36. At what level is MLP applied (ER MLP)
Answer
This is normally at Regional level but for further details see MLP policy
Back to top
37. How can I find out more about QSR & MLP policy and application (ER MLP)
Answer
Refer to MLP policy documents .
Back to top
38. Does the MLP calculation use the 'Overall' success rate or the 'Timely' success rate? (ER MLP)
Answer
For 2009/10 both Apprenticeship and Train to Gain MLP is calculated using the 'Overall' success rate measure.
Back to top
39. My MLP zip file contains two pdf reports - one for Overall that's colour-coded, a second one for Timely that's not. What's going on? (ER MLP)
Answer
The second 'Timely' report is provided as supplementary information only. It is not used to determine whether provision meets or falls short of minimum levels of performance and accordingly provision is not colour-coded.
Back to top
40. Some areas of provision appear in the main (Overall) MLP report but have been omitted from the supplementary (Timely) report - or vice versa. Why is this? (ER MLP)
Answer
The main report reflects the performance of provision by the overall measure - this circumstance will arise when an entire cohort of leavers succeed according to the overall method but are not timely leavers (i.e. within 90 days of expected end date). Accordingly, their aims will appear in the overall report but not the timely report.
Back to top
41. Why do I have a section for a local authority on MLP but not on QSR? (ER MLP)
Answer
This is probably because there were fewer than 20 learners in 2008/09 which is the threshold for QSR and more than 10 learners which is the threshold for MLP.
Back to top
42. When was the 2009/10 Employer Responsive Methodology for Period 15 available? (ER QSR/ ER MLP)
Answer
This was published on the Data Service website in November 2010.
Back to top
43. When will the 2010/11 Employer Responsive Methodology be available? (ER MLP)
Answer
This was published on the Data Service website in March 2011.
Back to top
