Learner Responsive 08/09 QSR Republication FAQs (June)

Follow this link to read a statement giving an overview on the action being taken to address the issue of duplicate records on Learner Responsive 08/09 QSR

Questions

1. What circumstances contributed to the appearance of the ‘duplicate records’?

2. Why were duplicate records created?

3. What is the scale of the issue?

4. How many providers have been affected?

5. What is the impact on these providers?

6. What will this do to National Figures?

7. What action is being taken?

8. What are the benefits of making this change across all providers?

9. Will my organisation’s success rates go down?

10. Will MLP reports also be reproduced?

11. When will the new reports be available to providers?

12. Will this change be applied to the 09/10 methodology as well?

13. What if I want to raise a query about this issue?

Question 1. What circumstances contributed to the appearance of the ‘duplicate records’?

Answer:

a) The calculation of providers’ Qualification Success Rate (QSR) was amended in the 08/09 version to match records on the start date of a learning aim rather than start year. This was in order to improve the accuracy of the identification and matching of learning aim records spanning multiple academic years on the same programme. Rules by which the data was matched utilised existing ILR guidance.

b) Some providers had not been sufficiently quality controlling their data to comply with this standard and since this had not been previously used in any statistics returned to the provider in the past this lack of data quality control had gone unnoticed.

c) The publishing of the QSR learner responsive methodology in mid-December 2009 left providers with little time to discover any anomalies in their data that would be affected by the change in methodology prior to the hard close date of F05 on 15th January 2010.

d) The time window for submission of F05 data was shortened in most providers due to the Christmas closure.

Back To Top

Question 2. Why were duplicate records created?

Answer: In a case where an aim was returned with different start dates across two years the QSR matching process would result in two aim records (producing a duplicate record so) In every such pair of records it is likely that at least one will show as a failure, leading to success rates being negatively affected.

Answer: The Data Service has carried out an impact analysis into the scale of the issue by analysing the number of duplicate records generated by the QSR calculation and the effect on QSRs by removing them.

Answer: 179 providers have been impacted by this issue.

Answer: 32% of providers will see a change in their overall success rate. 5% of providers will see a movement of one percentage point or more on their overall success rate. 27% of providers will see a movement of less than one percentage point. The remaining balance of providers will not see any change in their overall success rate.

All restated reports will be published and uploaded to the provider Gateway week commencing 28 June 2010.

Answer: All reports have been restated irrespective of whether the provider’s individual data has changed or not. This is due to the affect on the National success rate which will be impacted by this update.

Answer: As providers cannot re-submit data due to the implementation of the ‘hard close’ rule (first introduced in February 2009) for the final F05 return, the Data Service is undertaking mitigating action on behalf of all the organisations involved in the decision to take action (see below) to ensure that provider performance statistics are accurate.

The Skills Funding Agency, the YPLA, the information authority , Data Service, BIS, Ofsted and Framework for Excellence team jointly agreed to:

• Change the methodology of the LR 2008/09 QSRs to allow matching of records based on ‘start year’

• Recalculate the LR QSR ‘Mastertrim’ file

• Reflect the change in the SFR, this being the first place that the revised figure will be shown due to national statistics guidelines

• Revise the QSR reports and republish these on the provider gateway

• Reflect the change in FfE SR scores

Answer:

a) Removes any possibility that the QSR grade for learner responsive provision would be incorrect because of duplicate records in the system.

b) Ensures that all providers are treated equally.

c) It would raise those organisations who were close to a grade boundary by one, if not two, FfE SR grades.

d) The master file supplied for the publication of any future national statistics would be the same as that used to re-run the SFR and national success rate reports, meaning that all data, national averages etc. will match.

Answer: No. For the vast majority of providers there will be no change in your data and no change to your success rate. For a small number of providers there will be a small change to your data which will not affect your success rate. For the remainder changes to your data will materially affect your success rate

No success rates for any provider will be worse.

Answer: No. The Skills Funding Agency and the YPLA jointly agreed that it was not appropriate to re-run the Minimum Levels of Performance reports as a result of this issue, for a number of reasons:

in the small number of instances where this issue was a contributory factor in a provider falling below the MLP threshold, the LSC treated such cases as exceptions, and a Notice to Improve was not issued.

the MLP reports and the QSR reports are not directly comparable, since the Learner Responsive MLP uses weighted GLH in the methodology underpinning its calculation. The need to ensure the 2 sets of reports are aligned (MLP and QSR) is therefore not an issue.

Answer: Amended national success rate results will then be published in the next Statistical First Release on the 24 June.

Amended provider QSR reports will then be made available on the Provider Gateway during the week commencing 28 June.

Answer: No. The calculation of providers’ Qualification Success Rate (QSR) for 09/10 will revert back to the methodology that will give more accurate matching i.e. providers will be expected to match records on the start date of a learning aim rather than start year in their 09/10 returns.

Answer: If you have any specific queries about this issue, please contact the Data Service’s Service Desk in the first instance on: 0870 2670001.

Back To Top