FAQs for Learner Responsive National Success Rates Tables

FAQ:

Question 1: The spreadsheets won’t load into my version of Excel (NSRT) 

Question 2: Why is the provider’s location Local Authority analysis only available for 2008/09 onwards? (NSRT)

Question 3: Some of the provision that is unique to my institution doesn’t appear on your tables.  Why has this been omitted? (NSRT)  

Question 4: The success rates for the sub-cohorts don’t add up to 100% of the rate for the total cohort.  What’s going on?    (NSRT)

Question 5: How is the success rate calculated? (NSRT)

Question: 6 How is the achievement rate calculated? (NSRT)

Question 7: How is the retention rate calculated? (NSRT)

Question 8: My institution’s provision is radically different from the national mix.  How do I use the published success rates to apply their standards to our situation? (NSRT)

Question 9: What qualifications are not included? (NSRT)

Question 10: What is a ‘start’? (NSRT)   

Question 11 - I don’t understand the L17 & L23 local authority percentiles analysis – aren’t these supposed to be national figures? (NSRT)

Question 12 - The L17  or L23 local authority percentiles analysis percentiles don’t seem to be correct:  the figures for success, achievement and retention all seem wildly different from what I know providers in my local area achieve (NSRT)

Question 13: I can’t find a national success rate or percentile analysis for a particular type of provision either delivered by my provider or within my local authority.  How can I find a national success rate for, for instance, Entry To Employment provision? (NSRT)

Question 14: Why has the 'High Grades' measure been removed from the 2009/10 tables? (NSRT)

Question 15: Why are there Achievement Rate percentiles values of over 100% in some cases?  (NSRT)

---

Question 1: The spreadsheets won’t load into my version of Excel (NSRT)

Answer - Some versions of Excel (earlier than Excel 2007) have a constraint on the size of worksheet that they can handle (approx 65k rows of data).  Wherever this limit is likely to be breached we have split area analyses across two worksheets (e.g. the L17 learner post-code analysis is contained in two alphabetical sheets, A-L & M-Z).       

Background - This is a software design constraint in Excel.  Wherever possible users should consider upgrading their installation of Excel to 2007 or better.

Question 2: Why is the provider’s location Local Authority analysis only available for 2008/09 onwards? (NSRT)

Answer - Reliable records of the provider’s location post-code exist for the period 2008/09 onwards.  For earlier years we use a source of data that does not include this information, and it is not possible to extrapolate reliably back in time.  For this reason we have treated 2008/09 as the first year for which this analysis was available.  It has been extended into 2009/10 and will be rolled forward in future years.


Question 3: Some of the provision that is unique to my institution doesn’t appear on your tables.  Why has this been omitted? (NSRT)

Answer - The success rates tables are built from identifiable cohorts of learners undertaking learning aims expected to end in a particular year.  Because the tables show national rates, the threshold for identification of a cohort has been set to a learning aim undertaken by at least 30 learners at 2 or more institutions nationally.  

Question 4: The success rates for the sub-cohorts don’t add up to 100% of the rate for the total cohort.  What’s going on? (NSRT)

Answer - There are two possible reasons for this:
(i) success rate figures are rounded to the nearest integer at all levels
(ii) extremely small cohorts of achievers (fewer than 5 learners) are suppressed from the published tables in order to prevent the risk of  possible identification of individuals’ successes.  However, these extremely small cohorts’ figures are still included in the higher level cohort totals.


Question 5: How is the success rate calculated? (NSRT)

Answer - In simple terms the success rate is the number of aims achieved divided by the number started expressed as a percentage .


Question 6: How is the achievement rate calculated? (NSRT)  

Answer In simple terms the achievement rate is the number of aims achieved divided by the number actually completed expressed as a percentage   


Question 7: How is the retention rate calculated? (NSRT)

Answer - In simple terms the retention rate is the number of aims actually completed divided by the number started expressed as a percentage   


Question 8: My institution’s provision is radically different from the national mix.  How do I use the published success rates to apply their standards to our situation? (NSRT)

Answer - The national success rates tables provide a wide variety of analyses against which a single institution can be contextualised and interpreted.  Amongst these are tables that analyse by qualification type and level.  The rates for these types and levels can be adapted to the needs of your institution by amending them to reflect the proportion of provision you actually deliver.  A worked example demonstrating how to do this is included in the document National Success Rates Tables 2009/10 – Guidance notes published on the Data Service website.     


Question 9: What qualifications are not included? (NSRT)

Answer - Key Skills, Functional Skills and OLASS provision are excluded from the tables.  Other exclusions are documented in the Learner Responsive methodology


Question 10: What is a ‘start’? (NSRT)  

Answer - A ‘start’ is a learning aim that is planned to end in the year and the learner: -
• has not transferred within the provider
• is funded by the Skills Funding agency or the Young People’s Learning agency
• Is not under 16 at start of learning
• did not withdraw in the first six weeks 

Question 11 - I don’t understand the L17 & L23 local authority percentiles analysis – aren’t these supposed to be national figures? (NSRT)

Answer: In the case of the Local Authority post-code based percentiles, these measure the variability of the subset of providers with locations in that authority (A23) or learners resident in that authority (L17).   

Question 12 - The L17  or L23 local authority percentiles analysis percentiles don’t seem to be correct:  the figures for success, achievement and retention all seem wildly different from what I know providers in my local area achieve? (NSRT)

Answer - In these calculations, all cohorts of learners are given equal weight, irrespective of the size of the cohort.  This means that a percentile for a local authority can be made up of a cohort of the bulk of learners attending the local FE college, and many other cohorts of very small numbers of learners who have travelled outside their local authority to pursue a course of study.  In the case of L17 (learner home postcode) in particular, the equal weighting of each provider may have a significant bearing on the results for an authority.  Care should be exercised in the interpretation and use of these analyses.        

Question 13: I can’t find a national success rate or percentile analysis for a particular type of provision either delivered by my provider or within my local authority.  How can I find a national success rate for, for instance, Entry To Employment provision? ( NSRT)

Answer - The national success rates tables are derived from the same data sources used in the Qualification Success Rates (QSR) reports for 2009/10 and for previous years.  These data sources are in turn derived from the F05 ILRs for each year according to the published rules for recognising and defining success (‘the methodology’).   These rules exclude various categories of provision and aims.  Among these are Offenders Learning & Support Services (OLASS) provision, University for Industry (UfI), Skills for Life, Entry to Employment, etc.  Consequently, because providers are not measured by qualification success rates for these aims and types, national success rates tables do not provide contextual national rates for them.   
Background - The full list of exclusions of provision and aim types can be found in the published 2009/10 Learner Success Rates methodology.  This can be found on the Data Service website at:

http://www.thedataservice.org.uk/datadictionary/technicaldefinitions/derivedvariables/0809/summary.htm