Assessing students up to age 14 - A better future. Part 6 - Central Tracking of Progress

This blog is subject to these protocols

One of the benefits of SATS results is that they can be easily used by remote parties to identify signs of possible problems in schools.

However unfortunately the broad-brush nature of SATS results reveals little detail regarding the nature of any problems which may exist.

It is likely that those with oversight of schools, be they local authorities, academy chains or central government education services will wish to be able to access and analyse schools tracking systems.  In my opinion it would be wiser that they had access to detailed information regarding students progress than that they just had access to summative grades.

Alan Teece at Alfiesoft had particular insight into the practicalities of how systems could be connected to central government records.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 5 - More about online testing

This blog is subject to these protocols

When designing a system for online testing the following questions will need to be considered:

1. If the results of online tests are to lead to formal accreditation, how should they be administrated?
2. Should it be possible to do tests which do not lead to formal accreditation as well as those which do?
3. Should tests auto-adjust so that if students get many questions wrong easier questions appear?
4. Will the teacher be able to specify which questions should appear on the test?
5. Will the teacher be able to print paper tests?
6. Will criteria be accredited based on a single correct answer, a variety of correct answers, correct answers sustained over time, correct answers in contextualised or connected questions....
7. How will results be viewed and how will this interface with skills which cannot be effectively tested online?

Companies which are already well established in this field include:

Khan Academy :
Originally designed for teaching using YouTube videos rather than for testing and tracking, Khan Academy has set up a substantial infrastructure for online testing and tracking to work in parallel with its videos.  Khan Academy covers many subject areas to a very high level and is well resourced, so it is likely that schools and educators with limited resources will use it, but it does not interface with other student teaching and tracking resources and it may be weak in the specific teaching area which is most important to a group of students.

MyMaths:
MyMaths is well known for its high quality interactive teaching resources.  It is very popular in UK schools and has a substantial tracking infrastructure which teachers can use to rapidly see the progress of all their students with their tests and homeworks in specific subject areas.  However the tracking is by lesson rather than by specific learning point and only maths is available.  MyMaths was rapidly adopted by word of mouth because at an annual subscription of only around £400/year per school it was within the budget of a typical secondary school maths department.

Alfiesoft:
Alfiesoft differs from the products above because its design and emphasis starts from testing rather than teaching, so its designers have thought in more detail about the questions at the top of this blog rather than focusing on creating a system to assess an already designed system of online teaching.  Alfiesoft has an interface with SIMS.  It covers maths, English and science but it does not link to an online teaching system.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 4 - Being allowed to test what is testable.

Reflection and Context:

It's been our habit in the past to try and create a single method for assessing academic progress in mathematics and this has tended to swing between testing everything and getting the teachers to observe everything the student does.

As a head of mathematics I wanted to be able to efficiently and effectively test those aspects of mathematical progress which are easily testable so that my teaching of core techniques could be efficient and needs based, leaving as much time as possible for the nurturing of those skills students need to develop which cannot be easily tested. 

When we focus on teaching methods which allow students to connect maths and work out which mathematical techniques they need, a great deal of learning of mathematical techniques and vocabulary takes place but this progress is difficult to track.  Having a system which takes care of that tracking process for teachers empowers this type of learning.

In an article in MT210 (as Rebecca Teasdale), I described how I used MyMaths tracking to achieve this shift in teaching in a school where I was head of Maths.  We rapidly positively transformed results at this school to surpass all targets and re-engaged disaffected students. 

MyMaths had, at that time, interactive lessons and assessments on all national curriculum topics.  By requiring students to complete them I could ensure they studied all these topics at their own pace and that I had detailed information regarding gaps which needed to be filled.  But far more importantly, we could focus on creating lessons which properly engaged all students by subordinating teaching to learning, teaching with rich and applied tasks and teaching core vocabulary and techniques only when it was clear they were needed.

Key Recommendation:

That to accompany an electronic system which defines the skills students should be acquiring, automatic testing systems should be created which test and automatically accredit those skills which can be easily tested online.  

Subsidiary Points:

- It should be possible for teachers to manually accredit these skills should they wish to do so with alternative evidence.

- Students could be tested in a 'low stakes' way approximately once a term.  Tests should be created which are responsive to students' progress - seeking to test gaps which clearly need to be filled but adjusting during the test to ensure that students do not have overly negative experiences where they fail to answer any questions.  

- Teachers could select core or optional modules of work which would be in each students' test in a class to ensure that focused student preparation brings rewards.

Something a little more challenging:

I think most testable techniques and vocabulary could be accredited on four levels:

A: Some evidence that the student has mastered this skill

B: Sustained evidence that the student has mastered this skill

(both A&B could be tested online with B only being accredited after a student has demonstrated their ability to achieve that target in several tests).

C: Skill used fluently by the student in wider contexts.

D: Student understands the axiomatic or structural roots of the skill (this is not appropriate for arbitrary vocabulary).

And what about the skills which cannot be tested online?

While testing is suitable for most of the 'range and content' component of the National Curriculum, most aspects of Key Concepts, Key Processes, and Curriculum Opportunities cannot be easily tested.  It is my suggestion that these should not be accredited as pass/fail progress but that a process by which students sit down and select or create phrases which reflect how they perceive they are progressing with these aspects of the curriculum be embedded into classroom practice and this self assessment is then discussed with teachers, parents and other stakeholders.  

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 3 - Integrating formative (APP) and summative (SATS) assessment

This blog is subject to these protocols

The biggest efficiency and quality improvements will come when school tracking systems are developed which integrate formative assessment (APP) and summative assessment (SATS).

At present many schools spend a great deal of time recording the details of students progress but they usually only share this data among staff.  Students only receive credit for the grades they achieve in their SATS and other exams and these grades/levels reveal little except whether the students has learnt much this year or not.

However it is now been clearly demonstrated that the landscape of what is now possible has fundamentally changed.  Ian Lynch has a company called The Learning Machine which is licenced by Ofqual to issue criteria based qualifications which they call Ingots.

The Learning Machine have developed qualification which are specified by the criteria students have to achieve.  They have created tracking systems which must be used to record progress.  Schools are responsible for creating their evidence trails and if they subscribe to the system they can then issue as many qualifications as they are able to justify.  They are audited by The Learning Machine once a year.   So far they have specilised in developing qualifications to support emerging ICT systems such as Open Office and to assess and give formal credit for progress to students with learning difficulties.

This is precisely the kind of system which could positively transform assessment for students up to the age of 14.

However this model of accreditation needs to be developed to include:
- online testing and subsequent automatic accreditation for criteria which are easy to test (and ideally to online teachings systems which support students in filling gaps identified) so that teachers can focus on observing those aspects of student progress which are not easy to test.
- links to whole school registration systems to ensure it is easy for teachers to populate fields.
- subsequent links to behavior tracking systems and other student, staff or school tracking and development systems.
- shared access and use for teachers, student and parents.
- transferability between schools, especially from primary schools to secondary schools.
- links to the central database so that detailed data can be used to assess national and school progress by government authorities.

In subsequent blogs on this theme I will explore these points sharing insights into how they can be effectively addressed and the benefits their addressed will bring.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 2 - protocols for this series of blogs

As this series of blogs progresses I will refer to individuals and companies I have spoken to who have brought to life the individual components of future systems which I believe will transform education when they are combined.

I am very grateful to all who have taken the time to meet me or chat to me on the phone, in forums or by email.

However I am aware that by following this strategy it is inevitable that I will miss individuals and companies with similar or different relevant skills and products to offer.  I would like to warmly invite anyone who would like to gain exposure for their relevant ideas, services or products to add comments to the relevant blog which either raise relevant points or describe what they have to offer.

If you would like advice or guidance as to where to place your comments or would just like to communicate with me about any of these issues please do get in touch through linkedin or by commenting on this blog and giving your contact details.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 1 - key components of the way forward

It is of great benefit for students, teachers and parents to have detailed information regarding precisely what each student can do and where their knowledge gaps are.

However:
SATS, as a measure of school accountability are too narrow and skew what schools actually do away from best practice.

and

SATS levels do not provide much useful information.  It would be much better if detailed information regarding precisely what at student is doing well and what they are struggling with were shared.


Key Components of the Way Forward:
A coherent solution will require two key changes in practice.

1. The integration of formative and summative assessment.
In other words schools systems by which they track the detail of students' progress (APP systems) should lead directly to accreditation and SATS should be abolished.

2. Formative assessment must be made coherent by using different types of assessment for those targets which can be easily tested and those which cannot.
Most current systems of APP are not currently suitable for direct accreditation.  We need to specify and track targets which can be easily tested using different processes to those used to specify and track targets which cannot be easily tested.



In this next series of blogs I will explore how these changes can be achieved and some of the many benefits they will bring to students' education.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Les Ebdon, Vince Cable and Oxbridge Access

I’ve been working hard with students from tough backgrounds on Oxbridge access for the last 12 years. Every year we’ve prepared groups of state school students for interviews. These groups have always contained middle class state school students who have a professional parent and who have plenty of extra curricular activities to talk about and the personal confidence to talk about them. They have also contained students from disadvantaged backgrounds. No child from the latter group has ever been offered a place at Oxbridge from our cohorts even though their results are often better than the students from the first group while plenty from the first group have got in and this result is replicated again and again in the experiences of those I talk to.

In essence the problem is this. Oxbridge give a small amount of credit in their scoring criteria for an applicant coming from a low social status background. But it doesn’t balance out the points that students will lose because they will have few extra-curricular interests and they will not have the confidence to express themselves well at interview. If you did practice Oxbridge interviews with students as I do you would find, in general, that you have ranked their performance at interview by the educational and professional attainments of their parents. The effect disappears when the interview is removed and these students are accepted by and excel at the top universities which do not interview.

Now I well know that current Tory thinking is to take these students and put them into top public schools but this thinking does not understand that these students come from a large pool of students like them and it is virtually impossible to predict which deliver incredible results at A-level until they are around 17.

These students would fly at Oxbridge. There is no question about their academic ability. They are just not getting conditional offers. It simply is the case that until they leave home children are to a certain extent prisoners to their backgrounds. This effect disappears when you put them in the variety of wider company you find at Oxbridge. They don’t need to go to a school away from home (and it is dubious to whether we want to take them out of their communities as in a valuable part of what they offer to the mix at Oxbridge is their being from those communities) they just need a chance to get into top universities. We all want them to be there. We think Cameron and co. would have been better off had they been at Oxford with, for example, the straight A*s student I helped to prepare for an interview for law at Oxford this year whose dad isn’t around an whose mum is a teaching assistant who’s just had her salary cut by 20%. She was completely brilliant. She just didn’t know how to show it yet.

The problem is not that Oxbridge interviewers are prejudiced or that they don’t care. It’s just that they don’t see the detail of the reality of the vast, vast, differences between the backgrounds of two children from the same state school – one of whom is from a professional middle class family and the other of whom is from a disadvantaged background and they need to be heavily nudged to properly understand what they are dealing with. They don’t want to believe they need to face this because they don’t want to think they are doing anything wrong.

This issues has been discussed many times at consultations until the attending representative from Oxbridge understand it. However their insight does not get disseminated to the huge group of interviewing tutors who still fail to see the difference between a state school child who has parents from a university educated professional family and one who is not.

This is not about social re-engineering or anything ideological. It is about recognising that children from backgrounds where no-one in their family has been to university and the family income is very low are being prevented from getting to Oxbridge because the system is judging them not on their academic ability but on their lack of wider interests and their inability to present themselves well at interview.

The solution is that when top universities are interviewing students from disadvantaged backgrounds they need to probe their academic ability only and they may need to ‘work with them’ a little more at interview to help them put themselves across. 

The solution has been obvious for a very long time but it has not happened. Therefore there needs to be extra pressure.