Assessing students up to age 14 - A better future. Part 3 - Integrating formative (APP) and summative (SATS) assessment

This blog is subject to these protocols

The biggest efficiency and quality improvements will come when school tracking systems are developed which integrate formative assessment (APP) and summative assessment (SATS).

At present many schools spend a great deal of time recording the details of students progress but they usually only share this data among staff.  Students only receive credit for the grades they achieve in their SATS and other exams and these grades/levels reveal little except whether the students has learnt much this year or not.

However it is now been clearly demonstrated that the landscape of what is now possible has fundamentally changed.  Ian Lynch has a company called The Learning Machine which is licenced by Ofqual to issue criteria based qualifications which they call Ingots.

The Learning Machine have developed qualification which are specified by the criteria students have to achieve.  They have created tracking systems which must be used to record progress.  Schools are responsible for creating their evidence trails and if they subscribe to the system they can then issue as many qualifications as they are able to justify.  They are audited by The Learning Machine once a year.   So far they have specilised in developing qualifications to support emerging ICT systems such as Open Office and to assess and give formal credit for progress to students with learning difficulties.

This is precisely the kind of system which could positively transform assessment for students up to the age of 14.

However this model of accreditation needs to be developed to include:
- online testing and subsequent automatic accreditation for criteria which are easy to test (and ideally to online teachings systems which support students in filling gaps identified) so that teachers can focus on observing those aspects of student progress which are not easy to test.
- links to whole school registration systems to ensure it is easy for teachers to populate fields.
- subsequent links to behavior tracking systems and other student, staff or school tracking and development systems.
- shared access and use for teachers, student and parents.
- transferability between schools, especially from primary schools to secondary schools.
- links to the central database so that detailed data can be used to assess national and school progress by government authorities.

In subsequent blogs on this theme I will explore these points sharing insights into how they can be effectively addressed and the benefits their addressed will bring.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 2 - protocols for this series of blogs

As this series of blogs progresses I will refer to individuals and companies I have spoken to who have brought to life the individual components of future systems which I believe will transform education when they are combined.

I am very grateful to all who have taken the time to meet me or chat to me on the phone, in forums or by email.

However I am aware that by following this strategy it is inevitable that I will miss individuals and companies with similar or different relevant skills and products to offer.  I would like to warmly invite anyone who would like to gain exposure for their relevant ideas, services or products to add comments to the relevant blog which either raise relevant points or describe what they have to offer.

If you would like advice or guidance as to where to place your comments or would just like to communicate with me about any of these issues please do get in touch through linkedin or by commenting on this blog and giving your contact details.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Assessing students up to age 14 - A better future. Part 1 - key components of the way forward

It is of great benefit for students, teachers and parents to have detailed information regarding precisely what each student can do and where their knowledge gaps are.

However:
SATS, as a measure of school accountability are too narrow and skew what schools actually do away from best practice.

and

SATS levels do not provide much useful information.  It would be much better if detailed information regarding precisely what at student is doing well and what they are struggling with were shared.


Key Components of the Way Forward:
A coherent solution will require two key changes in practice.

1. The integration of formative and summative assessment.
In other words schools systems by which they track the detail of students' progress (APP systems) should lead directly to accreditation and SATS should be abolished.

2. Formative assessment must be made coherent by using different types of assessment for those targets which can be easily tested and those which cannot.
Most current systems of APP are not currently suitable for direct accreditation.  We need to specify and track targets which can be easily tested using different processes to those used to specify and track targets which cannot be easily tested.



In this next series of blogs I will explore how these changes can be achieved and some of the many benefits they will bring to students' education.

Assessing students up to the age of 14 – A better future  QUICK LINKS

Part 1: key components of the way forward

Part 2: protocols for this series of blogs

Part 3: integrating formative (APP) and summative (SATS) assessment

Part 4: being allowed to test what is testable

Part 5: more about online testing

Part 6: central tracking of progress

Part 7: supporting diversity

Part 8: student behaviour management

Part 9: bridging the primary/secondary divide

Part 10: at what age should this type of tracking begin? (and tracking inputs)

Les Ebdon, Vince Cable and Oxbridge Access

I’ve been working hard with students from tough backgrounds on Oxbridge access for the last 12 years. Every year we’ve prepared groups of state school students for interviews. These groups have always contained middle class state school students who have a professional parent and who have plenty of extra curricular activities to talk about and the personal confidence to talk about them. They have also contained students from disadvantaged backgrounds. No child from the latter group has ever been offered a place at Oxbridge from our cohorts even though their results are often better than the students from the first group while plenty from the first group have got in and this result is replicated again and again in the experiences of those I talk to.

In essence the problem is this. Oxbridge give a small amount of credit in their scoring criteria for an applicant coming from a low social status background. But it doesn’t balance out the points that students will lose because they will have few extra-curricular interests and they will not have the confidence to express themselves well at interview. If you did practice Oxbridge interviews with students as I do you would find, in general, that you have ranked their performance at interview by the educational and professional attainments of their parents. The effect disappears when the interview is removed and these students are accepted by and excel at the top universities which do not interview.

Now I well know that current Tory thinking is to take these students and put them into top public schools but this thinking does not understand that these students come from a large pool of students like them and it is virtually impossible to predict which deliver incredible results at A-level until they are around 17.

These students would fly at Oxbridge. There is no question about their academic ability. They are just not getting conditional offers. It simply is the case that until they leave home children are to a certain extent prisoners to their backgrounds. This effect disappears when you put them in the variety of wider company you find at Oxbridge. They don’t need to go to a school away from home (and it is dubious to whether we want to take them out of their communities as in a valuable part of what they offer to the mix at Oxbridge is their being from those communities) they just need a chance to get into top universities. We all want them to be there. We think Cameron and co. would have been better off had they been at Oxford with, for example, the straight A*s student I helped to prepare for an interview for law at Oxford this year whose dad isn’t around an whose mum is a teaching assistant who’s just had her salary cut by 20%. She was completely brilliant. She just didn’t know how to show it yet.

The problem is not that Oxbridge interviewers are prejudiced or that they don’t care. It’s just that they don’t see the detail of the reality of the vast, vast, differences between the backgrounds of two children from the same state school – one of whom is from a professional middle class family and the other of whom is from a disadvantaged background and they need to be heavily nudged to properly understand what they are dealing with. They don’t want to believe they need to face this because they don’t want to think they are doing anything wrong.

This issues has been discussed many times at consultations until the attending representative from Oxbridge understand it. However their insight does not get disseminated to the huge group of interviewing tutors who still fail to see the difference between a state school child who has parents from a university educated professional family and one who is not.

This is not about social re-engineering or anything ideological. It is about recognising that children from backgrounds where no-one in their family has been to university and the family income is very low are being prevented from getting to Oxbridge because the system is judging them not on their academic ability but on their lack of wider interests and their inability to present themselves well at interview.

The solution is that when top universities are interviewing students from disadvantaged backgrounds they need to probe their academic ability only and they may need to ‘work with them’ a little more at interview to help them put themselves across. 

The solution has been obvious for a very long time but it has not happened. Therefore there needs to be extra pressure.

Notes to Tweet to Michael Gove.

In response to the invitation by the Education Select Committee to Tweet a question to Michael Gove, I have Tweeted the following (as @Cyberrhetoric).

Why is the SoS not fulfilling his duty to ensure Ofsted comply with the law it become obliged to on 10/11/09? #AskGove

These notes are provided to ensure the Secretary of State for Education can provide an appropriate answer to the question.

Legal Notes:

The Order to the Legislative and Regulatory Reform Act (2006) which came into law in 1st October 2009 (part 1A (17)) obliges Ofsted to the law created by the Legislative and Regulatory Reform Act (2006).

This Law prescribes that “regulatory activities should be carried out in a way which is transparent, accountable, proportionate and consistent” and that “regulatory activities should be targeted only at cases in which action is needed”.
(http://www.legislation.gov.uk/ukpga/2006/51/section/21)

This Law was created to force regulators to implement the principles established by the Hampton Review (2005).
(http://webarchive.nationalarchives.gov.uk/+/http://www.bis.gov.uk/policies/better-regulation/improving-regulatory-delivery/assessing-our-regulatory-system)

Explanation of relevant aspects of the meaning of the law in practice:

In fulfilling their duties to protect against unacceptable practice, drive quality improvement and report to government on the state of the industries they regulate and the services provided, other UK regulators obliged to the same law have worked to ensure that:

1) they seek to establish and police the boundary between unacceptable and acceptable practice and to ensure that organisations are engaged in process of effective internal communication, detailed reporting or both a qualitative and an quantitative nature and professional cycles of quality improvement but they do not pre-define categories of quality as this has been show to suppress innovation, healthy diversity and professional freedom.

2) there is an expectation that when inspectors engage with organisations, inspectors will come to understand new practice with which they were not previously familiar so that they can properly report to government regarding new best practice so that they can accurately report to government regarding the quality of products and services provided.

3) regulators constantly strive to ensure that only inspectors who are highly respected professionals are used so that the purposes of inspection can be paramount at all times and so that inspectors who focus instead on process of inspection without being able to subordinate those processes to intelligent purpose focused discussion and are therefore likely to invoke disproportionate consequences are not employed.

Comments from the author of the question:

Having researched this issue extensively I have found significant evidence to suggest that Ofsted are seriously misrepresenting their legal obligations.
(http://mathseducationandallthat.blogspot.com/2011/08/ofsted-part-2-journey-to-heart-of.html)

Certainly it would not be in the personal interests of many currently in senior positions at Ofsted to see it enforced.

The actions the Secretary of State for Education has taken to reform Ofsted do not seem to understand this law. I therefore wonder if perhaps the Secretary of State for Education is unaware of his duties to enforce this law which will substantially improve the degree of appropriate professional freedom all schools are allowed?

MPs Debate the Use of Calculators

Yesterday's debate between Elizabeth Truss MP, Justin Thomlinson MP (Chair of the Parliamentary Committee for Financial Education) and Nick Gibb (Minister of State for Schools) is available here:
http://www.parliamentlive.tv/Main/Player.aspx?meetingId=9518&st=16:42:20

Here are some of the key insights they lacked:
1. Removing calculators from students will NOT create improvements in mental maths.  There is an intermediate step missing and this is the nurturing of mathematics teaching which uses the core structure and axioms of mathematics and does not teach algorithms and quotable facts without deep structural understanding.  Removing calculators could lead to more teaching of algorithms without structure which would compound current problems.
References: Elementary Mathematics by LiPing Ma.
http://mathseducationandallthat.blogspot.com/2011/05/how-do-chinese-do-it-introduction.html and subsequent posts
http://osiriseducational.co.uk/maths/transforming-maths-2010.html
In essence overuse of calculators (where it occurs) is a symptom of the problem more than being a cause of it.

2. Developing fluency in tables has very limited benefit unless students do it in the context of developing scaling and other structurally sound semiotic structures for multiplication.
http://mathseducationandallthat.blogspot.com/2011_04_01_archive.html

3. Both splitting and chunking are essential primitive structures for division.  It's also of great benefit for students to be aware of the scaling structure for division.
(Think - if you do 264 / 2 you use splitting - pulling 264 apart into two equal parts.  If you do 39 /13 you will probably chunk - thinking how many 13s are there in 39 rather than splitting 39 into 13 equal parts.  Different real life situations prompt different structural uses of division).

4. The issue of questions which should have been on non-calculator SATS papers appearing on calculator SATS papers had been identified by QCDA and they were in the process of addressing it by replacing those questions with appropriate calculator questions.  Appropriate uses of calculators include the use of calculators for questions where the results on the calculator demand interpretations and understanding that students could not access with non-calculator techniques and to assist the subordination of complex calculations in extended multi-stage functions which students could not otherwise access.
(contact me for further justification of this claim and more insight).

5. There are plenty of teenagers who are not experts on an appropriate variety of functions of smartphones and internet searches.  Yes they use smartphone but many of them only use them for a limited range of functions and they need the opportunity to reflect on the results internet searches generate to use them wisely in academic analysis.
(widely understood in education).

6. Yes it is important to focus on core skills but we have been doing this.  Some of the National Strategy has worked extremely well, other aspects have been seriously flawed.  Without insight into what has worked and what has not worked and why you cannot create an effective policy which will improve things.


Post Script:
At the beginning of the discussion Ms Truss asks why the issues of calculators in school has not been debated in parliament for so long.
The answer is because the quality of insight explored was far below that which would have been possible had the ministers involved attended a relevant consultation at ACME or WEF, which is what has been happening instead under the last government.

Cyberrhetoric

I now devote part of my time to developing the academic discipline of analysing the processes and outputs of mass online discussion (that's conversations in forums and the comments threads to blogs, news articles and so on).

My passion for this subject started in maths education - see for example this article:
http://www.scribd.com/doc/55142332/Exploring-Discussion-Forums
And I am delighted to have been asked to manage and moderate the inspirational group:
Math, Math Education, Math Culture on linkedin.com, a role which I love and I would warmly invite any enthusiast of maths education to join this warm and vibrant group.

But my interest in forums has now grown beyond education and so I've started a separate blog specifically for it.  The first post in my new blog (which I shall run in parallel with this one) contains my notes on cyberrhetoric - which is the ways in which participants in forums can post to help conversations which are stuck or abusive become productive and respectful -from the session I ran at the Mozilla Festival in Grenwich on Sunday.  As well as developing this study further, I will also post on the analysis of the ways in which mass online discussion can enhance democracy and generate intellectual capital and I hope to link to other articles about mass online discussion.  Here's the link:
http://cyberrhetoricbyrebeccahanson.blogspot.com/

I shall keep this blog for maths education and related issues in education.