Mohammed was only 13 years old and wasn’t especially tall or powerful, yet I was terrified of him. "I’ll fucking kill you. Do you get what I mean, geezer? I’ll fucking deck you!" he screamed at me as I asked him to leave my classroom. He had hit a boy over the head and spent much of the lesson swearing. By this time, I was trembling with rage and fear, and was relieved when he finally left the room.

Soon afterwards Mohammed was excluded from the school and I gave up teaching. It was 1997 and the chaos he had caused had sapped my confidence. Because the school was not a stereotypical inner-city comprehensive, but located in a prosperous London suburb, I felt doubly deflated; I felt that I had become horribly soft. In fact, the school did have discipline problems, with a significant rump of children from troubled backgrounds, but few teachers there were trained to cope with the more challenging ones such as Mohammed. Rowdy classes became riotous, lessons became war zones.

Several years later, with my spirits refreshed and missing the buzz and excitement of the classroom, I returned to full-time teaching, quickly becoming a head of department at a school in Havering, outer London. In this new position of responsibility I had to teach several children who had been excluded from other schools or had been passed on to me by more junior teachers. By this time, I had become a more tolerant pedagogue, less obsessed with results, more adept at handling disruption. I was calmer and more consistent in my approach. Some of my pupils were potentially just as aggressive as Mohammed had been, but I was able to cope with them; I’d learned to "give and take", to negotiate, to form good relationships with difficult children.

One child, John, had been permanently excluded from another school but had settled well at my new school and ultimately succeeded in attaining eight good GCSEs. I recently spoke to John about his life now and was delighted that everything was going well for him. He had trained to be an electrician and was set, he said, on earning better wages than me. "What I liked about it in your school," he told me, "was that my mates and some of the teachers taught me how to deal with my anger. Sometimes I used to get so mad, I would just punch anyone who was around me, but then I learned to walk away from rucks. And I think that helped me concentrate more. The school stuck with me even though I was out of order sometimes. They didn’t kick me out. That counts for a lot."

Talking to John, I began to think about Mohammed, who had been jailed soon after being permanently excluded from school. I recalled how there were times when he had been keen on learning, had even shown interest in Shakespeare and reading. He had wanted to succeed, but I, and many other teachers at the school, had been preoccupied only by what was wrong with him, meting out punishments and threats that had caused a vicious downward spiral. During my investigations in trying to find out what had happened to him, I learned from another former pupil at the school that Mohammed was still "up to no good"; he had become a drug dealer and had cut some heroin with washing powder and nearly killed a user.

Had I contributed to Mohammed’s troubles? Had my old school failed him? If extra resources had been available to give him proper care and attention, would we have spared society huge amounts of money and distress in the long term?

Mohammed fitted the typical profile of an excluded child. He was male, of mixed race, had special educational needs and was in foster care. He was permanently excluded in 1997, exactly at the point when the new Labour administration swept to power promising to address the problems presented by children like him. Tony Blair’s mantra, "Education, education, education", was as much about sorting out the Moh ammeds of this world, about being "tough on the causes of crime", as it was about improving results.

In spite of the government’s best efforts to massage the figures, exclusion rates have remained more or less steady for a decade; on average, roughly 9,000 children or more are permanently excluded from school every year and nearly 400,000 children given "fixed-term" exclusions, according to the Department for Children, Schools and Families. Eighty per cent of them are boys. Government figures show that Roma children are three and a half times more likely to be excluded than other children, and those from black or mixed ethnic backgrounds are twice as likely to be excluded as whites. Children in care are eight times as likely to be excluded, and those with special educational needs are three times more likely to be ordered to leave their school.

After 11 years of a Labour government, school exclusions continue to affect the underprivileged.

In 2007, as many as 140,000 pupils who were excluded for short periods from school were eligible for free meals, accounting for a third of such exclusions, even though these children make up only 12 per cent of the school population. But if schools were better equipped and staff better trained to deal with the persistent disruption exhibited by children from dysfunctional and deprived households, would exclusion rates be drastically reduced?

Meanwhile, society as a whole is paying an increasing cost. Significant research by the charity New Philanthropy Capital, which offers advice on giving, reveals that the average excluded child costs society more than £63,851 a year. This figure includes the future lost earnings of the child resulting from poor qualifications, and also costs to society in terms of crime, health and social services. In total, this amounts to £650m a year. This is probably a gross underestimate, since many excluded children are not accounted for in the figures.

The human cost of failing to deal with the problem is incalculable: carrying a knife is the most common offence among children excluded from school, and 50 per cent of men in prison were excluded. "Research shows that at the root of school exclusions, and much crime, is the inability of young people to communicate properly," says Lord Ramsbotham, former chief inspector of prisons. "If we addressed these problems in the classroom, many of our problems with antisocial behaviour would disappear.

"At the moment, what happens is that these young people, having been alienated from their families at an early age, are then excluded from school and turn to crime: drug-taking and dealing, knife crime and, in extreme but increasing cases, murder. Research shows that while poor parenting and low socio-economic status are major factors, school exclusion plays a significant environmental role in helping shape the criminals of tomorrow. The government needs to appoint a minister for inclusion to begin to address these issues."

Ofsted, in its report, Reducing Exclusions of Black Pupils from Secondary Schools: Examples of Good Practice, identified three interrelated features that significantly reduce exclusions: "Respect for the individual in school and a systematic, caring and consistent approach to behaviour and personal development, the courage and willingness to discuss difficult issues, a focus on helping pupils to take more control of their lives by providing them with strategies to communicate well and look after each other."

I know from my own experience that good mentoring really helps; the best schools allocate both "academic" and "professional" mentors to troubled pupils. The academic mentor will set clear, achievable targets twice a week which are then closely monitored, while professional mentors, usually drawn from the world of work, will show pupils opportunities beyond the classroom.

Frequently, these pupils have tailor-made numeracy and literacy lessons, and work in small groups with tutors to engage with the curriculum. Furthermore, pupils with particular psychological needs will have relevant lessons such as "anger management" classes or counselling sessions. While this may sound expensive, it needn’t be: some schools have met the costs easily by getting rid of expensive management posts and reallocating the resources into buying in mentors and academic tutors. The alternative of the pupil referral unit is far more expensive; with staff ratios of one teacher to six pupils, the units mean thousands more pounds are spent per pupil than in a mainstream school.

Studies show that targeted early intervention can significantly reduce the problems caused by school exclusions. Take the case of Abby, a child who at the age of 12 was in foster care and regularly in trouble at school in south London. She was confrontational; she fought with other children and abused teachers. But she was also on the autistic spectrum, a condition that was not dealt with properly at school. Frequently, she would misinterpret the teachers’ instructions, literally pulling her socks up in response to this metaphorical order. A series of fights and slanging matches with teachers led to her being permanently excluded before she could take any GCSEs. Once out of school, she quickly turned to petty crime such as shoplifting. Fortunately, her case was taken up by the National Teaching and Advisory Service (NT&AS), and some trained professionals were assigned to her who would supervise both her academic and social needs. Much to the astonishment of her former school, she attained seven GCSEs and is now at college.

"The link between youth offending and educational failure has of course been known about for years. But successive governments have failed to do much about it, although this government has undoubtedly done more than the others," says Tim Walker, the chief executive at NT&AS. "Organisations like mine can make a big difference if we intervene at the right point; we can put troubled children on the path to success."

One small but significant step to making exclusions a more constructive experience would be to grant children the right to appeal against their own exclusions, being assigned a trained "advocate" to represent them. A scheme like this has already been piloted in ten boroughs between 2005 and 2008 by Save the Children with its three-year EAR to Listen project, which gave excluded children an independent advocate to speak for them at exclusion panels and liaise between home and school generally.

That the project had an 80 per cent success rate in supporting children and young people to remain, re-enter and re- engage with education, but there is little political impetus behind spreading its good practice throughout the country. "The government has ratified the UN Convention on the Rights of the Child, which gives children the right to be heard and taken seriously in all matters affecting them, but we are nowhere near granting this to our excluded pupils," says Tom Burke, a spokesperson for the Children’s Rights Alliance for England. Since September 2007, schools have been obliged by law to promote pupil well-being. "We would hope that new guidance on the duty, which the government will require schools to implement next year, will add further weight to exclusion panels to considering a child’s rights when making exclusion decisions."

At the root of the social chaos caused by exclusions is a chronic lack of consistency. Some schools are eager to exclude disruptive pupils, while others are extremely reluctant to do so, even for serious offences. Schools anxious not to have their figures sullied by too many exclusions are choosing to operate a system of "internal" exclusions – locking pupils up in rooms with no windows, keeping them away from lessons in separate buildings, quietly telling them not to come to school at opportune moments such as when the inspectors are there. But this only creates worse problems for society in the long run; alienated and uneducated children, who are neglected by their schools and families, are left unsupervised to cause havoc.

It is only by being "consistently caring" towards these vulnerable children that we will clean up the mess. But no one in any of the three main political parties has had the courage to argue publicly that we should provide advocates for disruptive children on exclusion panels, insist on their having the right to appeal against their own exclusions, or that we should keep them in mainstream classes, if at all possible. Projects such as Save the Children’s EAR to Listen, which provide excluded children with the proper support to stay in school, have been proven to be by far the cheapest and most effective way of solving the problem. Save the Children estimates that providing advocates for our most vulnerable children should cost no more than £8.5m, compared with the £650m that taxpayers are currently paying to cover all the harm exclusions cause.

I spoke to Michael Gove, shadow secretary of state for children, schools and families, about the Tories’ attitudes towards exclusions. The Conservatives have pledged to scrap independent appeal panels, the only form of outside scrutiny that schools currently have when they exclude pupils, and Gove confirmed that the party aims to make this a manifesto commitment. "We want to introduce a number of measures within schools that will stop exclusions, such as early intervention strategies that will mean pupils will be dealt with effectively before the drastic step of excluding them is taken," he told me.

"We want to give headteachers the power to exclude pupils without being overruled by outside bodies so that they are secure that their authority won’t be challenged or, as is often the case at the moment, undermined."

How would he ensure consistency on exclusions? Home-school contracts, he said, would be clear about what behaviour was acceptable and what was not. This is not a new idea: Labour has attempted and failed to make home-school contracts work. In fact, such contracts are simply a list of rules for parents that have been drawn up by the school, and these can vary from place to place.

Gove was clear that the Tories would not accept the UN Convention on the Rights of the Child and its insistence on a child’s right to appeal his or her exclusion, but would allow individual schools to adopt this procedure if they felt it appropriate. This would give schools the chance to experiment in the ways in which they grapple with the issue of exclusion. "Our priority will be to stop exclusions in the first place," he said. "We believe our reading programme in primary school should have a big impact in this sense; the overwhelming majority of excluded children can’t read properly. If we ensure that all children can read by the time they are seven, we will have cut down greatly upon the causes of exclusion.

"If we stop the causes of exclusions, we won’t be kicking out kids on to the streets to cause mayhem. Our plans for improving pupil referral units will mean that all difficult children will be catered for in a supportive learning environment."

Overall, his plans seemed contradictory; and while he is sincere, I and many people in the system do not share his faith in the judgement of headteachers, on which so many of his plans rely. The Tories’ plans will increase the inconsistency in the processes by which children are excluded, giving them no redress whatsoever. They will create an even angrier generation of rejects than we have now. His plans for referral units will be costly, without ever getting to the root of the problem.

Labour, anxious about being labelled "soft", has not even attempted to argue with Gove’s proposals. The party prefers, instead, to sneak in guidance and legislation that bolsters children’s rights in a piecemeal and inchoate fashion. The media is partly responsible for this: for all the column inches devoted to antisocial behaviour and crime perpetrated by children, there is seldom any serious attempt to look at some of the mundane root causes. Pointing out the inconsistency in schools’ approaches towards exclusions and the need for proper, uniform disciplinary procedures, doesn’t make good copy. Screaming about the need to expel thugs and yobs from school does. As a result, the public is never properly informed about the issues and the debate remains banal.

There are straightforward, successful and cheap measures that could drastically reduce school exclusions tomorrow. But the political and educational will to implement them doesn’t exist. And so we are condemning our society to an ever rising tide of lawlessness.

Francis Gilbert’s "Parent Power: the Complete Guide to Getting the Best Education for Your Child" is published by Piatkus (£9.99) http://www.francisgilbert.co.uk

Exclusion by numbers

30% of permanent exclusions are for persistent disruptive behaviour (2007 figures)

27% are for physical assaults on staff

17% are for assaults on pupils

11% are for verbal abuse against an adult

5% are for verbal abuse against a pupil

3% for bullying, racist abuse and damage

2% are for sexual misconduct

8,680: number of permanent exclusions from primary, secondary and special schools in 2006/2007

363,270: fixed-term exclusions from state secondary schools

45,730: fixed-term exclusions from primary schools

20: number of times more likely that excluded children will end up in prison, compared to the general population

Source: Department for Children, Schools and Families

What is exclusion?

Source: Department for Children, Schools and Families

Headteachers have the right to remove a child from school for serious misbehaviour

"Fixed-term exclusion" is when the child is excluded temporarily (from one half-day to a maximum of 45 days in one school year). The school sets work for the period, which the child’s guardian is expected to supervise

"Permanent exclusion" is when the child is ordered to leave the school permanently. The child may then have a "managed move" to another school, or go to a "pupil referral unit", to be taught in very small classes. Some may drop out of education altogether

Portraits by Natalie Pecht