The letter below has just appeared in my inbox. It answers some of the questions I had been pondering, and a few I hadn’t thought of.
For ease of reference other relevant reading on this blog can be found here: Experts in Family Courts, Experts Upon Experts, and Experts, the Press and a Sloppy Approach to Evidence Based Reporting.
Mrs Westmacott et al
By email.
17 April 2012
Dear Sirs
Professor Ireland’s Research
Thank you for your letter, sent on behalf of a number of your colleagues, dated 8th April.
You ask several questions relating to Professor Ireland’s study, recently published by UCLAN. In responding, I follow the order set out in your letter.
How was the research instigated and was a process of tendering followed?
The research proposal was first discussed in the Experts Committee of the Family Justice Council in late 2008 and early 2009 and Professor Ireland was asked to submit a proposal to the Projects Committee of the Council. The issue first arose because of anecdotal evidence suggesting a level of concern about the quality of some psychologists reports used in expert evidence in family proceedings. Professor Ireland submitted a proposal and it was considered by the Projects Committee and then approved by the Executive Committee of the FJC. The proposal then went through the procurement and Data Access Panel processes required by the Ministry of Justice and HMCTS. Professor Ireland has indicated that the research proposal was given ethical approval by UCLAN.
How was this research team selected from any competitors?
Professor Ireland’s proposal met the criteria for a single tender.
What is the nature of FJC involvement with and support for this study?
The Council part funded the study which was submitted to the FJC, as agreed, for review and with an option to publish.
What is the FJC’s view of the quality of the study?
Four academic and professional reviewers commented on various drafts of the report, on behalf of the FJC, and all took the view that the report would require some amendment before the Council should publish it. Professor Ireland made a number of amendments but, ultimately, the FJC declined to exercise the option to publish the work. However, as only part funders the work was not the intellectual property of the FJC, but of UCLAN which has promulgated the material on their own website.
Does the FJC endorse the results of the study?
The FJC understands that the document published by UCLAN is a summary of the findings of the study. The Council feels the study would benefit from further academic assessment and review. To this end, the Council has invited the authors to submit a full write up of the paper for publication in a peer reviewed journal, so that the methodology, findings and conclusions can be appropriately debated in the academic and professional community, but understands that this has not yet been done. The FJC considers that the most robust evidence undergoes a process of academic peer review and achieves publication in appropriate journals. Any piece of evidence must be considered and evaluated, but naturally less weight can be given to material that has not undergone such a process. The document so far published on the UCLAN website does not currently meet the usually recognised criteria for peer reviewed publication.
Do you share any of our concerns about the execution, write up and publication of this research? The FJC has already commented on the limited methodology of the study and feels it important to take a balanced view of the aspects of the study that contain value, which were emphasised in the Council’s press release.
Would you reassure us that the Family Justice Council is neither recognising nor endorsing the “Ireland Standard”? The FJC has neither promulgated, nor recommended, the use of ‘Ireland criteria’ for expert witnesses. The FJC has been developing consensus based standards for experts and is due to publish these for consultation soon.
How do the FJC intend to use the research?
The FJC is working to improve quality, supply and best practice in the use of expert witness reports in all specialisms. The study provides a reminder that appropriate quality and validity measures must always be applied by the court to expert witness expertise, and reports, to ensure fitness for purpose in the family courts. The FJC has, from the outset, considered the study as a first step in a process intended to promote debate on how to improve the quality assurance of expert evidence used in family proceedings. The Council will be considering a number of pieces of work relating to improving supply, quality and best practice in the use of expert witness evidence at its next meeting on 29th April.
If anything in this letter is unclear please feel free to contact me on the telephone number or email address above.
Yours faithfully,
Alex Clark
Secretary to the Family Justice Council
In other words Camilla Cavendish is right and the established experts are seeking to ensure they are only assessed and criticised by one of their own, rather than someone who has the benefit of being objective.
But the established experts were assessed and criticised by other experts as a result of the family proceedings, with the result that the established medical orthodoxy was undermined or at any rate significantly challenged. How does that prove your point?
Mrs Westmacott et al object to Professor Ireland because she is not a child psychologist/psychiatrist and therefore ‘not one of us’ is the point that Camilla Cavendish made, based on the abusive correspondence received by The Times.
The Ireland Report looked at submitted reports, not at the cross-examinations carried out in the courts, her conclusions were that many of the reports were of poor quality and that it was not always clear that the authors of those reports were professionally qualified or members of recognised professional bodies. In addition, many of them seemed to have no active practice and to work solely as expert witnesses – and making a great deal of money from doing so.
The concern must be that if those expert witnesses are not in fact qualified, and/or don’t practice outside the courts, and if the judge and lawyers present are not aware of that then at best the court’s time is being wasted and at worst the courts may come to incorrect conclusions because of misleading evidence if the cross-examination that takes place does not reveal problems with the expert giving evidence.
Gladiatrix,
Apologies my response to your previous comment on this post was mistaken – due to the way my system shows comments I thought this was a further comment about the recent High Court case (hence my reference to the medical evidence etc).
One of the criticisms of the Ireland report is that although some of the experts may have appeared to be unqualified based on the limited information seen by the researchers, further enquiry may well have established that some of the experts were appropriately qualified. It is common for full CVs to be circulated prior to instruction but not incorporated into the bundle, and a summary CV to be provided within the body of the report, which practice would have given a distorted picture based on the research methodology applied in the Ireland report.
However, it is a concern if there are expert witnesses who aren’t qualified or who are not in clinical practice and I think that this information should be provided in every case prior to instruction AND included in the bundle in order to assist in cross examination.
It is unfair to single out Ms westmacott for criticism. The FJC letter was addressed to 12 HPC/bps registered psychologists, some of whom do very little EW practice, who expressed concern about the ethics of the way the research was conducted, the interpretation of data, the dissemination via Daily Mail and the failure to ensure miscarriages of justice had not occurred. All signatories recognise the need for robust research to evaluate the quality of experts and are supportive of FJC taking action to ensure high quality evidence is available in family proceedings.
Undertaking research based on court files is fraught with difficulty because of the incidence of missing documents. Any paper-based study (e.g those by Brophy or Masson) acknowledges this. It is not clear from Prof Ireland’s report whether CVs were missing, incomplete or inadequate. Assumptions have been made that it is the latter.
[…] ‘Evaluating Expert Witness Psychological Reports: Exploring Quality’ was a study by Professor Ireland in 2012 and funded partly by the Family Justice Council. However, they later distanced themselves from the conclusions of this study over concerns about its methodology. See this post on Pink Tape. […]