The letter below has just appeared in my inbox. It answers some of the questions I had been pondering, and a few I hadn't thought of.
For ease of reference other relevant reading on this blog can be found here: Experts in Family Courts, Experts Upon Experts, and Experts, the Press and a Sloppy Approach to Evidence Based Reporting.
Mrs Westmacott et al
17 April 2012
Professor Ireland’s Research
Thank you for your letter, sent on behalf of a number of your colleagues, dated 8th April.
You ask several questions relating to Professor Ireland’s study, recently published by UCLAN. In responding, I follow the order set out in your letter.
How was the research instigated and was a process of tendering followed?
The research proposal was first discussed in the Experts Committee of the Family Justice Council in late 2008 and early 2009 and Professor Ireland was asked to submit a proposal to the Projects Committee of the Council. The issue first arose because of anecdotal evidence suggesting a level of concern about the quality of some psychologists reports used in expert evidence in family proceedings. Professor Ireland submitted a proposal and it was considered by the Projects Committee and then approved by the Executive Committee of the FJC. The proposal then went through the procurement and Data Access Panel processes required by the Ministry of Justice and HMCTS. Professor Ireland has indicated that the research proposal was given ethical approval by UCLAN.
How was this research team selected from any competitors?
Professor Ireland’s proposal met the criteria for a single tender.
What is the nature of FJC involvement with and support for this study?
The Council part funded the study which was submitted to the FJC, as agreed, for review and with an option to publish.
What is the FJC's view of the quality of the study?
Four academic and professional reviewers commented on various drafts of the report, on behalf of the FJC, and all took the view that the report would require some amendment before the Council should publish it. Professor Ireland made a number of amendments but, ultimately, the FJC declined to exercise the option to publish the work. However, as only part funders the work was not the intellectual property of the FJC, but of UCLAN which has promulgated the material on their own website.
Does the FJC endorse the results of the study?
The FJC understands that the document published by UCLAN is a summary of the findings of the study. The Council feels the study would benefit from further academic assessment and review. To this end, the Council has invited the authors to submit a full write up of the paper for publication in a peer reviewed journal, so that the methodology, findings and conclusions can be appropriately debated in the academic and professional community, but understands that this has not yet been done. The FJC considers that the most robust evidence undergoes a process of academic peer review and achieves publication in appropriate journals. Any piece of evidence must be considered and evaluated, but naturally less weight can be given to material that has not undergone such a process. The document so far published on the UCLAN website does not currently meet the usually recognised criteria for peer reviewed publication.
Do you share any of our concerns about the execution, write up and publication of this research? The FJC has already commented on the limited methodology of the study and feels it important to take a balanced view of the aspects of the study that contain value, which were emphasised in the Council’s press release.
Would you reassure us that the Family Justice Council is neither recognising nor endorsing the “Ireland Standard”? The FJC has neither promulgated, nor recommended, the use of ‘Ireland criteria’ for expert witnesses. The FJC has been developing consensus based standards for experts and is due to publish these for consultation soon.
How do the FJC intend to use the research?
The FJC is working to improve quality, supply and best practice in the use of expert witness reports in all specialisms. The study provides a reminder that appropriate quality and validity measures must always be applied by the court to expert witness expertise, and reports, to ensure fitness for purpose in the family courts. The FJC has, from the outset, considered the study as a first step in a process intended to promote debate on how to improve the quality assurance of expert evidence used in family proceedings. The Council will be considering a number of pieces of work relating to improving supply, quality and best practice in the use of expert witness evidence at its next meeting on 29th April.
If anything in this letter is unclear please feel free to contact me on the telephone number or email address above.
Secretary to the Family Justice Council