|ACUPUNCTURE AND MOXIBUSTION
|Year : 2016 | Volume
| Issue : 1 | Page : 74-79
The importance of evidence for the integration of traditional and complementary medicine into Western healthcare?
Allied Health Sciences, School of Health & Social Care, London South Bank University, London, UK
|Date of Submission||01-Sep-2015|
|Date of Acceptance||01-Mar-2016|
|Date of Web Publication||7-Sep-2020|
Professor of Traditional Chinese Medicine and Integrated Health, School of Health & Social Care, London South Bank University, London
Source of Support: None, Conflict of Interest: None
Policy makers should consider improving access to treatments normally considered outside of healthcare in the west, particularly given the recent WHO strategy. However, research on integrating traditional and complementary medicines (T&CM) into conventional health care is lacking. T& CM approaches, may provide a cost effective option given the increase in chronic diseases with evidence of effectiveness of T& CM being key if such approaches are to be offered as part of integrative patient care, in most countries, an integrated healthcare approach is unavailable to patients. Evidence based clinical guidelines on T& CM should influence care provision. However evidence that is available may not even be cited. Poor use of evidence may possibly be a reflection of lack of knowledge and/or biases of guideline developers but additionally stakeholders vary in their perceived need for evidence to implement health policy change. Journal editors should focus on publishing high quality manuscripts, with clear rationales, design and reporting frameworks. Researchers should consider carrying out careful feasibility studies prior to conducting trials taking into account the UK Medical Research Council’s framework on complex interventions. Examples of feasibility studies highlighting their importance for trial design are given in this paper. If preparation for more definitive studies is conducted this will in turn improve the quality of research being designed, executed and published and subsequently encourage the inclusion of evidence by guideline developers. Robust evidence is necessary in order to inform governments’ health policies providing appropriate T&CM interventions which can ultimately help to provide best patient care.
Keywords: Integrated medicine, traditional and complementary medicine, clinical guidelines, evidence based medicine, Delphi Technique, acupuncture
|How to cite this article:|
Robinson N. The importance of evidence for the integration of traditional and complementary medicine into Western healthcare?. World J Tradit Chin Med 2016;2:74-9
|How to cite this URL:|
Robinson N. The importance of evidence for the integration of traditional and complementary medicine into Western healthcare?. World J Tradit Chin Med [serial online] 2016 [cited 2022 Jul 7];2:74-9. Available from: https://www.wjtcm.net/text.asp?2016/2/1/74/294427
| The Need for an Integrated Approach to Healthcare|| |
The recent WHO Traditional Medicine strategy 2014-2023 has stated clear aims for harnessing the use of traditional medicine to improve population health. [http://www.who.int/ medicines/publications/traditional/trm_strategy14_23/en/]. Data from the WHO has reported that out of 129 countries, 80% recognized the use of acupuncture. Integrating relevant T& CM into health care systems by building on WHO’s strategy will need to focus on developing the knowledge/ evidence base, to ensure safe practice through education, training and promote universal health coverage to improve health outcomes,. As the population grows older, with complex conditions, multi-morbidities, and an increasing use of polypharmacy, Western medicine (WM) is facing the challenge of rising healthcare demands and associated increasing costs. Chinese medicine, acupuncture, in particular may offer a low cost, intervention with the potential for managing chronic disease and has a good safety profile [3,4].
However an integrated approach into healthcare is rarely available to populations in the West. If the best care is to be offered and provided to patients, interventions which have an appropriate evidence base should be both accessible and available. The situation is not helped by the lack of a standard definition for integrated medicine as researchers & clinical practitioners from different countries interpret the term differently and attach different meanings. In addition there are different regulations and policies between countries which limit the scope of practice which affects how research is undertaken. In addition integrated medicine has been defined in various ways and may be offered in combination with Western medicine or through Western medicine referral to a complementary/traditional practitioner. In much of Europe, clinicians have to be medically qualified in order to practice acupuncture and there are differences in legal status, regulation and practice. This was highlighted in a European Union project (CAMBRELLA) http://cordis.europa.eu/result/rcn/57185_en.html.
| The Importance of Evidence|| |
There needs to be sufficient evidence governments are able to inform decision-making for health care policy. Even if evidence is available, the problem is to ensure it is used appropriately to inform policy so that it can support clinical practice guideline development and inform the provision of best patient care. On Jan 8th 2014, the Lancet published a special collection of articles on ‘Research: increasing value, reducing waste’? [http://www.thelancet.com/series/research]. This publication highlighted the huge number of articles published which evaluated the same research question. Because of the lack of standards and guidelines for interventions for specific conditions, this can result in less credibility for papers that are published. The publication also highlighted the well-known negative trial publication bias as well as a range of other issues associated with the quality of research. It is also equally important to publish well designed negative trials as well as positive trials and journal editors need to support such submissions by authors to ensure that bias in publication is reduced.
There are various barriers to providing Chinese medicine as an integrative approach in the West which include; diversity in education, training, knowledge, bias, regulation, access, the evidence base and availability of Clinical Practice Guidelines. Publications of systematic reviews and meta-analyses on the use of acupuncture for specific conditions has been increasing, with the USA initially leading the field, followed by China, still the integration of acupuncture into WM clinical practice has been slow. In addition, primary research on the true effectiveness of pragmatically integrating acupuncture into a package of care has been lacking. This is in part due to problems in deciding on an agreed definition for IM which would facilitate such reviews and also deciding on the various components of IM and this makes it difficult to conduct and to report IM research. Issues are the complex nature of IM, the lack of a standard definition, government policies, methodological challenges and lack of reporting guidance. In our research we have analyzed the key documents in China, UK, USA and Australia and have identified key elements which should be present if a treatment is to be considered as integrated. These elements include supporting the body’s innate healing response, providing a holistic approach / individualized approach, optimum treatment combining Complementary and Alternative Medicine (CAM) and conventional medicine, the patient-practitioner relationship and a collaborative approach, multidisciplinary teamwork. These should be aspects of all patient care, but clinical guideline development does not focus on these elements.
| Clinical Practice Guidelines|| |
Research should form the basis of clinical decision making but even if studies are published, it may be the case that they fail to be used effectively to influence care provision. An example of this is how research is included as evidence in Clinical Practice Guidelines (CPGs) and whether it influences guideline recommendations and subsequently and perhaps more importantly whether it influences clinical practice. In 2014, the 279 National Health Service (NHS) UK clinical guidelines from the National Institute of Health Care Excellence (NICE) were analyzed for some mention of Complementary and Alternative Medicine (CAM). A total of 60 guidelines were identified with acupuncture being positively recommended in 3 guidelines. Acupuncture was specifically recommended as a potential intervention which could be considered for chronic low back pain, headaches, migraine. Acupuncture and electro acupuncture were specifically not recommended for the treatment of osteoarthritis. The use of acupuncture for low back pain is in danger of being removed from UK current guideline recommendations, despite the fact that there is now more positive and better evidence. The individual patient meta-analysis of over 18,000 patients with chronic pain published by Vickers in 2012 is particularly compelling as it has shown effectiveness of acupuncture for a range of chronic pain conditions including osteoarthritis of the knee. They concluded that acupuncture was a ‘reasonable referral option’ given there were significant but modest differences between true and sham acupuncture, so acupuncture was more than a placebo. Although it is recognized that there are other factors in addition to the specific effects of needling which are important contributors to the therapeutic effects of acupuncture. Despite the evidence apparently demonstrating that acupuncture does have a therapeutic effect, this evidence has yet to be quoted in any guidelines on the treatment of chronic pain. These reasons are unclear particularly given the fact that many indications for the use of WM interventions is far from robust. It may be the way acupuncture is perceived or it may be associated with negative biases inherent in the formation of guideline developers or perhaps it may be that there is a lack of cost effectiveness data is becoming a major driver.
In a similar mapping on Chinese Clinical practice guidelines, Beijing University of Chinese medicine explored all Clinical Guidelines identified from the China Guideline Clearinghouse (CGC). When TCM-related contents were extracted from 604 CPGs only 74(12%) recommended TCM and only five guidelines (7%) had applied evidence grading. 14 guidelines (19%) recommended acupuncture, but only four guidelines reported level II evidence related to acupuncture. This is disappointing as in 2009, publications of acupuncture research far outstripped that of other countries, and it appears that this is not taken seriously by guideline developers. Another important issue to mention is that the evidence used for these Chinese CPGs did not include the evidence from trials and systematic reviews carried out in the West, even though some of the research was of good quality. A recent systematic review provides evidence to explain the mechanisms which may be operating in acupuncture such as the brain’s response to acupuncture stimuli which includes somatosensory, affective and cognitive processing. In this systematic review the differences between verum and sham acupuncture were explored to see whether acupuncture manipulation produced changes in the brain if there were differences between patients and healthy volunteers, and how or if different acupuncture points changed fMRI images. While the results are heterogeneous, most studies have suggested that acupuncture can modulate the activity within specific brain areas, and the evidence based on meta-analyses has confirmed some of these results. More high quality studies with more transparent methodology are required.
Therefore there seems to be problems in how evidence is being used to inform CPG guidelines and their recommendations, in both the UK and China. This suggests that the available best evidence may not be being used in facilitating CPG development for integration into best clinical practice. This also has implications for future updating and development of Western medicine clinical practice guidelines for both China and the West. This means ensuring that there is inclusivity in providing non-biased evidence for use by guideline developers. This problem may however not be unique as research on Western clinical evidence has shown that for 2500 treatments of Western medical interventions supported by good evidence 50% had unknown effectiveness and only 15% shown to be beneficial. This work was not extrapolated to see whether all evidence was included (positive or negative) in any of the guidelines developed. As Sackett said “Some questions about therapy do not require randomised trials (successful interventions for otherwise fatal conditions) or cannot wait for the trials to be conducted.” Perhaps the issue lies in improving research quality.
| The Importance of Feasibility Studies|| |
Increasingly feasibility studies are conducted in preparation for randomised controlled trials. This is primarily to explore potential uncertainties regarding the intervention and its delivery, acceptability of the intervention to patients, recruitment and how to conduct the trial to optimise results. It is during a feasibility study that researchers may identify the key issues which may mean further developmental phases are required before carrying out a definitive trial.
The following 3 studies, briefly described here employed the Medical Research Council’s (MRC) framework for studying complex interventions. This framework has been used not only to look at the feasibility of the approach but is particularly helpful when trying to investigate integrative personalised approaches (due to the variations in types and frequencies and choice of treatments) often used within pragmatic studies during the complex care pathway particularly involved in treating musculo skeletal diseases (MSDs). This framework provides a necessary step in giving the researcher opportunities to identify the influencers emerging and which should be considered in the planning of a future definitive controlled trial as well as designing and registering appropriate protocols. This approach will help in providing a robust design which will ensure publication in a reputable journal as well as ensuring a significant contribution to the evidence base.
Postgraduate studies can help to develop initial hypotheses by conducting such feasibility studies. By exploring potential issues (e.g. the ease of recruitment, acceptability of outcome measures) prior to embarking on a full trial, the parameters can be defined and the sample size determined for a fully powered study. In building the evidence for Chinese medicine for musculoskeletal conditions (MSDs) postgraduate studies at London South Bank University have focused on 3 main areas.
The first, a pragmatic mixed methods feasibility study was carried out on ‘Integrative medicine for the treatment of MSDs at the Royal London Hospital for Integrative Medicine (RLHIM) in London. This hospital is part of the Western medical hospital - University College London. RLHIM is unique in that it is the only National Health Service hospital that offers an integrated approach to care by providing a range of CAM including acupuncture for diverse conditions. Referral and treatment pathways for MSDS patients can include western diagnostic testing as well as providing acupuncture, homeopathy, physiotherapy, dietetics, hypnosis/CBT, occupational therapy, and an has insomnia clinic.
This feasibility, pragmatic, mixed methods, prospective, observational study involved newly referred patients with MSDs to the RLHIM. Recruitment to the study was (Jan 2013–April 2014) with 12 month follow-up (Jan 2014 – April 2015). Each patient recruited into the study was treated individually as per routine clinical practice receiving different ‘packages’ of integrative care. This could involve both Western and complementary approaches, depending on the patient’s needs. Therefore, integrative treatments provided at the RLHIM for MSDs varied with different types, frequency and complexity of treatments and assessments.
The mixed methods study design incorporated quantitative and qualitative data (semi- structured interviews and focus groups pre and post treatment). Patient assessment occurred at baseline, 4, 8 and 12 months.
The primary quantitative outcomes were pain as measured by the Visual Analogue Scale (VAS) and the SF36 Bodily pain dimension. The secondary outcome measures were the short form Brief pain Inventory (sfBPI) and the SF 36 other dimensions. The complexity of the types of treatments received by patients varied over time but 50% received acupuncture.
Improvements in pain severity were observed, with a moderate to large effect size as measured by VAS and sfBPI at 4 months compared with baseline data pretreatment. This improvement was maintained between 4 and 12 months.
Improvements were also observed in pain interference, physical function, and mental function, with a small to moderate effect size as measured by sfBPI and SF36. The results suggested that VAS and sfBPI may be sensitive to changes observed in integrative treatment for MSDs.
Qualitative data helped in the interpretation of the changes obtained as a result of receiving integrative treatments and also included participants’ changes in the way they understood MSDs and possible treatment mechanisms, the effect on daily life, referral and expectations. Participants’ acceptability of integrative treatment was revealed in their high appointment attendances and their positive experiences of the patient-practitioner relationship and their expectations of treatment at the hospital. It also demonstrated the feasibility of conducting the research within routine hospital care and the issues to be considered when planning an RCT. Participants experienced a variety of problems accessing treatment at this hospital because in the UK referral to secondary care is managed through primary care (general practice - GP referral). This may depend on whether the GPs have a commissioning contract, the cost to their budget and/ or whether they feel there is enough evidence to suggest that the patient could benefit.
Using a mixed method approach was valuable and the two quantitative outcome measures used, the VAS and SfBPI were appropriate and sensitive. Integrative treatment was perceived as a beneficial approach for the relief of patients’ pain and the study provided important information for the future delivery of the MSDs pathway at the hospital and the barriers and successes of an integrative treatment programme. It also demonstrated that it was feasible to carry out a pragmatic study and will provide helpful guidance for planning a future trial.
The next postgraduate study relates to a feasibility study on ‘Acupuncture use for Phantom Limb pain’. Amputation of a limb occurs for various reasons such as peripheral vascular disease, diabetes or trauma. Phantom Limb pain (PLP) is a common condition experienced by approximately 70% of amputees with little effective pharmacological intervention. As acupuncture has been shown to be effective for pain conditions it was thought that it may be helpful for those with PLP. The first challenge was to design an acupuncture protocol for the treatment of lower limb amputees with phantom limb pain and secondly to evaluate the feasibility and acceptability of implementing this in a randomised controlled trial within rehabilitation care. A mixed methods approach was adopted which had several parts. A systematic literature review,, a Delphi study of acupuncture practitioners opinions of how to treat PLP,, a qualitative study of patients acceptability of having acupuncture and a feasibility RCT.
Systematic reviews of controlled studies from 18 Chinese, Korean and English databases identified mainly Chinese articles and suggested that the number of trials and quality of evidence from controlled trials was limited,. Only one RCT and four non randomized controlled trials were identified but a narrative review of 36 case studies/ reports (28 acupuncture) suggested that there were positive indications for the use of acupuncture .The narrative review of case reports also suggested that there were positive indications for the use of acupuncture with some potentially common acupoints emerging. A subsequent Delphi study investigated how acupuncture practitioners would treat phantom limb pain given 2 particular case scenarios. A protocol for acupuncture treatment of PLP was developed based on the Delphi study and the systematic literature review which was fed back to participants and agreed through consensus.
At the same time a qualitative study with amputees with PLP explored their lived experience of PLP to understand how quality of life was affected and to investigate whether patients felt that acupuncture was acceptable as an intervention. Key themes identified included issues around amputation and the loss of a limb, effects on quality of life, physical and real pain, acceptance, being informed about PLP, views on having acupuncture as an intervention, and attitudes on potential research outcome measures. The first part of this study has recently been published in the British Journal of Pain.
The original rationale was that acupuncture has been shown to be effective for chronic pain and therefore may be effective for PLP but there was no consensus/ treatment guidelines for acupuncture treatment of phantom limb syndrome. These 3 separate phases detailed above have informed a small feasibility RCT in an amputee rehabilitation unit in South London to compare acupuncture treatment with routine care. The protocol has been published in Trials. The acupuncture protocol developed as a result of the Delphi will be piloted. This study has recently been completed and being prepared for publication. It will add to the small evidence base on the use of acupuncture for this condition.
A post graduate study on osteoarthritis of the knee will investigate carrying out a pragmatic pilot RCT comparing 2 different acupuncture techniques. Although there have been trials on the use of acupuncture for this condition, they have not necessarily mimicked clinical in practice. This study has been designed to look at specific components involved in acupuncture by comparing acupuncture with warm needling acupuncture. Various components are involved in the process of acupuncture which include the physical components of needling (location, depth, size, and number), and non-specific components such as time, attention, expectation. It is important to isolate components of practice to try to identify what are the components which have the largest effect. There is also controversy by some who believe a sham or placebo approach is a necessary inclusion in a randomized controlled trial and others who do not. This is why in our research we have argued against this in our recent paper and for this study we have adopted a comparative effectiveness design.
The study is informed by a traditional practice approach where moxa depending on pattern identification has been consistently recommended in TCM text books and is used pragmatically in clinical practice but it is rare that trials have evaluated its effectiveness. A systematic review is currently being undertaken with Chinese and Korean colleagues but much of the literature is Chinese. In addition semi structured interviews have been carried out with a diversely trained group of UK acupuncture practitioners to ascertain treatment of OA knee. This will be followed by a survey to a diverse group of practitioners in order to confirm the treatment protocol prior to the piloting the RCT. It is intended that participants will receive 10 treatments, points around the knee and optional points based on syndrome differentiation. Data collection will take place at baseline, 4, 8 and 16 weeks. The primary outcome measures will be the WOMAC and SF36. Qualitative interviews will also take place with patients to look at expectations, adequacy of blinding, medication use. This planned mixed methods study has been registered and we think that we have managed to satisfactorily overcome the issue of blinding which will be necessary for comparison.
Although TCM practitioners in the EU and China vary in their perceived need for research evidence and the evidence gaps, the practitioner perception of the evidence base is equally important. International variations in acupuncture training, education and policy have also compounded this. In addition our recent paper, again using a Delphi technique explored Chinese TCM journal Editors opinions on the publication of research in this field. They agreed that the highest quality articles were submitted to Western journals, that there was a need for rigorous methodology and that repeating poor studies was not helpful in providing an authoritative viewpoint. Journal editors should look for high quality manuscripts, with an interesting topic, with a clear positioning and rationale and should follow appropriate checklists. If adhered to this will in turn affect the quality of research being designed and executed.
| Conclusion|| |
It is vitally important that competent professionals, both clinically skilled and trained in research are involved in the development of the evidence base to ensure that it is appropriate and ongoing and above all it is utilized. Otherwise studies will be designed and conducted by researchers who do not understand traditional systems of medicine or by practitioners who do not fully understand research practices. If this persists, good quality effectiveness data will continue to be lacking and will compromise integration into any healthcare system.
| Acknowledgements|| |
Grateful thanks go to my 3 PhD students, Xiao-Yang Hu, Esme Trevelyan and Ian Appleyard for all their effort and hard work throughout their postgraduate studies.
| References|| |
Falkenberg T, Smith M, Robinson N. Traditional and integrative approaches for global health. European Journal of Integrative Medicine
Burton A, Smith M, Falkenberg T. Building WHO’s global Strategy for Traditional Medicine. European Journal of Integrative Medicine
Witt CM, Brinkhaus B, Reinhold T, Willich SN. Efficacy, effectiveness, safety and costs of acupuncture for chronic pain - results of a large research initiative. Acupuncture in Medicine
2006, 24:Suppl 33-39. doi:10.1136/aim.24.Suppl.33.
Kim S-Y, Lee H, Chae Y, Park H-J, Lee H. A systematic review of cost-effectiveness analyses alongside randomised controlled trials of acupuncture. Acupuncture in Medicine
2012, 30(4): 273-85.
Hu XY, Lorenc A, Kemper K, Adams J, Robinson N. Defining Integrative Medicine for Narrative and Systematic Reviews: a suggested checklist for reporting. European Journal of Integrative Medicine
2015, 7(1): 76-84. doi:10.1016/j.eujim.2014.11.006.
Robinson N, Lorenc A, Ding W, Jun J, Bovey M, Wang X. Exploring practice characteristics and research priorities of practitioners of traditional acupuncture in China and the EU - a survey. Journal of Ethnopharmacology
2012, 140(3): 604-613.
Giovanardi CM, Fauci AJ, Mazzanti U, Roberti di Sarsina P, Tassinari M, Lilli A, Barbanera MLl, Devecchi M, Farella GMG, Poini A. Toward a definition of the standards for research in acupuncture: The Bologna International Symposium. European Journal of Integrative Medicine
2015, 7(4): 429-431. doi:10.1016/j.eujim.2015.07.028.
Han J-S, Ho Y-S. Global trends and performances of acupuncture research. Neuroscience and Biobehavioral Reviews
2011, 35: 680-687.
Hu XY, Chen N, Chai Q, Yang G, Trevelyan E, Lorenc A, Liu J, Robinson N. Integrative treatment for low back pain: An exploratory systematic review and meta-analysis of randomised controlled trials. Chinese Journal of Integrative Medicine
2015;10/2015. doi:10.1007/ s11655-015-2125-2 [Epub ahead of print]
Lorenc A, Leach J, Robinson N. Clinical guidelines in the UK: Do they mention Complementary and Alternative Medicine (CAM) - are CAM professional bodies aware? European Journal of Integrative Medicine (Special issue on Clinical guidelines for Integrated Practice)
2014, 6(2): 164-175. http://dx.doi.org/10.1016/j.eujim.2013.11.003
Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson Hugh, Foster Nadine E, Sherman KJ, Witt CM, Linde, K, for the Acupuncture Trialists’ Collaboration. Acupuncture for Chronic Pain Individual Patient Data Meta-analysis. Arch Intern Med
2012, 172(19): 1444-1453. doi:10.1001/archinternmed.2012.3654.
Ren J, Li X, Sun J, Han M, Yang G-Y, Li W-Y, Robinson N, Lewith G, Liu J-P. Is Traditional Chinese Medicine Recommended in Western Medicine Clinical Practice Guidelines in China? A systematic analysis. BMJ Open
2015, 5: e006572. doi:10.1136/bmjopen-2014-006572.
Huang W, Pach D, Napadow V, Park K, Long X, Neumann J, et al. Characterizing Acupuncture Stimuli Using Brain Imaging with fMRI - A Systematic Review and Meta-Analysis of the Literature. PLoS ONE
2012, 7(4): e32960. doi:10.1371/journal.pone.0032960.
Hu X, Hughes J, Fisher P, Lorenc A, Purtell R, Park A, Robinson N. A Pragmatic Observational Feasibility Study on Integrated Treatment for Musculoskeletal Disorders (FIT): Design and Protocol. Chinese Journal of Integrative Medicine
2016, 22(2): 88-95. doi: 10.1007/s11655- 013-1557-9. Epub 2013 Dec 13.
Hu X-Y, Hughes J, Fisher P, Lorenc A, Robinson. A pragmatic observational feasibility study on integrative treatment for muscu-loskeletal disorders: Quantitative results. European Journal of Integrative Medicine
2015, 7(6). 680. http://dx.doi.org/10.1016/ j.eujim.2015.07.007
Xiao-Yang Hu, Ava Lorenc, John Hughes, Peter Fisher, A-La Park, Rachel Purtell, Nicola Robinson. Integrative Treatment for Musculoskeletal Disorders: A Mixed Methods Study. European Journal of Integrative Medicine
2015, 7(5), (Supplement 1) 9.
Trevelyan EG, Turner WA, Robinson N. Acupuncture for the treatment of phantom limb pain in lower limb amputees: study protocol for a randomised controlled feasibility trial. Trials
2015, 16: 158. doi:10.1186/s13063-015-0668-3.
Hu X, Trevelyan E, Yang G, Lee MS, Lorenc A, Liu J, Robinson N: The effectiveness of acupuncture/TENS for phantom limb syndrome. I: A systematic review of controlled clinical trials. European Journal of Integrative Medicine
2014, 6(3): 355-364.
Hu X, Trevelyan E, Yang G, Lorenc A, Lee MS, Liu J, Robinson N. The effectiveness of acupuncture and TENS for phantom limb syndrome II. A narrative review of case studies. European Journal of Integrative Medicine
2014, 6(3): 365-381. http://dx.doi.org/10.1016/j.eujim. 2014.02.001
Trevelyan EG, Turner WA, Robinson N: Developing an acupuncture protocol for treating phantom limb pain: A Delphi consensus study. Acupuncture in Medicine
2015, 33(1): 42-50. 1-9. doi:10.1136/ acupmed-2014-010668
Trevelyan EG, Robinson N. Delphi methodology in health research: how to do it? European Journal of Integrative Medicine
2015, 7(4): 423-428. doi:10.1016/j.eujim.2015.07.002
Trevelyan EG, Turner WA, Robinson N. Perceptions of phantom limb pain in lower limb amputees and its effect on quality of life: a qualitative study. British Journal of Pain
2015 1-8. doi:10.1177/ 2049463715590884.
Appleyard I, Lundberg T, Robinson N. Should systematic reviews assess the risk of bias from sham-placebo acupuncture control procedures? European Journal of Integrative Medicine (Special issue on Clinical guidelines for Integrated Practice)
2014, 6(2): 234-243.
Hu X-Y, Robinson N, Yu H, Liu J-P. Establishing the key elements required to publish a good article: Consensus among Chinese TCM and integrative medicine journal editors - a Delphi study. Journal of Traditional Chinese Medicine
2015, 35(3): 1-8.