ISSN - Versão Impressa: 0102-3616 ISSN - Versão Online: 1982-4378

8
Visualizações
Acesso aberto Revisado por pares
Original Article

Evaluation of Interobserver Agreement in the Diagnosis of Posterior Pelvic Ring Lesions Using Plain Radiography*

Avaliação da concordância interobservador no diagnóstico de lesões do anel pélvico posterior usando radiografia simples*

Leonardo Comerlatto1, Alberto Braun Batista1, Natália Henz Concatto2, Ary da Silva Ungaretti Neto1, Ramiro Zilles Gonçalves1

DOI: 10.1055/s-0039-1697014


 

ABSTRACT:

OBJECTIVE To evaluate the interobserver agreement of a radiologist, two hip specialist orthopedists with experience in the treatment of pelvic and acetabulum fractures, two general orthopedists, two orthopedics residents and two radiology residents regarding the diagnosis of posterior pelvic ring injuries using plain radiography.
METHOD A cross-sectional study conducted in September 2017. The exams of 20 patients who had been treated for traumatic lesions of the posterior pelvic ring were randomly selected.. A total of nine examiners from different medical fields evaluated the plain pelvic radiographs; those were compared with computed tomography, scans, which are considered a confirmatory diagnostic criterion. Interobserver agreement was assessed using the Kappa test (κ) and 95% confidence intervals (95%CIs).
RESULTS A total of 28 lesions were found by computed tomography (23%; 95%CI: 16-32%) among all of the cases evaluated. The interobserver agreement between plain radiography and computed tomography was moderate among physicians with more experience: a radiologist (κ = 0.461; 95%CI: 0.270-0.652), hip specialists 1 and 2 (κ = 0.534; 95%CI: 0.348-0.721 and κ = 0.431; 95%CI: 0.235-0.627 respectively), and general orthopedists 1 and 2 (κ = 0.497; 95%CI: 0.307-0.686 and κ = 0.449; 95%CI: 0.254-0.645 respectively). Among the orthopedics and radiology residents, the interobserver agreement was considered poor. High false negative values were found among all of the examiners, especially regarding posterior iliac fractures and sacrum fractures.
CONCLUSION Professionals with greater experience in the field have a better ability to identify posterior pelvic ring lesions by plain radiography, but we emphasize that plain pelvic radiography was susceptible to false negative evaluations among all of the professionals assessed.

Keywords:
medical education; bone fractures; radiology; traumatology.

RESUMO:

OBJETIVO Avaliar a concordância interobservador de radiologista, dois ortopedistas especialistas em quadril com experiência no tratamento de fraturas da pelve e acetábulo, dois ortopedistas gerais, dois residentes de ortopedia, e dois residentes de radiologia no diagnóstico de lesões do anel pélvico posterior usando radiografia simples.
MÉTODO Estudo transversal, realizado em setembro de 2017. Foram selecionados retrospectivamente e de modo aleatório exames de 20 pacientes atendidos previamente com lesões traumáticas do anel pélvico posterior. Nove examinadores de diferentes áreas médicas avaliaram as radiografias simples de bacia, que foram comparadas com a tomografia computadorizada, considerada critério confirmatório de diagnóstico. A concordância interobservador foi analisada pelo teste de Kappa (κ), e com intervalos de confiança de 95% (IC95%).
RESULTADOS Foram encontradas pela tomografia computadorizada 28 lesões (23%; IC95%: 16-32%) em todos os casos avaliados. A concordância interobservador entre a radiografia simples e a tomografia computadorizada foi moderada nos médicos com mais experiência: o radiologista (κ = 0,461; IC95%: 0,270-0,652), os especialistas em quadril 1 e 2 (κ = 0,534; IC95%: 0,348-0,721; e κ = 0,431; IC95%: 0,235-0,627, respectivamente), e os ortopedistas gerais 1 e 2 (κ = 0,497; IC95%: 0,307-0,686; e κ = 0,449; IC95%: 0,254-0,645, respectivamente). Já com relação aos residentes de ortopedia e radiologia, a concordância interobservador foi considerada fraca. Nos diagnósticos de todos os examinadores, foram encontrados altos valores falso-negativos, principalmente nas fraturas da região posterior do ilíaco e nas fraturas do sacro.
CONCLUSÃO Profissionais com mais experiência na área apresentam melhor capacidade de identificação de lesões do anel pélvico posterior por radiografia simples; porém, salienta-se que a radiografia simples de pelve esteve suscetível a avaliações falso-negativas da parte de todos os profissionais estudados.

Palavras-chave:
educação médica; fraturas ósseas; radiologia; traumatologia.

FIGURES

Citation: Comerlatto L, Batista AB, Concatto NH, Ungaretti Neto AS, Gonçalves RZ. Evaluation of Interobserver Agreement in the Diagnosis of Posterior Pelvic Ring Lesions Using Plain Radiography*. 54(6):673. doi:10.1055/s-0039-1697014
Note: * Study developed at the Orthopedics and Traumatology Service, Hospital Cristo Redentor - Grupo Hospitalar Conceição, Porto Alegre, Rio Grande do Sul, Brazil.
Received: March 05 2018; Accepted: May 07 2018
 

INTRODUCTION

Plain radiographic assessment of the pelvis remains the initial diagnostic pillar for pelvic ring fractures, and it is recommended by the Advanced Trauma Life Support (ATLS), which was developed by the American College of Surgeons (ACS) Committee on Trauma (COT). However, injuries to the structures of the posterior pelvic ring (posterior portion of the iliac bone, sacroiliac joint and sacrum) are difficult to diagnose, and may cause delay and failure in the treatment of these lesions.1

Professionals with longer experience are considered to have a greater ability to diagnose posterior pelvic ring lesions by plain radiography.2 However, studies show a high frequency of misdiagnosis with the use of plain radiography, especially in lesions involving the posterior pelvic ring structures, and they deem an imperative the performance of computed tomography (CT) scans, which are considered the gold standard in the diagnosis of these lesions.3 To this end, the aim of the present study was to evaluate the interobserver agreement of a radiologist, hip specialist orthopedists, general orthopedists, orthopedics residents and radiology residents regarding the diagnosis of posterior pelvic ring injuries using plain radiography.

 

MATERIALS AND METHOD

A cross-sectional study conducted in September 2017 at the Orthopedics and Traumatology Service of Hospital Cristo Redentor (HCR) - Grupo Hospitalar Conceição (GHC), in the city of Porto Alegre, Southern Brazil. The research was approved by our institution's Ethics in Research Committee (under CAAE: 72595617.7.0000.5530). Retrospective examinations of 20 patients with traumatic injuries of the posterior pelvic ring previously treated in the emergency unit were randomly selected by lot. Only cases with simultaneous plain radiographic evaluation of the anteroposterior (AP) view of the pelvis and with CT scans were included in the study; the CT scans were used as confirmatory diagnostic criteria.

Examiners and image evaluation

The nine examiners were physicians, among them a radiologist with at least ten years of experience in trauma emergency, two orthopedic specialists in hip surgery with at least ten years of professional experience and with experience in pelvic and acetabulum fracture surgery, two general orthopedists, two orthopedics residents, and two radiology residents, all with experience in emergency care.

The images were evaluated as follows: a) a room reserved for slide projections was allocated to the nine examiners; b) a chart with the pelvis design was presented, enabling the examiner to mark the lesion sites in the posterior pelvic ring regions: posterior iliac, sacroiliac joint and sacrum (Figure 1) for each case presented, as well as for the absence of injury, if they so deemed it. In the present study, 120 anatomical sites were evaluated by each examiner; c) the radiographs of the selected cases were presented to the examiners through slide projections, with 30 seconds to consider each case. All of the selected cases presented an identifiable anterior pelvic ring lesion on the radiographs.

Data analysis

The data regarding the categorical variables were presented by frequency (%). Interobserver agreement was assessed by the Kappa test (κ) and 95% confidence intervals (95%CIs), and the values assumed for agreement were considered as follows: 0.20: poor; 0.21-0.40: fair; 0.41-0.60: moderate; 0.61-0.80: good; 0.81-1.00: very good.4-6 The CT diagnosis was assumed as a reference evaluation. All analyses were performed using the Statistical Package for the Social Sciences (SPSS, IBM Corp., Armonk, NY, US), version 22.0 for Windows.

 

RESULTS

A total of 28 lesions in the posterior pelvic ring were determined by CT from a total of 120 possible injuries (23%; 95%CI: 16%-32%), since our schematic chart enabled the identification of 6 lesion sites for each of the 20 cases. Table 1 shows the interobserver agreement between the CT and the plain radiographic evaluation. Among the most experienced examiners, the agreement was moderate: radiologist (κ = 0.461; 95%CI: 0.270-0.652), hip specialists 1 and 2 (κ = 0.534; 95%CI: 0.348-0.721 and κ = 0.431; 95%CI: 0.235-0.627 respectively), and general orthopedists 1 and 2 (κ = 0.497; 95%CI: 0.307-0.686 and κ = 0.449; 95%CI: 0.254-0.645 respectively).When comparing the CT results with the diagnoses made by the orthopedics and radiology residents, the interobserver agreement was considered poor (Table 1).

Table 1. Interobserver agreement of the diagnoses (by plain radiography) of lesions in the regions of the posterior pelvic ring, compared to the CT scans
Parameters Lesion diagnosis CT agreement
Negative Positive % Kappa (95%CI)
CT (reference) 092 (77) 028 (23)    
Radiologist 196 (80) 024 (20) 82 0.461 (0.270-0.652)
Hip specialist 1 100 (83) 020 (17) 85 0.534 (0.348-0.721)
Hip specialist 2 100 (83) 020 (17) 82 0.431 (0.235-0.627)
General orthopedist 1 198 (82) 022 (18) 83 0.497 (0.307-0.686)
General orthopedist 2 101 (84) 019 (16) 82 0.449 (0.254-0.645)
Orthopedics resident 1 101 (84) 019 (16) 81 0.397 (0.198-0.596)
Orthopedics resident 2 101 (84) 019 (16) 79 0.344 (0.143-0.545)
Radiology resident 1 198 (82) 022 (18) 78 0.346 (0.146-0.545)
Radiology resident 2 198 (81) 022 (19) 77 0.329 (0.130-0.529)

Abbreviations: 95%CI, 95% confidence interval; CT, computed tomography.

Note: Data presented as frequency (%).

High false negative values were found in the diagnoses of all of the examiners: radiologist (46%), hip specialists 1 and 2 (46% and 54% respectively), general orthopedists 1 and 2 (46% and 54% respectively), orthopedics residents 1 and 2 (57% and 61% respectively) and radiology residents 1 and 2 (57% and 57% respectively).

Figure 2 shows the 28 lesions according to their respective anatomical sites: 3 posterior iliac fractures, 10 sacroiliac joint injuries and 15 sacral fractures were found on the CT. Figure 3 shows the lesions found at each anatomical site in relation to the respective examiners.

 

DISCUSSION

In the present study, we identified that experienced professionals have a greater ability to diagnose posterior pelvic ring lesions by plain radiography. However, plain radiography was susceptible to a high percentage of false negative evaluations among all of the examiners when compared to the CT scans, that is, there was great difficulty in identifying posterior iliac and sacrum fractures.

Posterior pelvic ring injuries are severe and difficult to diagnose,7-10 often requiring urgent stabilization to reestablish the polytraumatized patient from the hemodynamic point of view,11-14 and/or subsequent surgical procedures to fix fractures or dislocations, enabling the patient to return to his or her best functional condition.

Pelvic radiography is indicated as one of the routine exams in high-energy trauma care, but in some situations this exam is insufficient for the diagnosis, classification and procedure definition.15-19 The factors that make pelvic assessment difficult through isolated plain radiography are the lack of patient preparation, with the presence of artifacts such as gas, fecal content, bladder distension, and the complex three-dimensional conformation of the pelvis.20,21 Due to the angle of inclination of the sacrum, visualization is limited in the pelvis AP.1

The CT is often not available in non-specialized centers,8,22-24 and with it the patient is subjected to a higher radiation dose. Publications have shown a high frequency of failure to diagnose pelvic lesions on radiographs, especially lesions involving the sacroiliac joint and the sacrum.25,26 The identification of up to 30% of sacrum fractures is late, which has a negative effect on long-term outcomes.3,27,28 Montana et al29 found diagnosis failure in 28% of sacroiliac joint dislocations, and in 57% of iliac and sacral fractures adjacent to the sacroiliac joint, with the isolated use of plain radiography.

The present study has some limitations: a) the inferior quality of some of the selected radiographs, a usual occurrence in the emergency care context; and b) the definition of the number of selected cases, as well as of the examiners, was performed through a convenience sample.

 

CONCLUSION

Professionals with greater experience in the field have a better ability to identify posterior pelvic ring lesions by plain radiography; however, we emphasize that simple pelvic radiography was susceptible to false negative diagnoses among all of the professionals assessed, especially regarding fractures of the posterior region of the iliac and sacrum.

 

ACKNOWLEDGEMENTS

We would like to thank the Hip Group (HCR/GHC) for their support and collaboration.

 

REFERENCES

Vaccaro AR, Kim DH, Brodke DS, et al. Diagnosis and management of sacral spine fractures. Instr Course Lect 2004;53:375-385 Link PubMed
Bucholz RW, Heckman JD, Court-Brown CM, Torneta P III. Rockwood and Green's fractures in adults. 8th ed. Philadelphia: Lippincott; 2015
Schicho A, Schmidt SA, Seeber K, Olivier A, Richter PH, Gebhard F. Pelvic X-raymisses out on detecting sacral fractures in the elderly - Importance of CT imaging in blunt pelvic trauma. Injury 2016;47 (03):707-710 Link DOI
Liggieri AC, Tamanaha MJ, Abechain JJK, Ikeda TM, Dobashi ET. Concordância intra e interobservadores das diferentes classificações usadas na doença de Legg-Calvé-Perthes. Rev Bras Ortop 2015;50(06):680-5 Link DOI
Machado DG, Cerqueira SAC, de Lima AF, Mathias MB, Aramburu JPG, Rodarte RR. Statistical analysis on the concordance of the radiological evaluation of fractures of the distal radius subjected to traction. Rev Bras Ortop 2016;51(01):11-15 Link DOI
Altman DG. Practical statistic formedical research. 3rd ed. London: Chapman and Hall; 1995
Young JW, Burgess AR, Brumback RJ, Poka A. Pelvic fractures: value of plain radiography in early assessment and management. Radiology 1986;160(02):445-451 Link DOI
Denis F, Davis S, Comfort T. Sacral fractures: an important problem. Retrospective analysis of 236 cases. Clin Orthop Relat Res 1988;227(227):67-81
Kokubo Y, Oki H, Sugita D, et al. Functional outcome of patients with unstable pelvic ring fracture. J Orthop Surg (Hong Kong) 2017;25(01):2309499016684322
Dominguez S, Liu P, Roberts C, MandellM, Richman PB. Prevalence of traumatic hip and pelvic fractures in patients with suspected hip fracture and negative initial standard radiographs-a study of emergency department patients. Acad Emerg Med 2005;12(04): 366-369 Link DOI
Heetveld MJ, Harris I, Schlaphoff G, Sugrue M. Guidelines for the management of haemodynamically unstable pelvic fracture patients. ANZ J Surg 2004;74(07):520-529 Link DOI
Osborn PM, Smith WR, Moore EE, et al. Direct retroperitoneal pelvic packing versus pelvic angiography: A comparison of two management protocols for haemodynamically unstable pelvic fractures. Injury 2009;40(01):54-60 Link DOI
Rommens PM, Hofmann A, Hessmann MH. Management of acute hemorrhage in pelvic trauma: an overview. Eur J Trauma Emerg Surg 2010;36(02):91-99 Link DOI
Suzuki T, Smith WR, Moore EE. Pelvic packing or angiography: competitive or complementary? Injury 2009;40(04): 343-353 Link DOI
Garras DN, Carothers JT, Olson SA. Single-leg-stance (flamingo) radiographs to assess pelvic instability: how much motion is normal? J Bone Joint Surg Am 2008;90(10):2114-2118 Link DOI Link PubMed
Kool DR, Blickman JG. Advanced Trauma Life Support. ABCDE from a radiological point of view. Emerg Radiol 2007;14(03): 135-141 Link DOI
Koo H, LeveridgeM, Thompson C, et al. Interobserver reliability of the young-burgess and tile classification systems for fractures of the pelvic ring. J Orthop Trauma 2008;22(06):379-384
Savolaine ER, Ebraheim NA, Rusin JJ, Jackson WT. Limitations of radiography and computed tomography in the diagnosis of transverse sacral fracture from a high fall. A case report. Clin Orthop Relat Res 1991;(272):122-126 Link PubMed
Henes FO, Nüchtern JV, Groth M, et al. Comparison of diagnostic accuracy of Magnetic Resonance Imaging and Multidetector Computed Tomography in the detection of pelvic fractures. Eur J Radiol 2012;81(09):2337-2342 Link DOI
Shin DS, Jang HG, Hwang SB, Har DH, Moon YL, Chung MS. Twodimensional sectioned images and three-dimensional surface models for learning the anatomy of the female pelvis. Anat Sci Educ 2013;6(05):316-323 Link DOI
Cohen AS, McNeill JM, Calkins E, Sharp JT, Schubart A. The "normal" sacroiliac joint. Analysis of 88 sacroiliac roentgenograms. Am J Roentgenol Radium Ther Nucl Med 1967;100(03): 559-563 Link DOI
Hudson S, Boyle A, Wiltshire S, McGerty L, Upponi S. Plain Radiography May Be Safely Omitted for Selected Major Trauma Patients UndergoingWhole Body CT: Database Study. Emerg Med Int 2012;2012:432537 Link PubMed
Vo NJ, Gash J, Browning J, Hutson RK. Pelvic imaging in the stable trauma patient: is the AP pelvic radiograph necessary when abdominopelvic CT shows no acute injury? Emerg Radiol 2004; 10(05):246-249 Link DOI
Nüchtern JV, Hartel MJ, Henes FO, et al. Significance of clinical examination, CT and MRI scan in the diagnosis of posterior pelvic ring fractures. Injury 2015;46(02):315-319 Link DOI
Gill K, Bucholz RW. The role of computerized tomographic scanning in the evaluation of major pelvic fractures. J Bone Joint Surg Am 1984;66(01):34-39 Link DOI
St Pierre RK, Oliver T, Somoygi J,Whitesides T, Fleming LL. Computerized tomography in the evaluation and classification of fractures of the acetabulum. Clin Orthop Relat Res 1984;(188):234-237 Link PubMed
RouttML Jr, Simonian PT, SwiontkowskiMF. Stabilization of pelvic ring disruptions. Orthop Clin North Am 1997;28(03):369-388 Link DOI
Nystrom LM, McKinley TO, Marsh JL. Accuracy in radiographic assessment of pelvic ring fracture deformity: analysis of current methods. J Orthop Trauma 2013;27(12):708-715 Link DOI Link PubMed
MontanaMA, Richardson ML, Kilcoyne RF, Harley JD, ShumanWP, Mack LA. CT of sacral injury. Radiology 1986;161(02):499-503