External validation of a clinical prediction rule on the need for radiologic imaging to identify urological disorders in adult patients with febrile urinary tract infections

INTRODUCTION

Febrile urinary tract infection (UTI) is a relatively common illness. About 12–13 outpatients with pyelonephritis per 10,000 females have been detected annually in the United States.(1) Approximately 25% of patients with sepsis are considered to have UTIs.(2) Febrile UTI, a tissue inflammation of the urinary tract, can manifest as pyelonephritis, acute prostatitis or urinary-source bacteraemia.(3) Although it usually presents as a mild disease not requiring urgent intervention, febrile UTI can be extremely severe in the presence of hydronephrosis (caused by an obstruction of the urinary tract by a urinary tract stone, an enlarged prostate, etc). Among patients with sepsis and urinary tract stones, those not receiving surgical decompression have shown higher mortality than those receiving it.(4) Urgent decompression, therefore, is now established as a requisite step in the management of infected hydronephrosis secondary to ureteric stones.(5) Radiologic assessment for obstructions and underlying urological abnormalities is, likewise, a routine step in the clinical approach to UTI. Van Nieuwkoop et al analysed radiologic findings obtained by ultrasonography assessment and computed tomography (CT) of the urinary tract at hospitals in the Netherlands to create a prediction rule based on the presence of three indicators: a history of urolithiasis, a urine pH of ≥ 7.0, and an estimated glomerular filtration rate (eGFR) ≤ 40 mL/min/1.73 m³. The negative predictive values (NPVs) predicted by this rule for clinically relevant radiologic findings and urgent urological disorders were 89% and 100%, respectively.(6) The study also suggested that the prediction rule could reduce unnecessary radiologic imaging studies by as much as 40%, without allowing any relevant radiologic findings or urgent urological disorders to pass unnoticed. An algorithm developed by Johnson and Russo for the management of acute pyelonephritis in adults calls for an initial assessment of the three indicators used to judge the need for genitourinary imaging using the prediction rule created by Van Nieuwkoop et al.(7) According to the management algorithm, there was no need to perform radiologic imaging in cases wherein none of the three indicators were found.

The prediction rule used to determine the need for radiologic imaging for the identification of urological disorders has yet to be assessed through an external evaluation in adult patients with febrile UTI. In this study, we applied the prediction rule created by Van Nieuwkoop et al(6) to an external patient population in order to assess whether the rule was valid and appropriate for clinical use.

METHODS

External validation of a retrospective observational cohort was conducted at the Japanese Red Cross Wakayama Medical Center (770 beds) to validate the prediction rule used to determine the need for radiology. All adult (age > 18 years) patients diagnosed with febrile UTI at our emergency department from 1 January 2017 to 30 September 2018 were included in this study. Data for all patients with positive urine culture was obtained from the laboratory’s database. The clinical data used was obtained from the patients’ medical records. This study was approved by the ethical committee of the Japanese Red Cross Wakayama Medical Center.

The definition of febrile UTI was satisfied when all of the following findings were observed according to the inclusion criteria used in the article by Van Nieuwkoop et al:(6) fever (≥ 38.0°C and/or fever and chills over the last 24 hours); at least one symptom of UTI (dysuria, frequency, urgency, perineal pain, flank pain or costovertebral tenderness); a positive nitrite dipstick test result or leucocyturia (defined as a positive leucocyte esterase dipstick test result or the presence of > 5 leucocytes/high-power field in a centrifuged sediment); and a positive urine culture. The exclusion criteria were recent treatment (within one month before presentation) for urolithiasis or hydronephrosis, pregnancy, superinfection with another infectious disease, current treatment by haemodialysis or peritoneal dialysis, a history of kidney transplantation, known presence of polycystic kidney disease, current nephrostomy, no CT imaging and current urethral catheterisation.

The primary outcome was radiologic findings classified as clinically relevant or clinically irrelevant. Clinically relevant findings included urgent urological disorders (pyonephrosis, renal abscess, obstructive urinary tract stone or any other urinary obstructions accompanying UTI) and non-urgent urological disorders (all other urological disorders requiring treatment). Clinically irrelevant findings included normal findings and any findings attributable to acute pyelonephritis or incidental nonurological disorders. The CT findings were categorised by the clinicians who ordered the CT imaging. Urgent intervention included nephrostomy, ureteral stent placement and urethral catheterisation.

We followed the original prediction rule used to determine the need for radiologic imaging. Each of the three indicators used (history of urolithiasis, urine pH ≥ 7.0 and eGFR ≤ 40 mL/min/1.73 m³) was assigned one point, resulting in a possible range of scores from 0 to 3. The power of the prediction rule was determined by calculating the sensitivity, specificity, positive predictive value (PPV) and NPV of the rule. PASW Statistics software version 18.0 (SPSS Inc, Chicago, IL, USA) was used for statistical data analysis.

RESULTS

A total of 185 patients were included out of 40,724 patients who visited our emergency department, and 136 patients were analysed after excluding 49 patients according to the following exclusion criteria: current urethral catheterisation (n = 13), current nephrostomy (n = 11), multiple infections (n = 10), no CT imaging (n = 7), recent treatment for hydronephrosis or urinary stones (n = 2), and other (n = 6). The patients had a median age of 76 (interquartile range 65–86) years, and 29% of them were male. More than half of the patients reported experiencing costovertebral tenderness (60%) (Table I).

Table I

Patient demographics (n = 136).

The following pathogens were positive in the urine cultures of the 136 patients: Escherichia coli in 93 (68%) patients, Klebsiella sp. in 11 (8%) patients, Enterococcus sp. in 6 (4%) patients and other pathogens in 26 (19%) patients. Blood cultures were positive in 64 (54%) out of 119 (88%) patients tested.

All 136 patients underwent radiologic imaging with CT (with or without medium contrast), 54 (39%) of whom had clinically relevant findings (Table II). The radiologic outcomes were classified as normal or attributable to pyelonephritis (e.g. swelling of the kidney, localised hypodense lesions in the parenchyma) in 68 (50%) patients, urgent urological disorder in 40 (29%) patients and non-urgent urological disorder in 14 (10%) patients (Table II). 32 (80%) out of 40 patients with the radiologic outcome of an urgent urologic disorder underwent an intervention.

Table II

Computed tomography findings of febrile urinary tract infection.

All 136 patients were scored according to the original prediction rule (score range: 0–3 points). 52 (38%) patients scored 0 points, 66 (49%) scored 1 point, 16 (12%) scored 2 points and 2 (1%) scored 3 points. At a cut-off of 1 point for any clinically relevant radiologic finding or urgent urological disorder, the predictive values were as follows: sensitivity of 72% and 78%, specificity of 45% and 45%, PPV of 46% and 37%, and NPV of 71% and 83%, respectively (Table III).

Table III

Predictive value of the prediction rule (score ≥ 1) for different radiologic outcomes.

DISCUSSION

We externally evaluated the prediction rule used to determine the need for radiologic imaging for patients with febrile UTI at an emergency department in Japan. Compared to the original study by Van Nieuwkoop et al,(6) the prediction rule showed a low NPV for any clinically relevant radiologic finding (71%) or any urgent urological disorder (83%).

Our results determined, in other words, that if we had skipped radiologic imaging in cases presenting with none of the three indicators, as prescribed by the algorithm for acute pyelonephritis advocated by Johnson et al,(7) urgent urological disorders would have been overlooked in nearly 20% of the patients with UTI who showed none of the three indicators. This NPV for urgent urological disorders is unacceptable in a clinical setting, given that interventions will be delayed when no urinary obstructions are pointed out in radiologic findings.

The prevalence of urolithiasis in a different study population can be expected to influence the accuracy of the prediction rule. Geographic variations in the presence of urolithiasis stemming from factors such as seasonal changes of temperature and differences in genetics or dietary habits have been observed worldwide. While we were unable to find any data on the prevalence of urolithiasis in the Netherlands, one previous study showed a lower prevalence of urolithiasis in European countries (5%–9%) than in Japan (10%).(8) Meanwhile, the proportion of patients with a history of urolithiasis in the original study conducted in the Netherlands (11%) was not much different from the proportion in our study in Japan (15%).(6)

Owing to the good access to CT imaging in our hospital, most patients presenting with UTI in this study underwent CT to check for obstructions or underlying urological abnormalities. Only seven patients in our study were excluded for not undergoing CT imaging. On the other hand, the radiologic imaging in the original study included both ultrasonography and CT: 84 patients underwent ultrasonography and 23 underwent CT imaging.(6) CT is considered superior to ultrasonography in detecting urological abnormalities. Ultrasonography showed approximately 80% sensitivity in identifying either hydronephrosis or urinary tract stones, compared to CT, which showed 100% sensitivity.(9,10)

Our study detected clinically relevant findings among patients with UTI more frequently than the original study did (39% vs 19%).(6) Although the prediction rule was less accurate in the present external validation than in the original study, we believe that a more accurate evaluation could have been achieved if we had used highly sensitive radiologic imaging.

This study has several limitations. First, patients suffering loss of consciousness were not included, because having at least one positive sign of UTI symptoms was essential for inclusion. Thus, the result of this prediction rule could not be applied to any patient who suffered loss of consciousness as an apparent result of sepsis. Second, our study population is likely to have been smaller than the original study population investigated by Van Nieuwkoop et al, as our subjects showed positive results for urine culture. We doubt, however, that the accuracy of the UTI diagnosis was reduced by this difference in inclusion criteria. Third, positive urine cultures were returned after we reached the decision on whether to perform radiologic imaging at the emergency department in our study. The patient population to which the prediction rule was applied might, therefore, have differed from the populations in actual clinical settings.

In conclusion, the prediction rule used to decide upon the need for radiologic imaging in patients with febrile UTI proved to have fairly poor accuracy at an emergency department in Japan, rendering it difficult to apply the prediction rule in the Japanese clinical setting. Additional studies in other different settings are required.