Association between Platelet-to-Lymphocyte Ratio, C-reactive Protein to Albumin Ratio, Red Cell Distribution Width, and APACHE II Score in Predicting Prognosis and Mortality in Sepsis
PDF
Cite
Share
Request
Original Article
VOLUME: ISSUE:
P: -

Association between Platelet-to-Lymphocyte Ratio, C-reactive Protein to Albumin Ratio, Red Cell Distribution Width, and APACHE II Score in Predicting Prognosis and Mortality in Sepsis

1. Selçuk University Faculty of Medicine Department of Anesthesiology and Reanimation, Division of Critical Care, Konya, Türkiye
No information available.
No information available
Received Date: 08.10.2024
Accepted Date: 16.03.2025
Online Date: 14.04.2025
PDF
Cite
Share
Request

Abstract

Aim

Intensive care unit (ICU) provide critical care and treatment to enhance patient outcomes. There is limited data on new scoring systems that predict prognosis during sepsis. The study aimed to assess the prognostic value of platelet-to-lymphocyte ratio (PLR), C-reactive protein to albumin ratio (CAR), red cell distribution width (RDW), acute physiology and chronic health evaluation II (APACHE II) score, and procalcitonin (PCT) in sepsis.

Materials and Methods

This retrospective observational study covered patients diagnosed with sepsis who were admitted to the ICU between January 2018 and April 2023. Clinical data were recorded within 24 hours after ICU admission, including age, sex, comorbidities, APACHE II score and blood test results.

Results

A total of 446 patients were included. The mortality rate at 28 days was 69.7%. Two hundred and forty-six patients were male (55.2%), 200 were female (44.8%) with a mean age of 72 years. Patients were divided into two groups based on their survival status on the twenty-eighth day of ICU stay: survivors (Group 1) and non-survivors (Group 2). There were 131 (31.3%) and 311 (69.7%) patients in Group 1 and Group 2, respectively. We did not observe any statistically significant differences in terms of Glasgow Coma scale, APACHE II score, C-reactive protein, CAR, PLR and RDW values between groups. PCT was significantly higher in Group 2 compared to Group 1 (p<0.05), and was found to be a significant predictor of 28-day mortality.

Conclusion

Further studies are needed to determine whether RDW, PLR, and CAR scores can effectively predict prognosis in sepsis.

Keywords:
Platelet-to-lymphocyte ratio, C-reactive protein to albumin ratio, red cell distribution width, sepsis

Introduction

Sepsis is a sudden and potentially life-threatening organ dysfunction that poses significant mortality risks, particularly when it is combined with shock and multiorgan failure (1). While the exact mechanisms underlying sepsis remain complex and not completely understood, the disrupted response of the body to infection forms its basis (2). Innovative treatment approaches, as well as timely identification and the utilization of evidence-based treatment protocols, are possibilities for enhancing the outcomes of patients affected by sepsis (3). New dependable, real-time accessible, and practical risk indicators could enhance the management of septic patients by promptly identifying high-risk patients and subgroups. This might facilitate prompt and aggressive treatment, as well as appropriate allocation of intensive care resources (4).

In recent years, various hemogram-derived indices have been suggested for screening and predicting the outcomes of sepsis and bacteremia (5). To enhance screening, assessing these indices in conjunction with established markers of systemic inflammation, including C-reactive protein (CRP), leucocytosis and procalcitonin (PCT), is suggested (6). For better prognostication, these indices could be used alongside established intensive care unit (ICU) scoring systems, such as Simplified Acute Physiology score II (SAPS II), Acute Physiology and Chronic Health Evaluation II (APACHE II), and Sequential organ failure assessment (SOFA) (2). Platelet-lymphocyte ratio (PLR), CRP-to-albumin ratio (CAR), and red cell distribution width (RDW) were selected as biomarkers due to their emerging role in assessing the prognosis of sepsis. Unlike traditional markers such as the SOFA score and lactate, which have been extensively utilized for years, these newer biomarkers may provide additional insights into systemic inflammation and immune response dynamics. Their potential to enhance prognostic accuracy and accessibility in clinical practice makes them valuable alternatives in sepsis research.

RDW is traditionally used for the differential diagnosis of anemias. An elevated RDW indicates disrupted erythrocyte homeostasis and impaired erythropoiesis. Abnormal metabolic conditions, including inflammation, oxidative stress, nutritional disorders, and dyslipidemia, have been reported to be associated with RDW values exceeding the upper limit of 14.5% (7). RDW is a noteworthy predictor of mortality for sepsis (4, 8) in studies examining the relationship between RDW and sepsis morbidity and mortality. Recently, a meta-analysis on the prognostic role of RDW in sepsis, indicated that patients with increased RDW are more likely to have higher mortality (9).

High levels of CRP, triggered by cytokine stimulation under inflammatory conditions, are linked to poor prognosis and increased mortality. Similarly, low serum albumin levels are known to be associated with increased mortality rates. Likewise, studies examining the CAR in the context of systemic inflammation have shown that this parameter is a significant predictor of prognosis in cases of infection and malignancy (10, 11). Furthermore, numerous studies have highlighted the prognostic significance of the PLR in assessing systemic inflammatory responses, showing its ability to represent the intricate interplay between the immune response, coagulation, and inflammation (12).

Nevertheless, the limitations of these researches were geographical and possible publication bias. Hence, more scientific and multicenter trials on the prognostic role of RDW, CAR, PLR in sepsis are still needed. Therefore, this study aimed to evaluate the prognostic value of the RDW, CAR and PLR for predicting the prognosis and mortality sepsis and the feasibility of using any of these parameters with the APACHE II scoring system.

Materials and Methods

Study Design and Setting

This retrospective observational study was performed in the 48-bed Anesthesiology and Reanimation Intensive Care Unit of the Selçuk University Faculty of Medicine. The protocol of this study was approved by the Clinical Research Ethics Committee of Konya Selçuk University (decision number: 2023/261, date: 02.06.2023).

Selection of Patients

The study included patients who were diagnosed with sepsis and admitted to the ICU between January 2018 and April 2023. Sepsis and septic shock patients were grouped within the same cohort. Patients aged <18 years,  pregnant or breastfeeding patients, patients admitted with hematological disorders or active bleeding, or who had received blood and blood products during hospitalization before admission to the ICU or within the first three days of admission to the ICU, and had a length of stay of less than 24 hours were excluded from the study. The RDW, PLR, CAR values, and APACHE II scores at ICU admission were recorded. Patients were divided into two groups on the basis of their survival status on the twenty-eighth day of ICU stay: survivors (Group 1) and non-survivors (Group 2). These two groups were comparatively analyzed for factors contributing to differences.

Measurements

Clinical data including age, sex, comorbidities (diabetes mellitus, hypertension, history of cardiac disease, malignancy, chronic obstructive pulmonary disease, chronic renal failure), and clinical outcomes, including in-hospital mortality status and ICU length of stay, were recorded. Laboratory parameters: platelets, neutrophils, lymphocytes, urea, creatinine, CRP, albumin, pH, lactate, PCT, RDW levels, Glasgow Coma scores, APACHE II scores, CAR, PLR were recorded within 24 hours after ICU admission.

Outcome

To determine the relationship between RDW, PLR, CAR, PCT values, APACHE II scores, and sepsis prognosis, the outcome was considered.

Statistical Analysis

All the statistical analyses were performed using the R statistical language, version 4.2.1 (www.r-project.org). To check the normality of the data, Shapiro-Wilk tests and Q-Q plots were used. The Levene test was used to assess the homogeneity of the variances. Numerical variables are presented as the mean ± standard deviation, medians with ranges (minimum-maximum), or medians with interquartile ranges (IQRs, 1st quartile-3rd quartile), as appropriate. Categorical variables are also described as counts (n) and percentages (%). The demographic and clinical characteristics of the survivors and non-survivors are compared via the Mann-Whitney U test, Student’s t-test or Welch’s t-test for numerical variables, and the Pearson chi-square test or chi-square test with Yates continuity correction for categorical variables. Two-tailed p-values <0.05 were considered statistically significant.

Results

Of the 649 patients in the study, 21 patients with hematological malignancies, 104 patients who did not survive in the first 24 hours, and 78 patients who received blood or blood products in the first 3 days were excluded from the study. Therefore, the study group consisted of 446 subjects aged between 18-99 years who were followed up in the ICU due to sepsis. With a mean age of 72 years, A total of 246 patients were male (55.2%), and 200 were female (44.8%). There were 131 (31.3%) and 311 (69.7%) patients in Group 1 and Group 2, respectively. There were no significant differences in baseline characteristics such as age, sex, or underlying disease between Group 1 and Group 2 (Table 1).

The mean overall APACHE II score and Glasgow Coma score were 25.51±8.28 and 7.75±4.42, respectively, and there were no significant differences between the groups. The 28-day mortality rate was 69.7% (n=311). Among the laboratory findings, the serum urea [64 (IQR: 43-106 vs. 82 IQR: 48.5-116, p=0.011)] and creatinine [1.12 (IQR: 0.69-1.98)] vs. 1.33 [(IQR: 0.86-2.44), p=0.033] levels were lower in non-survivors than in survivors, whereas the platelet counts [(239 IQR: 165-326.5 vs. 209 IQR: 141-294.5, p=0.044)] were significantly higher in non-survivors than in survivors. CAR and PLR levels were similar between survivors and non-survivors. We also did not observe any statistically significant differences in CRP, albumin, lymphocyte, pH, lactate, or RDW values between the survivors and the non-survivors. The median length of ICU stay was 12 days (range: 2-140 days) for survivors and was 15 days (range: 2-450 days) for non-survivors; however, this difference was not statistically significant.

Discussion

The 28-day mortality rate in our study was 69.7%. This value is slightly higher than that reported in the literature. This difference occurred because our study included patients from a tertiary ICU.

In this retrospective study conducted at a single center, we aimed to assess the clinical utility of the PLR, CAR, and RDW in critically ill sepsis patients. Our findings indicated that these parameters were ineffective in predicting early mortality in this context. Although the PLR, CAR, and RDW proved impractical for prognostication, we found that PCT could serve as a valuable supplement to CRP or the APACHE II score in predicting mortality in sepsis patients.

Sepsis is defined by an abnormal host response to infection, leading to changes in the hemostatic system that affect the quantity and function of white and red blood cells, as well as platelets (2). Numerous studies have identified PLR as a novel inflammatory indicator in various disorders, including cancers, atherosclerosis, and acute kidney injury (13-16). In contrast, a study conducted in Türkiye in 2016 reported no significant difference in the PLR between patients with sepsis and those with septic shock (17). Similarly, in our study, the PLR was not significantly correlated with sepsis prognosis. This finding may be explained by several factors. First, the PLR might not be significant because it was monitored only in the first 24 hours and was not checked later. Second, we categorized patients with sepsis without distinguishing between sepsis and septic shock. If we had reassessed them later and evaluated the PLR after diagnosing septic shock or sepsis, we might have found a significant difference between the groups, which were defined as survivors (Group 1) and non-survivors (Group 2).

Several studies have demonstrated a relationship between CAR, prognosis, and mortality in ICU patients (18-22). Park et al. (18) reported that CAR levels within the first 24 hours of ICU admission were significantly linked to 28-day mortality. An association between higher CAR levels and increased 30-day mortality was reported by Oh et al. (19). Similarly, Kim et al. (20) identified the CAR for 180-day mortality in patients with sepsis and septic shock as an independent risk factor. In a Turkish study, the CAR and neutrophil-to-lymphocyte ratio values were found to be associated with 90-day mortality in ICU patients with acute ischemia (21). Bender et al. (22) also reported a relationship between CAR values and mortality in patients with acute intracranial hemorrhage. In contrast, our study revealed no correlation between CAR values and 28-day ICU mortality. Although elevated CAR levels indicate increased inflammation and protein loss, the literature suggests that the CAR can be a prognostic factor for mortality even in non-infected patients. However, our results do not support this finding. The lack of correlation in our study could be due to the small sample size of ICU patients. Additionally, the sensitivity and specificity for predicting 28-day mortality were inadequate. Ranzani et al. (23) reported that the CAR at discharge was associated with 90-day mortality. Had we conducted our study on the basis of 90-day mortality, we might have obtained significant results.

The potential pathophysiological mechanisms of the close association between RDW and mortality in septic patients are not the focus of this study. Thus, we can only speculate about them on the basis of literature (24). RDW is an indicator of anisocytosis and therefore shows variability in erythrocyte volume (24). A study by Cheng et al. (25) revealed that RDW was associated with increasing age. Dankl et al. (4) reported that, septic patients with elevated RDW appeared to be older. However, the relationship between RDW and mortality persisted regardless of age. Notably, Fontana et al. (26) studied 122 septic patients and reported no correlation between RDW and sepsis prognosis or microcirculatory alterations. In our study, increased RDW were not a predictor of poor outcomes among septic patients. Ju et al. (27) reported that despite a single measurement at admission, serial RDW measurements on the first, fourth, and seventh days, and a continuing increase in RDW values are more effective in the prediction of mortality in aged patients with septic shock. We conducted this study on the basis of single RDW levels at ICU admission, and we might have found a significant difference between the groups if we measured RDW values more than once. Another explanation could be related to patient selection; previous studies usually included consecutive patients admitted for different reasons to the ICU, whereas we considered only septic patients.

PCT is the inactive propeptide of calcitonin, released by C cells of the thyroid gland, hepatocytes, and peripheral monocytes. While PCT demonstrates greater specificity for bacterial infections compared to CRP and other conventional markers, its levels may also be elevated in non-infectious conditions (28). Lee et al. (28) studied presepsin, PCT, and CRP prognostic value in sepsis and found that the prognostic value of presepsin was superior to that of PCT and CRP in patients with sepsis and septic shock. According to Kim et al. (29), PCT appears to have a limited capacity to predict sepsis-related mortality. For diagnosing bloodstream infections and bacteremia, studies have shown that PCT has a high diagnostic performance (30-32). PCT has been proven more effective than white blood cell and CRP for distinguishing blood contamination from true bloodstream infection in patients with coagulase-negative staphylococci growth in their blood cultures (30). Additionally, two other studies examined the use of PCT to predict bacteremia in patients with community-acquired pneumonia and urinary tract infections (31, 32).

The diagnostic value of CRP and PCT has been evaluated in multiple registries which have yielded results varying. Silvestre et al. (33) investigated the diagnostic and prognostic roles of CRP in a prospective registry involving 158 patients with sepsis and septic shock. Their investigation revealed no association between CRP concentrations on day 1 and sepsis severity. Additionally, higher CRP levels are not associated with ICU mortality (33). In a prospective study of 349 patients, PCT proved to be more effective than CRP in diagnosing septic shock (34). However, neither PCT nor CRP appeared to have a high predictive value for 30-day all-cause mortality in sepsis and septic shock patients (34). The current study confirms that CRP levels have no prognostic significance in patients with sepsis or septic shock, which is in line with the studies mentioned above. In contrast to CRP, we found that the situation was different for PCT, which is a highly sensitive (p<0.05) parameter for predicting 28-day mortality from sepsis, in the ICU. Schupp et al. (34) investigated PCT from diagnostic and prognostic points of view in sepsis and reported that PCT has poor predictive value for both aspects. The reasons for these different results can be categorized under two different headings. One is that their study was planned prospectively, and ours was retrospectively. Another observation is that they measured PCT values on different days from ICU admission until the tenth day in the ICU, and they observed that the PCT values decreased even if the sepsis persisted. We measured PCT once at the time of ICU admission. A new study can be planned to include repeated recordings instead of single time-point measurements.

Study Limitations

The potential limitations of our study should be taken into account. First, it had a retrospective design. Second, we did not consider the origin of sepsis. Additionally, we had no data on bacterial cultures, including the rate of positive cultures for each patient and the most common bacteria along with their resistance status. This requires further clarification in future research. Third, there could be bias regarding the influence of multimodal personalized treatment, which comprises antibiotics, adjuvant therapy and source control techniques. The lack of association between the outcome measures and the results may be because the results were dependent on the type of treatment. The final limitation is the number of measurements. This measurement plan may influence our results, as we relied on single-time point measurements rather than multiple time points.

Conclusion

On the basis of the findings of this negative study, we may infer that an elevated PCT could be useful for predicting 28-day mortality in sepsis patients. However, the CAR, PLR, and RDW are not associated with mortality in this specific clinical setting, even when evaluated alongside the APACHE II score.

Ethics

Ethics Committee Approval: This retrospective observational study was performed in the 48-bed Anesthesiology and Reanimation Intensive Care Unit of the Selçuk University Faculty of Medicine. The protocol of this study was approved by the Clinical Research Ethics Committee of Konya Selçuk University (decision number: 2023/261, date: 02.06.2023).
Informed Consent: Retrospective study.

Authorship Contributions

Surgical and Medical Practices: Concept: Y.C., Y.Ş.B., Design: Y.Ş.B., Data Collection or Processing: Y.Ş.B., Analysis or Interpretation: Y.C., Y.Ş.B., Literature Search: Y.Ş.B., Writing: Y.C., Y.Ş.B.
Conflict of Interest: The authors declare that they have no conflict of interest.
Financial Disclosure: There are no financial conflicts of interest to disclose.

References

1
Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801-10.
2
Liberski PS, Szewczyk M, Krzych ŁJ. Haemogram-derived indices for screening and prognostication in critically ill septic shock patients: a case-control study. Diagnostics (Basel). 2020;10:638.
3
Marik PE. The management of sepsis: science & fiction. J Thorac Dis. 2020;12:S1-S4.
4
Dankl D, Rezar R, Mamandipoor B, Zhou Z, Wernly S, Wernly B, et al. Red cell distribution width is independently associated with mortality in sepsis. Med Princ Pract. 2022;31:187-94.
5
Orfanu AE, Popescu C, Leuştean A, Negru AR, Tilişcan C, Aramă V, et al. The importance of haemogram parameters in the diagnosis and prognosis of septic patients. J Crit Care Med (Targu Mures). 2017;3:105-10.
6
Boran ÖF, Yazar FM, Boran M, Urfalıoğlu A, Bakacak Z, Yıldız M, et al. The preseptic period and inflammatory markers in the prediction of the course of sepsis. Med Sci Monit. 2018;24:3531-9.
7
Deniz M, Ozgun P, Ozdemir E. Relationships between RDW, NLR, CAR, and APACHE II scores in the context of predicting the prognosis and mortality in ICU patients. Eur Rev Med Pharmacol Sci. 2022;26:4258-67.
8
Dogan P, Guney Varal I. Red cell distribution width as a predictor of late-onset Gram-negative sepsis. Pediatr Int. 2020;62:341-6.
9
Zhang L, Yu CH, Guo KP, Huang CZ, Mo LY. Prognostic role of red blood cell distribution width in patients with sepsis: a systematic review and meta-analysis. BMC Immunol. 2020;21:40.
10
Lee SH, Lee Y, Park JS, Cho YJ, Yoon HI, Lee CT, et al. Characterization of microbiota in bronchiectasis patients with different disease severities. J Clin Med. 2018;7:429.
11
Liu Y, Gao Y, Liang B, Liang Z. The prognostic value of C-reactive protein to albumin ratio in patients with sepsis: a systematic review and meta-analysis. Aging Male. 2023;26:2261540.
12
Qiu X, Wang Q, Zhang Y, Zhao Q, Jiang Z, Zhou L. Prognostic value of neutrophils-to-lymphocytes ratio and platelets-to-lymphocytes ratio in sepsis patients with lymphopenia. Biomarker Insights. 2024;19:11772719231223156.
13
Lu C, Gao P, Yang Y, Chen X, Wang L, Yu D, et al. Prognostic evaluation of platelet to lymphocyte ratio in patients with colorectal cancer. Oncotarget. 2017;8:86287-95.
14
Chawla A, Huang TL, Ibrahim AM, Hardacre JM, Siegel C, Ammori JB. Pretherapy neutrophil to lymphocyte ratio and platelet to lymphocyte ratio do not predict survival in resectable pancreatic cancer. HPB (Oxford). 2018;20:398-404.
15
Gary T, Pichler M, Belaj K, Hafner F, Gerger A, Froehlich H, et al. Platelet-to-lymphocyte ratio: a novel marker for critical limb ischemia in peripheral arterial occlusive disease patients. PLoS One. 2013;8:e67688.
16
Zheng CF, Liu WY, Zeng FF, Zheng MH, Shi HY, Zhou Y, et al. Prognostic value of platelet-to-lymphocyte ratios among critically ill patients with acute kidney injury. Crit Care. 2017;21:238.
17
Duman A, Akoz A, Kapci M, Ture M, Orun S, Karaman K, et al. Prognostic value of neglected biomarker in sepsis patients with the old and new criteria: predictive role of lactate dehydrogenase. Am J Emerg Med. 2016;34:2167-71.
18
Park JE, Chung KS, Song JH, Kim SY, Kim EY, Jung JY, et al. The C-reactive protein/albumin ratio as a predictor of mortality in critically Ill patients. J Clin Med. 2018;7:333.
19
Oh TK, Song IA, Lee JH. Clinical usefulness of C-reactive protein to albumin ratio in predicting 30-day mortality in critically ill patients: a retrospective analysis. Sci Rep. 2018;8:14977.
20
Kim MH, Ahn JY, Song JE, Choi H, Ann HW, Kim JK, et al. The C-reactive protein/albumin ratio as an independent predictor of mortality in patients with severe sepsis or septic shock treated with early goal-directed therapy. PLoS One. 2015;10:e0132109.
21
Kocatürk M, Kocatürk Ö. Assessment of relationship between C-reactive protein to albumin ratio and 90-day mortality in patients with acute ischaemic stroke. Neurol Neurochir Pol. 2019;53:205-11.
22
Bender M, Haferkorn K, Friedrich M, Uhl E, Stein M. Impact of early C-reactive protein/albumin ratio on intra-hospital mortality among patients with spontaneous intracerebral hemorrhage. J Clin Med. 2020;9:1236.
23
Ranzani OT, Zampieri FG, Forte DN, Azevedo LC, Park M. C-reactive protein/albumin ratio predicts 90-day mortality of septic patients. PLoS One. 2013;8:e59321.
24
Salvagno GL, Sanchis-Gomar F, Picanza A, Lippi G. Red blood cell distribution width: a simple parameter with multiple clinical applications. Crit Rev Clin Lab Sci. 2015;52:86-105.
25
Cheng CK, Chan J, Cembrowski GS, van Assendelft OW. Complete blood count reference interval diagrams derived from NHANES III: stratification by age, sex, and race. Lab Hematol. 2004;10:42-53.
26
Fontana V, Spadaro S, Bond O, Cavicchi FZ, Annoni F, Donadello K, et al. No relationship between red blood cell distribution width and microcirculatory alterations in septic patients. Clin Hemorheol Microcirc. 2017;66:131-41.
27
Ju XF, Wang F, Wang L, Wu X, Jiang TT, You DL, et al. Dynamic change of red cell distribution width levels in prediction of hospital mortality in chinese elderly patients with septic shock. Chin Med J (Engl). 2017;130:1189-95.
28
Lee S, Song J, Park DW, Seok H, Ahn S, Kim J, et al. Diagnostic and prognostic value of presepsin and procalcitonin in non-infectious organ failure, sepsis, and septic shock: a prospective observational study according to the Sepsis-3 definitions. BMC Infect Dis. 2022;22:8.
29
Kim H, Hur M, Moon HW, Yun YM, Di Somma S; GREAT Network. Multi-marker approach using procalcitonin, presepsin, galectin-3, and soluble suppression of tumorigenicity 2 for the prediction of mortality in sepsis. Ann Intensive Care. 2017;7:27.
30
Schuetz P, Mueller B, Trampuz A. Serum procalcitonin for discrimination of blood contamination from bloodstream infection due to coagulase-negative staphylococci. Infection. 2007;35:352-5.
31
Müller F, Christ-Crain M, Bregenzer T, Krause M, Zimmerli W, Mueller B, et al. Procalcitonin levels predict bacteremia in patients with community-acquired pneumonia: a prospective cohort trial. Chest. 2010;138:121-9.
32
van Nieuwkoop C, Bonten TN, van’t Wout JW, Kuijper EJ, Groeneveld GH, Becker MJ, et al. Procalcitonin reflects bacteremia and bacterial load in urosepsis syndrome: a prospective observational study. Crit Care. 2010;14:R206.
33
Silvestre J, Póvoa P, Coelho L, Almeida E, Moreira P, Fernandes A, et al. Is C-reactive protein a good prognostic marker in septic patients? Intensive Care Med. 2009;35:909-13.
34
Schupp T, Weidner K, Rusnak J, Jawhar S, Forner J, Dulatahu F, et al. C-reactive protein and procalcitonin during course of sepsis and septic shock. Ir J Med Sci. 2024;193:457-68.