An osteoporosis knowledge assessment instrument – development and validation

Daniela Taneva; Angelina Kirkova-Bogdanova; Marieta Todorova; Veselina Bukova

doi:10.3897/folmed.66.e120535

Abstract

Introduction: The consequences of osteoporotic fractures are extremely detrimental to the individual as well as to society. Adopting effective preventative measures is a top public health priority.

Aim: This paper deals with the development and validation of an osteoporosis knowledge measurement tool.

Materials and methods: The study sample included 335 healthy women aged between 25 and 51 years. The osteoporosis knowledge measurement tool is an adapted version of the osteoporosis knowledge assessment tool (OKAT). To determine the validity and reliability of the tool, we examined the psychometric properties. Nonparametric methods were used for the statistical analysis.

Results: Flesch reading ease index was 55.14. The Cronbach’s α value was 0.884. The corrected item-total correlations varied between 0.340 and 0.611. The items’ mean difficulty was 0.46. The mean discrimination index was 0.61. The mean score of the sample was M=12.64±5.164, a little bit higher than 50% of the success rate.

Conclusion: The tool can be used in different settings to assess educational needs and plan interventions. The results indicate a need for educational and preventive initiatives.

Keywords

evaluation, awareness, osteoporotic fractures, prevention

Introduction

Osteoporosis is a disease with significant social implications that initially presents as an inconspicuous condition but can cause serious complications in later stages. Osteoporotic fractures are a serious public health problem. The consequences of these fractures have major negative economic effects on society as a whole as well as on individuals. They are associated with long-term difficulties in carrying out daily activities, long-term treatment, permanent disability, and job loss. Impaired quality of life and dependence on care provided by relatives or medical personnel leads to social isolation and low self-esteem.^[1]

An important public health priority is the implementation of effective prevention strategies. Bone density before menopause is as important as bone loss after menopause in predicting the risk of future fractures. Aging people can increase their bone density through healthy behaviors that include proper diet, physical activity, and a healthy lifestyle. Women of active age from 25 to 51 are a very important target group for prevention. Results of a study^[2] showed that more than half of the surveyed women over 45 were not informed about the disease, 85.7% did not take enough calcium, and 30.5% had a family history, which is a risk factor for osteoporosis.

There are studies in the scientific literature that assess the risk of developing osteoporosis among women of active age, but the implementation of prevention methods aimed at lifestyle changes requires active work in the group and at the individual level. Calcium-rich foods and exercise are recommended to increase bone density among premenopausal women. Endicott^[3] believes that health education about risk factors and preventive measures for osteoporosis should begin well before menopause. Education campaigns aimed at raising awareness of the condition, its risk factors, and ways to reduce them should target both women active reproductive age and young people in school. Increasing health knowledge is a predictor of motivated engagement for long-term preventive behavior. Planned health education aims at changing attitudes, beliefs, and a greater self-efficacy^[4] and is built upon a valid and reliable diagnostic tool to assess the knowledge of osteoporosis of the targeted population. For this purpose, we developed an adapted version of OKAT^[5] that takes into account the cultural peculiarities and lifestyle of women in Bulgaria.

Aim

This paper deals with the development and validation of an osteoporosis knowledge measurement tool.

Materials and methods

Sample selection and description

The participants, 335 healthy women aged between 25 and 51 years, were selected randomly from the city of Plovdiv and the Plovdiv region. After being informed about the purpose of the study, the women agreed to participate voluntarily and anonymously. A paper-based questionnaire was administered to the participants. The study was approved by the Institutional Ethics Committee of the Medical University of Plovdiv (protocol No. 1/19.01.2023). The opinion of the Committee was that the research meets the standards of ethics and complies with the requirements of the Helsinki Declaration, the principles of good clinical practice, Bulgarian laws, and regulations for conducting clinical and scientific research with the participation of people.

Development of the instrument

Our osteoporosis knowledge measurement tool is an adapted version of the OKAT.^[5] OKAT was translated from English to Bulgarian by a professional licensed translator with experience in translating medical literature. We examined the statements in detail, paraphrased some of them, and left others without change. We added new statements and produced an assessment tool that contained 27 statements. Then we submitted the questionnaire to three experts - a rheumatologist, an endocrinologist, and a specialist in general medicine. On the experts’ advice, we modified some of the questions to avoid ambiguity and improve validity. Each of the items had three answer options: yes, no, and I do not know. Indication of the correct answer was considered a correct response, and incorrect responses are considered those that indicated an incorrect answer and the “I do not know” answer. We assigned 1 point for each correct response and 0 points for each wrong one.

Validation procedure

To determine the validity and reliability of the questionnaire, we examined the following psychometric properties:

Flesch reading ease. We did not find a Flesch reading ease formula validated for the Bulgarian language, so we calculated this index according to the formula proposed by Ivanov et al. ^[6] for text readability in academic texts in Russian as a Slavonic language close to Bulgarian. The scores ranged between 0 and 100 - a greater value meant easier reading and better understanding by people.
Internal consistency measurement Cronbach’s α. We excluded the items that would increase the scale’s homogeneity of omitted. We considered acceptable Cronbach’s α >0.70.
Corrected item-total correlations. Items with negative or low correlation should not be included in the assessment tool, because they do not correlate enough with the scale. A correlation lower than 0.3 was considered negligible. ^[7]
Items discrimination index (DI). We calculated it as the difference between the mean score on an item of the students in the first 27th percentile of scores on all items ranging from the highest to the lowest and the mean score on the same item of the students in the last 27th percentile. DI of 0.40 and up indicates a good item’s distinguishing ability. ^[8]

Items’ difficulty level was defined as the ratio between correct responses and all answers. Higher values mean easier questions. The optimal range was 20-80%.^[8] We accepted items with a difficulty level lower than 0.75.^{[5, 9]}

Statistical analysis

We did not use any software for the determination of the Flesch reading ease index, due to the lack of the corresponding functionality for the Bulgarian language in MS Word. The index was calculated independently by two of the authors. We accepted that the index was accurate when the results of the two separate calculations were identical. We calculated the difficulty and discrimination index of the statements in MS Excel. The rest of the statistical analysis was done in SPSS v. 23. The nonparametric methods of Kruskal-Wallis and Mann-Whitney tests, and Spearman correlation were used for the analysis of ordinal variables and variables that were not normally distributed. Central tendencies were reported with a mean value and a standard deviation (M±SD). We assumed a level of statistical significance α=0.05.

Results

The Flesch reading ease index was 55.14. The Cronbach’s α value was estimated at 0.790. Item-total statistics indicated that removing items 2, 3, 17, and 19 would increase Cronbach’s α value. These items also had negative corrected item-total correlations. We removed items 2, 3, 17, and 19, performed the analysis again, and received Cronbach’s α=0.884 for a scale with 23 items that do not necessitate any omissions. The corrected item-total correlations varied between 0.340 and 0.611. The items’ difficulty varied between 0.23 and 0.66. The mean difficulty of all items was 0.46. The DI was negative for items 2, 3, 17, and 19. For the rest of the statements the DI varied between 0.43 and 0.81, mean D-value 0.61. We removed items 2, 3, 17, and 19 from the scale. The psychometric properties by items of the osteoporosis knowledge measurement tool are presented in Table 1.

The mean score of the sample in the developed assessment scale was M=12.64±5.164.

Table 1.

Download as

CSV

XLSX

Psychometric characteristics of the scale with items 2, 3, 17, and 19 removed

Items	Difficulty	Discrimination index	Item-total correlation
1. Osteoporosis leads to an increased risk of bone fractures.	0.66	0.57	0.496
4. Higher bone density in childhood protects against the development of osteoporosis later in life.	0.35	0.43	0.370
5. Women suffer more from osteoporosis.	0.55	0.67	0.512
6. People with lighter skin color are at a higher risk of developing osteoporosis.	0.23	0.46	0.369
7. Low bone density can be the cause of bone fracture in minor traumas.	0.55	0.81	0.611
8. Most people develop osteoporosis by the age of 80.	0.39	0.67	0.486
9. After the onset of menopause (the cessation of menstruation), most women can expect at least one fracture.	0.34	0.64	0.438
10. A family history of osteoporosis is an important prerequisite for the development of the disease.	0.50	0.81	0.605
11. Smoking can contribute to the development of osteoporosis.	0.37	0.59	0.422
12. Moderate physical activity outdoors protects against osteoporosis.	0.47	0.78	0.569
13. Playing sports in childhood prevents the development of osteoporosis in adulthood.	0.47	0.67	0.487
14. Exposure to direct sunlight for at least 30 minutes a day prevents the development of osteoporosis.	0.55	0.61	0.496
15. The daily intake of milk and milk products supplies the body with enough calcium.	0.58	0.62	0.474
16. Fish is a good source of calcium.	0.63	0.52	0.401
18. Eggs are a good source of calcium.	0.59	0.58	0.473
20. Raw nuts are a good source of calcium.	0.59	0.66	0.556
21. Daily alcohol use suppresses the formation of new bone density.	0.37	0.60	0.444
22. Calcium supplements alone can prevent bone loss.	0.36	0.49	0.406
23. I can determine my risk of developing osteoporosis based on my lifestyle.	0.36	0.57	0.453
24. Hormone therapy contributes to bone loss at any age.	0.30	0.53	0.425
25. It is important to prevent osteoporosis before the age of 40.	0.66	0.60	0.489
26. There is an effective therapy for osteoporosis.	0.42	0.59	0.419
27. Osteoporosis is a treatable disease.	0.37	0.50	0.340

Discussion

We aimed to create an instrument assessing knowledge about osteoporosis validated for the Bulgarian population. We have carefully selected statistical methods to investigate the reliability of a psychometric test for measuring knowledge. In validating a similar instrument, the test-retest method was used.^{[9, 10]} We did not use the test-retest method on purpose because there was a chance that the first filling out of the questionnaire would pique the respondents’ interest, and they would seek more information about the disease and its causes. This would skew the results of a second fill and the statistical results would be biased. In the development of OKAT as a valid and reliable tool to measure knowledge in osteoporosis, the principal component factor analysis was used.^{[5, 9]} We did not perform a factor analysis because of the dichotomous type of the variables. There has been considerable controversy surrounding the appropriateness of using factor analytic techniques for dichotomous variables.^[11] The factor analysis in the case of dichotomous variables will often lead to artificial factors.^[12]

We achieved the set validity and reliability requirements. The questionnaire we created based on OKAT with added statements had a good Flesch reading ease index, 55.14, higher than those reported by Winzenberg et al.^[5] (45) and Tardi et al.^[9] (44). The calculated Cronbach’s α=0.884, after removing four of the statements, showed a good internal consistency of the test, also supported by the corrected item-total correlations, all within the preset limits. Items’ difficulty range was satisfactory, there were neither very easy nor very difficult questions. The differences between mean, median, and mode indicated a normal to slightly easy assessment, which was also confirmed by the average difficulty being slightly less than 0.50. The four reliability-compromising statements were found to have a negative discrimination index. They were removed from the pool. All questions in the final version had a discrimination index within the desired range. This measure shows the item’s ability to differentiate between “good” and “poor” participants. The DI indicates the extent to which the answers to the question are a result of knowledge rather than guesswork. The mean D-value for the tool was 0.61%, which is higher than those reported by Tardi et al.^[9] (50.4%), and Winzenberg et al.^[5] (44%).

The mean test score was 55% of the maximum of 23 points and was higher than the scores reported by Winzenberg et al.^[5] (44%) and Sayed-Hassan et al.^[10] (less than 50%).

Conclusion

The results show that we succeeded in developing a valid and reliable osteoporosis knowledge measurement tool for the Bulgarian population. It can be used in different settings to assess educational needs and plan interventions. The mean score of the sample was a little bit higher than 50% of the success rate for this assessment, which indicates a need for educational initiatives and preventive activities.

Acknowledgments

This work was supported by the Medical University of Plovdiv, via the inter-institutional project No. 10/2022.

Author contributions

All authors have contributed equally to the development, research and writing of the manuscript.

Competing Interests

The authors have declared that no competing interests exist.

References

1. Tarrant SM, Balogh ZJ. The global burden of surgical management of osteoporotic fractures. World J Surg 2020; 44(4):1009–19.

2. Saw SM, Hong CY, Lee J, et al. Awareness and health beliefs of women towards osteoporosis. Osteoporosis Int 2003; 14(7):595–601.

3. Endicott RD. Knowledge, health beliefs, and self-efficacy regarding osteoporosis in perimenopausal women. J Osteoporosis 2013:853531. doi: 10.1155/2013/853531

4. Lin LP, Lai WJ, Hsu SW, et al. Early osteoporosis risks and associated factors among caregivers working in disability institutions: IOF one-minute osteoporosis risk check. Int J Environ Res Public Health 2020; (9):3319.

5. Winzenberg T, Oldenburg B, Frendin S, et al. The design of a valid and reliable questionnaire to measure osteoporosis knowledge in women: the Osteoporosis Knowledge Assessment Tool (OKAT). BMC Musculoskeletal Disorders 2003; 4:17.

6. Ivanov V, Solnyshkina M, Solovyev V. Efficiency of text readability features in Russian academic texts. Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference “Dialogue 2018”. Moscow; 2018: 2–11.

7. Mukaka MM. A guide to appropriate use of Correlation coefficient in medical research. Malawi Med J 2012; 24(3):69–71.

8. Taib F, Yusoff MSB. Difficulty index, discrimination index, sensitivity and specificity of long case and multiple choice questions to predict medical students’ examination performance. Journal of Taibah University Medical Sciences 2014; 9(2):110–4.

9. Tardi P, Szilagyi B, Makai A, et al. The development of a reliable and valid instrument to measure the osteoporosis-related knowledge: validation of the Hungarian version of Osteoporosis Knowledge Assessment Tool (OKAT). BMC Public Health 2021; 21(Suppl 1):1515.

10. Sayed-Hassan RM, Bashour HN. The reliability of the Arabic version of osteoporosis knowledge assessment tool (OKAT) and the osteoporosis health belief scale (OHBS). BMC Research Notes 2013; 6:138.

11. Parry C, McArdle JJ. An applied comparison of methods for least squares factor analysis of dichotomous variables. Appl Psychol Meas 1991; 15(1):35–46.

12. Kubinger KD. On artificial results due to using factor analysis for dichotomous variables. Psychology Science 2003; 45(1):106–10.