Readability scores of the two selected public service information texts by the NSW Government
This morning I googled “nsw corona/covid restrictions”. The top hits all refer to NSW Government websites, including these two: “What you can and can’t do under the rules” and “Public Health Orders and restrictions.”
Both sites are clearly structured with lots of subheadings and dot points. But they are also long and text-heavy, and the font on “Public Health Orders and restrictions” is so small that the page literally looks like small print.
In both cases, you need to be a fairly motivated reader to work your way through them, and “Public Health Orders and restrictions,” in particular, looks more like it is aimed at lawyers than a regular reader.
Readability statistics for both texts confirm this impression. The Flesch-Kincaid Grade Level of “What you can and can’t do under the rules” is 11.0 and that of “Public Health Orders and restrictions 14.2. The Flesch Reading Ease Score of “What you can and can’t do under the rules” is 48.1 and that of “Public Health Orders and restrictions 33.2.
What is readability?
Readability essentially refers to how easy or difficult it is too read a particular text.
There are a number of measurements of readability. For English, the two most famous of these are Flesch-Kincaid Grade Level and the Flesch Reading Ease measures. Both are algorithms that are now built into most word processing software.
The basic idea behind these two measures is twofold:
- The more words per sentence, the less readable a text is.
- The more syllables per word, the less readable a text is.
Consider the examples in the table.
| Words per sentence | Syllables per word | Flesch-Kincaid Grade Level | Flesch Reading Ease | |
| People need air. | 3 | 1 | 1.3 | 90.9 |
| Humans require oxygen. | 3 | 2.66 | 13.1 | 6.3 |
Comparing the two sentences, it is obvious that the first is much easier to comprehend than the second. Although they have the same number of words, the multi-syllabic words of the second sentence require much more precise and specialized knowledge.
Source: Flesch, How to write plain English
In short, to easily read the second sentence, you need to have a much higher level of (a) English language proficiency, and (b) science knowledge than for the first.
The Flesch measurements of readability
The Flesch measurements were developed by Rudolf Flesch, an Austrian-American writing consultant.
The Flesch-Kincaid Grade Level tells us who we can expect to have the requisite knowledge to read a particular text: 1.3 means that a first grader can be expected to read “People need air.” 13.1 tells us that 13 years of formal education are required to easily read “humans require oxygen.”
The Flesch-Kincaid Grade Level follows the US system of school grades from 1 to 10. A number greater than 10 indicates the years of formal education required.
The Flesch Reading Ease Scale is also linked to grade levels and formal schooling. The higher the level, the easier the text. The 90.9 score of “People need air” tells us that the text is very easy. All that’s needed to easily read it is an elementary school education.
At the other end of the scale, a score of 6.5 suggests that a college education is required to comfortably read “humans require oxygen” (As an aside, don’t get too focused on the precise score. Measuring the readability of a 1-sentence text is problematic, and I’m doing it to make a point. All readability scores are useful indicators but there are also obvious problems, which I’ll discuss some other time.)
Why does readability matter?
Let’s now go back to the two public service messages from the NSW Government with which I started this post: one requires 11 years of formal education and the other 14 years of formal education. With a Flesch Reading Ease score of 48.1 and 33.2 both are considered difficult to read, at the reading level of a college student.
This means that a fair number of Australians will have difficulties reading these public service messages.
Australia has compulsory education and, by international standards, a fairly well-educated population, as findings from the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC) show. Even so, these data also show that there is significant variation in the reading abilities of Australians.
| PIAAC Level (2011-12) | Percentage of Australian population | Number of Australians |
| Below Level 1 | 3.7 | 620,000 |
| Level 1 | 10.0 | 1,700,000 |
| Level 2 | 30.0 | 5,000,000 |
| Level 3 | 38.0 | 6,300,000 |
| Level 4 | 14.0 | 2,400,000 |
| Level 5 | 1.2 | 200,000 |
While the PIAAC levels do not directly match the Flesch levels, one thing is for sure, and that is that those Australians who score at or below Level 1 on PIAAC will definitely not be able to read texts that requires more than 10 years of formal education.
In other words, at least 13.7% of the Australian population, probably more, will not be able to read the two key texts about NSW’s Covid-19 restrictions. This amounts to a total of at least 2,320,000 people.
In the interest of full disclosure: these are the readability stats for this blog post
In a crisis where the actions of each and every one of us have a huge impact on the overall course of the pandemic, this is a very large number.
Attention to inclusive communication has certainly increased over the course of the pandemic, and here on Language on the Move we have focused on public health communication to linguistically diverse populations. The same goes for the special issue of Multilingua devoted to “Linguistic diversity in a time of crisis,” which will be published later this week.
Despite greater attention to the important role of inclusive communication, the fact that key information about Covid-19-related restrictions are out of the reach of more than 13.7% of the Australian population – irrespective of whether English is their main language or not – is concerning.
What is even more concerning is that the two examples I have chosen are not exceptional: two recently published research papers about the readability of Covid-19 online patient education materials and public health information websites in various countries found that these materials are generally pitched at a readability level that is too difficult for significant segments of the population.
Conduct your own research
Choose an English language text about Covid-19 and assess its readability by measuring its Flesch-Kincaid Grade Level and Flesch Reading Ease Scale. You can do that by copying and pasting the text into MS Word. Include additional qualitative observations of readability and discuss whether the readability matches the reading levels of the target audience.
To learn more how reading is learned and why some people never become highly proficient readers, despite going through compulsory schooling, watch this lecture:
References
Mishra, Vishala, & Dexter, Joseph P. (2020). Comparison of Readability of Official Public Health Information About COVID-19 on Websites of International Agencies and the Governments of 15 Countries. JAMA Network Open, 3(8), e2018033-e2018033. doi:10.1001/jamanetworkopen.2020.18033
Szmuda, T., Özdemir, C., Ali, S., Singh, A., Syed, M. T., & Słoniewski, P. (2020). Readability of online patient education material for the novel coronavirus disease (COVID-19): a cross-sectional health literacy study. Public Health, 185, 21-25. doi:https://doi.org/10.1016/j.puhe.2020.05.041
Language challenges of the Covid-19 pandemic
Visit here for the full Language on the Move coverage of language aspects of the COVID-19 crisis. The special issue of Multilingua with 14 peer-reviewed research papers about “Linguistic diversity in a time of crisis” will be published later this week; in the meantime, all the papers are available ahead of print here.
This work is licensed under a
These scales immediately reminded me of the “readability analyses” (which include the Flesch-Kincaid test) that are performed in the US after major presidential speeches (https://www.cmu.edu/news/stories/archives/2016/march/speechifying.html), and as long as I’ve known about them I’ve been fascinated with how the various “grade” levels of the speeches are either elevated or bemoaned in the US media. The results aren’t always what you would think! Speeches that score low on the scale are deemed too ignorant by the left-wing media, but not by the right. Often the right-wing media criticises presidents who use language/grammar that score higher because they are seen to be too “elitist”. I’m sure this happens in other cultures as well, but over the last 20-30 years it has become an extremely polarising trend in the US. I know that written and spoken words can’t be compared directly (as Eskenazi says in the Carnegie Mellon article), but the data certainly seems similar.
Wow – that is so interesting that presidential speeches are scored for “readability”. All of course in trend with the fact that politicians no longer lead but just try to get votes …
I found this blog post quite interesting in analyzing texts in terms of the readability.
The following text is a scientific journal article:
McKibbin, W. J., & Fernando, R. (2020). The global macroeconomic impacts of COVID-19: Seven scenarios.
“The attack rates (proportion of the entire population who become infected) and case-fatality rates (proportion of those infected who die) and the implied mortality rate (proportion of total population who die) assumed for China under seven different scenarios are contained in Table 2 below.”
The above quote has been selected to demonstrate the ease vs. the difficulty of reading a particular text.
Words: 44
Flesh-Kincaid Grade Level: 11.1
Flesh Reading Ease Score: 34.9
Average Words per Sentence: 11
Average Syllables per Word: 1.9
Could be broken down into 4 sentences (more simplified).
Above: The reading level is a college student and is deemed as difficult to read as it is a high-level grade.
Thanks, D.L! I would add that this level of difficulty is fair enough in a scientific journal article, presumably aimed at a specialist audience.
Hi Ingrid,
I looked at two very different texts from the NSW Department of Education website.
The first was part of a home-schooling guide for parents and carers, and provided instructions for teaching handwriting to a child of kindergarten age. It was very clear and easy to read, with short sentences and few multisyllabic words, and I wasn’t surprised to find that it had quite a high readability score.
Flesch Reading Ease score = 71.5
Flesch-Kincaid Grade level = 7.4.
https://docs.google.com/document/d/1uVSPghiGeeSLCgb_OfeQGnL81Z1Zb8bbA59ooJ5wxQ/edit#
The second text was part of a guide to NSW school students for term 3 this year, outlining rules for school attendance under the current COVID-19 restrictions. The target audience for this information would presumably be parents and carers from a range of different backgrounds and with varying levels of education and English literacy skills. It was surprising, then, that this was such a dense, wordy text – with sections that seemed unnecessarily complicated to me, such as:
“Where their symptoms continue beyond 10 days, students should provide documentation from their GP confirming their symptoms are typical for their condition.”
“All schools will be vigilant when implementing infection control, physical distancing and personal hygiene protocols to protect the health and safety of students and staff.”
https://education.nsw.gov.au/covid-19/advice-for-families
This text had a Flesch Reading Ease score of 48.8, and a Flesch-Kincaid Grade Level of 11.6 – much less readable than the home-schooling text available on the same website, and aimed at a very similar target audience.
The different levels of readability of these two texts might reflect the different circumstances in which they were developed: the home schooling guide was already in place before the current crisis, and seems to show a lot more planning and forethought than the COVID-19 attendance guidelines.
Thanks, Claire! I suspect that the home-schooling text was written by teachers, and the “Advice for families” wasn’t …
This is the text I found on The Guardian website:
“What is the advice and rules on face masks?
At a national level, the health department said in advice published on 9 July that masks were recommended where community transmission is occurring and where physical distancing was difficult. In Victoria the premier, Daniel Andrews, made it mandatory to wear masks or face coverings – including scarves or bandannas – when in public in the entire state from midnight Sunday 2 August. There are a few exceptions: people with a medical reason and children under 12. Those who have a professional reason “or if it’s just not practical, like when running” are also exempt but they will be expected to carry a face covering at all times “to wear when you can”. Teachers will not need to wear a face covering while teaching, but students will. Teachers and students will need to wear a mask on the way to and from school.”
(https://www.theguardian.com/australia-news/2020/sep/09/australia-best-face-masks-reusable-washable-cloth-mask-where-to-buy-how-wear-advice-reuse-for-covid-19-coronavirus-melbourne-victoria-vic)
– Characters per word: 4.8
– Syllables per words: 1.6
– Words per sentence: 22.8
– Flesch Reading Ease score: 56.5
– Flesch-Kincaid Grade Level: 11.3
The text is composed of many long sentences, which can be difficult to keep track of and can be fatiguing for the readers. The Flesh Reading score indicates that the text is fairly difficult to read. A score of 11.3 in the Flesh-Kincaid grade level indicates that the average students in 11th grade can read and understand the text.
Thanks, Audrey! Goes to show that explaining restrictions and all their exceptions and contexts of application is quite a challenge …
Thank you for the interesting post Ingrid.
I looked on the Australian government COVIDSafe app for a relevant update and found this link:
https://www.nsw.gov.au/media-releases/western-and-south-west-sydney-testing-bzlit
The first sentence summarises some of the essential information:
“Anyone who lives or works in Sydney’s West and South West and has even the mildest of COVID-19 symptoms is being urged to come forward for testing, as mystery COVID-19 cases continue to emerge in the region.”
One interesting aspect of the article is they used a large number of quotes instead of crafting a direct readable message, perhaps to give it validity. As a result of that there are a number of difficult slang expressions which make it even more challenging to understand such as “breaking the back of this wave of transmission.”
Their urgent message could easily get lost, especially for those with lower levels of literacy. A simple direct message would have been so much more effective especially when people’s lives are at risk.
Words: 271
Characters: 1490
Sentences: 9
Paragraphs: 9
Sentences per Paragraph: 1
Words per Paragraph: 30.1
Characters per Word: 4.7
Flesch Reading Ease: 45.7
Flesch-Kincaid Grade Level: 14.3
So, this text came out at 14.3 on the Flesch-Kincaid Grade Level, meaning one would need a college education to be able to fully comprehend it and 45.7 on the Flesch Readability Score which is between difficult and fairly difficult.
For urgent public announcements, where many citizens lives could be severely impacted, it is highly problematic and unsuitable to use such complex language.
It is obvious going forward that there is a dire need for linguists to cooperate with the government to craft messages that are direct, simple and easy to read.
Agree, Ally! In fact, linguistics should be part of the high school curriculum. Everyone needs to understand how language works!
Hi Ingrid!
I chose this article by CBS News titled “Qiagen says its new COVID-19 test delivers results in 15 minutes.”
https://www.cbsnews.com/news/qiagen-covid-test-15-minutes/
Word count: 262
No. of syllables: 453
No. of sentences: 14
No. of syllables per word: 1.7
No. of words per sentence: 18.7
Readability score: 40 – Difficult
I consider the sentences in this article are lengthy and one could lose the connection if not focused. According to the Flesh-Kincaid scale, the readability of this article is aimed at college students. I believe the way this article is presented corresponds to the target audience of this news network, as mainly college students or middle-aged persons with a high literacy level look for this type of content.
Thanks, Alex! Good connection between readability level and audience.
Thank you for this interesting post, Ingrid. I have decided to look at the article from the Japan times. The article title is “Extend support for foreign students in Japan”.
https://www.japantimes.co.jp/opinion/2020/04/30/editorials/extend-support-foreign-students-japan/
“The COVID-19 outbreak in Japan has left hundreds of thousands of university students in a chaotic situation. Lectures that were supposed to start in April only recently began to be offered online. Part-time jobs that helped students to support themselves have vanished, while the incomes of many of their parents have dwindled since the government declared the state of emergency in April.”
Words: 62
Syllables: 98
Sentences: 3
The average of syllables per word: 1.58
The average of words per sentence: 20.66
Readability score: 61
Therefore, this text is plain English based on the readability chart. When conveying important information, to avoid complex vocabulary, such as technical jargon, needs. The text needs to include English speaking and non-English speaking communities. The readers also need to have a standard level of English for understanding the text.
Thanks, Yuta! English media texts from outside the Anglosphere are often good examples of what can be done in terms of readability without sacrificing complexity.
Thank you, Ingrid, for the interesting article.
Generally, I think that articles written in English only hard for language learners but never thought of Australians inability to read before. Of course, everyone needs to get correct and updated information, especially during this pandemic.
I normally read the 9news reports to get Coronavirus news. I find some of them hard to understand sometimes, especially medical term words.
I tried one recent article on the readable checker. The result based on 7readability formulas showed the following:
Grade Level: 10 Reading Level: difficult to read.
Reader’s Age: 14-15 yrs. old (Ninth to Tenth graders)
Total # of words: 2087
Total # of unique words: 738 (35% of total text)
Total # of sentences: 133
Total # of characters: 10025
Average # of characters per word: 4.8
Average # of syllables per word: 2
Total syllables in text: 3399
I believe this kind of news would be very difficult for lower-level learners and readers and it needs to be simplified for those people which seems not fair for them (including me).
Thanks, Enkhzaya! The other side of the coin is, of course, that complex issues may require complex language …
The following text is from the NSW Health website, from the following link:
https://www.health.nsw.gov.au/Infectious/covid-19/Pages/stats-nsw.aspx
“NSW Health undertook a time-limited survey to measure the recovery status of patients with COVID-19 three weeks after the onset of illness, by interviewing the case. Cases reporting resolution of all COVID-19 symptoms were considered to have recovered. Cases who had not recovered at three weeks were called in the following weeks until recovery. By the week ending 11 July 2020, approximately 85% of cases had recovered. See the COVID-19 Weekly Surveillance Report. NSW Health has suspended this survey while focussing on the recent increase in community transmission of COVID-19.”
Words: 90
Sentences: 6
Average Words per Sentence: 15
Average Number of Syllables: 1.8
Flesch Reading Ease: 38.3
Flesch-Kincaid Grade Level: 11.6
Based upon this extract, it can be said that the article is nearing a difficult level for reading due to the high Grade Level it has received.
Another example that texts with monosyllabic words are not necessarily easy to understand. The use of “case” in this text (“by interviewing the case” etc.) is rather unusual and understanding it requires high school science education (so the Flesch-Kincaid Grade Level seems pretty spot on here)
Thank you Ingrid for another great post! I had a look at an article on ABC regarding the Victoria Covid restrictions : “Your Victorian coronavirus restriction questions answered on work, JobKeeper, surgery, masks, travel and kids”. It is rather a long title containing many words and syllables. However, it is fairly straightforward and easy to read and comprehend, I suppose. I found following information as well:
Words: 642
Sentences: 26
Average number of syllables: 1.5
Average number of words per sentence: 24
In order to assess the readability level of text you really need to look at the whole text, not just the headline.
Thank you for providing us with this useful tool for readability of English texts. I took the following excerpt from the official website of the Australian Government Department of Health.
“COVID-19 is caused by a virus. Antibiotics do not work on viruses.
There is no cure or treatment to prevent COVID-19. Studies show an antiviral drug called remdesivir can help very sick adults and young people get out of hospital quicker. This drug now has provisional approval by the Therapeutic Goods Administration (TGA), so it can be used in Australia in some cases. This includes very sick people in hospital who need oxygen or more help to breathe. There is also evidence that a steroid called dexamethasone may lower the risk of death for some people with COVID-19. This includes people who are very sick and need oxygen or are on a ventilator. A number of other possible treatments are being tested.
Some reports suggest hydroxychloroquine can be used to treat COVID-19. The Therapeutic Goods Administration has not approved hydroxychloroquine for treating COVID-19, in Australia.
We are supporting research to find effective treatments for the virus. There is no approved vaccine for COVID-19, but global efforts to develop a vaccine continue.”
Words: 177
Sentences: 13
Average Words per Sentence: 13.62
Average Syllables per Word: 1.65
Flesch-Kincaid Grade Level: 9.2
Flesch Reading Ease Score: 53.4
The text has an average grade level of 9.2 so it should be easy for a 14 to 15-year-old to understand.
For me, the text is fairly difficult to read. There are some words I have never met before, such as remdesivir, dexamethasone or hydroxychloroquine.
Good point that counting number of syllables and words in a sentence is only a quantitative measure. In the end, assessing the readability of a text is more complex. “Man is an obligate aerobe” has a much lower Flesch-Kincaid grade level than “Humans require oxygen” but, I would argue, is much more difficult to understand.
“COVID-19 risk in Pakistan is high. CDC recommends travelers avoid all nonessential international travel to Pakistan. Some examples of essential travel may include traveling for humanitarian aid work, medical reasons, or family emergencies. Older adults, people of any age with certain underlying medical conditions, and others at increased risk for severe illness should consider postponing all travel, including essential travel, to Pakistan.” ( https://wwwnc.cdc.gov/travel/notices/warning/coronavirus-pakistan )
Words: 62
Syllables: 71
Ave number of Syllable: 1.2 approx
Number of sentences: 3
Average number of words/sentence :20.6
Readability score: 85
based on the readability chart, It can be concluded that this text is fairly easy .
This is very interesting to know the readability level of the text quantitatively . It is very important to convey such an important information in easy language. Text needs to cover all English speaking and Non English speaking community. The readers of this text need to have basic level of English.
Indeed. Public service communication should be easy to understand.
Hi to Ingrid and all,
I researched some general info on symptoms and how to avoid getting COVID19 etc. on the Victorian Governments Dept. of Health site. It was an interesting read i.e. engaging, however, it didn’t fair well on the readability test. It was deemed grade 11 or above so in fact it wouldn’t reach as wide an audience as one would hope. It was fairly lengthy so perhaps this played into the readability score somewhat too.
“Information for everyone” (at least everyone able to read it!) https://www.dhhs.vic.gov.au/victorian-public-coronavirus-disease-covid-19
Vic. State Government – Dept. Health and Human Resources
Flesch-Kincaid Reading Ease 47.6
FK Grade Level 11.2
Gunning Fog Score 12.9
SMOG Index 9.9
Coleman Liace Index 13
Automated Readability Index 10.8
Text Statistics
No. of Sentences 157
No. of words 2886
No. of complex words 452
Percent of complex words 15.66%
Average words per sentence 18.38
Average syllables per word 1.66
Regards,
Peter
Thanks, Peter, for looking up a broader range of readability tests! It’s a good exercise that teaches us not to trust our intuition when it comes to readability. For people like us – with a university education – it’s almost impossible to imagine what it’s like to read at elementary level as an adult. What comes closest is maybe the experience of travelling in a country where you don’t read the language (and then imagine no English signage and no English speakers to help you out …)
I looked up an article related to Covid-19 from ABC news. The title of the article is that “Why COVID-19 loss of smell can last so long, and how the symptom could help with coronavirus screening”.
“Smell, which also plays into the flavours we recognise in food, is separate to this. When air flows over the mucous membranes in your nose, chemicals in the air dissolve into your mucous and are detected directly by receptors in the cells that line your nasal cavity (scientists call this the olfactory epithelium), which send signals to your brain. “The olfactory epithelium essentially is hardwired into our brain,” Dr Russell says.”
-Words: 71
-Syllables: 114
-Sentences: 3
-Average syllables per word: 1.6
-Average words per sentence: 23.6
-Readability score: 46
So, this text can be considered difficult to read. Also, this 46 score requires the level of students above “college”.
Nice example of actually teaching readers new vocabulary (“scientists call it olfactory epithelium”)
Hi Ingrid, I never heard about using this material for measuring the level of the text (easy or difficult), a very interesting tool. I have read the article from SBS news about the international students facing on COVID-19 period.
‘For international students, understanding the advice around the coronavirus pandemic in a foreign country can be confusing. Some say their universities have been brilliant in providing support, others say they expect more when it comes to communication and reduced fees.’ And the result of the whole text is
Words: 40
Syllables: 77
Sentences: 3
The average of syllables per word: 1.9
The average of words per sentence: 13.3
Therefore the level of readability score starts around 47 which is fairly difficult to very difficult. However, if we use these scores to compare the level of students (school grades), it will match on college level.
I have a question in comparing the grade. Is the readability scores for a native speaker or general readers?
Good question! Most quantitative measurements don’t take linguistic diversity into consideration, and L1 and L2 readers certainly face different challenges. Having said that, inclusive communication is beneficial for all kinds of readers from diverse backgrounds. So, good readability is helpful all around!
Btw, the “native speaker” construct is controversial. To the degree that it means anything it only applies to spoken language. No one is born with an innate ability to read and write 🙂
I have decided to look at British article from BBC. The one I chose was about Australia: “Covid: Australian anti-lockdown suspect’s arrest draws controversy”. The title itself contains words with many syllables. The findings are as follows:
Word count: 152
Sentences: 9
Average words per sentence: 18.21
Average syllables per word: 1.55
Readability score: 12.27
According to the chart, this article is in the “very difficult” range, however, I found the article quite easy to read and the words used are mostly simple English, what I am thinking is multiple usages of such words as: “Australia” (3 syllables), “coronavirus” (5 syllables) may have influenced the results a bit.
Thanks, Moni! Good lesson that readability is one of those areas of language where our intuition really doesn’t get us very far. What is easy for me may be difficult for the next person, and vice versa.
Dear Ingrid ,
I checked readability for the following text from NSW health on school advice:
Advice from Dr Kerry Chant, Chief Health Officer NSW Ministry of Health
14 August 2020
In response to the recent cases of COVID-19 in school staff and students in NSW, further measures are being put in place to minimise the risk of COVID-19 transmission in school communities.
These measures will be effective Wednesday 19 August 2020.
NSW Health requires schools to:
• exclude students and staff with even mild symptoms of COVID-19
• encourage immediate testing for any symptomatic children or staff
• prohibit return to school for anyone with symptoms of COVID-19 until a negative COVID test result has been reported
• ensure that adults always maintain physical distancing
• ensure good hand hygiene at all times (e.g. upon entry to the school, entry to the classroom and upon exit of the classroom)
• for activities that result in the mixing of students and staff from schools from the local area (e.g. local inter-school sports competitions), develop and ensure compliance with a COVID safety plan based on the COVID safety plan template for community sport activities
• for local activities, limit spectators to one parent only, where parental supervision is required
• carefully consider face-to-face professional learning for staff and, if proceeding, ensure COVID safety through physical distancing and limiting group numbers, and
• avoid professional learning activities that require staff to mix across schools.
Readability statistics:
Words: 217
Characters:1190
Paragraphs: 14
Sentences: 3
Averages:
Sentences per paragraph: 1.0
Words per sentence: 17.3
Characters per word: 5.2
Readability:
Flesch reading ease: 46.0
Flesch Kincaid Grade level: 11.1
Passive sentences: 33.3%
Readability is 46.0 which is considered as difficult to read! Therefore, it will be difficult to read for level 1 and below level 1 literacy level population!
Thanks, Nusrat! Of course, teachers and principals – i.e. the target audience of this text – can be expected to have these kinds of literacy levels. So this looks like a good match between readability and target audience.
Thank you for teaching this interesting and useful tool for readability of English texts. I assessed a text from https://www.9news.com.au/coronavirus below.
“Coronavirus disease ‘COVID-19’ is a type of coronavirus that caused a global outbreak. The disease causes respiratory illness and symptoms include pneumonia and bronchitis. It can be caught from viral particles and be detected through cough, difficulties to breath and fever. These symptoms can appear as far as 14 days after exposure.”
Words: 52
Syllables: 91
Sentences: 4
Average syllables per word: 1.75
Average words per sentence: 13
Flesch Reading Ease: 42.3
Flesch-Kincaid Grade Level: 10.5
Therefore, the readability score of this text is around 40 which can be recognised as “difficult” to read.
Thanks, Kyohei! I was surprised by your finding because I would have expected this site to pitch their texts at an easier level …
This is interesting. Usually my thought about an article or text from government is ‘will everybody get the chance to read it’ as in my country our government’s social media platform are not very well-known for some reason; however, your article take my wondering to another level which is ‘if people are able to access the information but can they really understand it’. Leading to how nice it would be for the formal text from the government to make it more readers’ friendly and for all level of reader, especially for covid information becuase everyone, not just literate people, together can help stop spreading the virus.
Thanks, Vichuda! You are right – readability is but one of many considerations when it comes to making information accessible to mass audiences.
I took the following excerpt from WHO regarding potential treatment for COVID19:
“Hydroxychloroquine or chloroquine, a treatment for malaria, lupus erythematosus, and rheumatoid arthritis, has been under study as a possible treatment for COVID-19. Current data shows that this drug does not reduce deaths among hospitalised COVID-19 patients, nor help people with moderate disease.*
The use of hydoxychloroquine and chloroquine is accepted as generally safe for patients with malaria and autoimmune diseases, but its use where not indicated and without medical supervision can cause serious side effects and should be avoided.”
My calculation on the readability of the text is shown below:
– Words: 79
– Sentences: 3
– Syllables: 153
– Average number of syllables per word: 26.3
– Average number of words per sentence: 1.94
-> The readability is approximately 13 and the levels fall under college graduate level
With the above result, it is apparent that the text is complicated and demanding for not only the general reader but also those at college level. The primary rationale for this because the text contains many specilaized vocabularies and complex sentence structures. Hence, people with insufficient literacy knowledge might not comprehend the delivered information easily.
Thanks, G! That’s right but important to keep the target audience in mind, too. WHO does not generally target a mass audience but governments and medical authorities. The text looks like it’s suitable for such specialist audiences.
Thank you for this post, Ingrid. The Flesch measurements are really valuable and realistic tools to measure the readability of texts. This makes me think about whether news writers on Covid-19 are aware of these tools and the readability of their texts. I applied these tools to an article called “Australia’s state by state coronavirus lockdown rules and restrictions explained” from theguardian.com.
“As of Friday 7 August, residents returning from Victoria will be required to go into mandatory hotel quarantine for 14 days. From Saturday 8 August, Queensland has closed its border to people from NSW or the ACT and anyone who attempts to enter without a permit will be turned away at the border. Residents can travel to Tasmania if they haven’t been in a designated hotspot, but they will be required to undertake government-supervised quarantine. Only those with exemptions can travel to Western Australia. Residents traveling to South Australia will be required to self-isolate for 14 days. Those traveling to the Northern Territory who have been in a declared hotspot will be required to undertake government-supervised quarantine at a cost of $2,500 per person. The entire greater Sydney region is now classified as a hotspot.”
– Word count: 135
– Sentences: 7
– Average Words per Sentence: 19.28
– Average Syllables per Word: 5.17
– Flesch-Kincaid Grade Level: 11.9
– Flesch Reading Ease Score: 44.9
The text, therefore, can be considered as difficult to read and can be read by people who received approximately twelve years of formal education.
Thanks, Lilly! I think most media do put considerable research into trying to understand their audience, and journalists have a pretty good idea of the level they are aiming at.
I read one piece of news from ABC NEWS. Here are some excerpts from me.”Over 26.2 million people across the globe have been diagnosed with COVID-19, the disease caused by the new respiratory virus, according to data compiled by the Center for Systems Science and Engineering at Johns Hopkins University.” Basically, the complexity of other sentences in this news is similar to the above citation. Therefore, in my view, it is not easy for all of people to go. It needs kind of specialized knowledge to be understood. Their target audience may be someone who has been trained by formal schooling for 10 years at least.
Thanks, Han! Did you try to get some actual readibility statistics?
Yes. However, I am sorry about that this one has already buried by another latest news so that it cannot be found any more. Accordingly, I chose to analyse the citation as an example. In this sentence, if I could regard a clause as a ‘sentence’, I would give the findings as below:
average words per sentence: 12
average syllables per words: 3.2
readability score: 55
reading level:10th to 12th grade (high school)
Hi Ingrid. First of all, I did not know that there is a model for measuring readability of text. I practice on using the readability measurement by measuring the first paragraph of your post (This morning I googled “nsw corona/covid restrictions”. The top hits all refer to NSW Government websites, including these two: “What you can and can’t do under the rules” and “Public Health Orders and restrictions.”). These are what I found
Words: 34
Syllables=53
Sentences: 2
Syllables per words= 1.5
Words per sentence= 17
The readability score is around 42, which is from fairly difficult to difficult.
My question regarding the readability measurement tool is whether it is applicable for other languages. Or is there any tool that we can use to measure other languages?
Thank you
Good question, Yudha! Some other languages have similar tools – you’ll probably have to do your own research what is available in the particular language you are interested in 🙂
Fascinating. I went to ABC to read through some of their articles regarding Covid-19, and I found it interesting to see how the different articles keep contradicting each other. One article may say things are getting better in Australia – while right under another one says we are still to expect the worst. What are people supposed to believe, when even the same news outlet is giving contradicting information all the time? It really disrupts the reliability the population will have for them if this keeps going.
Thanks, Chris! Already back in March the WHO said that the pandemic was accompanied by an “infodemic” – too much information, often conflicting – and that the infodemic was just as dangerous as the pandemic. As time goes on, we can see how true that is and how the crisis response – and whole societies – are fracturing because of a deep erosion of trust in public communication.
I copied a piece of news related to COVID-19 from CNN official website: “The University of Missouri has at least 424 active student Covid-19 cases, according to the school’s Covid-19 dashboard. These numbers are for the Columbia, Missouri campus. A total of at least 591 student cases have been reported since the university began receiving data from the Boone County Department of Public Health and Human Services on Aug. 19. Classes at the University of Missouri resumed Monday, August 24. Christian Basi, Director of the University of Missouri’s News Bureau, told CNN that today’s active case number only increased by nine new cases from yesterday. The university hopes this significant downturn in new active cases is indication they have turned a corner.”
Here is my research with respect to the readability of the text:
Flesch-Kincaid Grade Level: 11.7
Flesch Reading Ease Score: 38.7
Reading Level: College ( Difficult to read )
Average Words per Sentence: 15.6
Average Syllables per Word: 1.8
Sentences: 7
Words: 109
Therefore, readers who have got at least 11 years of education are able to easily read the text. Given that the higher the score, the easier the text, the ease score of this text is 38.7, so that it is difficult to read. Flesch grade level and ease scale are very interesting and helpful as I never heard about it before. I always think reading articles and news is very easy for English native speakers, but it seems that it is also difficult for them to read.
Thanks Ingrid for sharing it to us!
Thanks, Yuan Li! Really sobering to think that 30-40% of Americans don’t read English well enough to comfortably read the news on the website of one of their main TV networks …
Hi Ingrid! The lecture and blog this week puts many things in perspective. I looked at a section of the NSW Health website where there are instructions regarding what to in order to get tested for COVID19. The readability score was just above 90, thus categorising it as very easy. The reason I chose to look at this is because a few days ago, an ESL student e-mailed me asking how to get tested for COVID19.
We are not allowed to speak to students face to face, so all my communication with students is via Zoom or email. My first thought was to send them the link to this website. I looked through the website first and realised that the student was going to have issues navigating this website. I had communicated with this student before and was aware that they generally have comprehension issues. I did not think this met the readability requirements for this student and I would think, many other ESL students. This student would get confused regarding the instructions about how to find a location using suburb/postcode. Many of our students get confused saying their address. For example, if you ask them about suburb/postcode, they start naming street or unit number. They can read all the numbers and letters but often have trouble telling you where they live. So, I took screenshots of each step. Then, I had to get the student’s address from our system so I could tell them their postcode and how to search using postcode. If the student was sitting with me, I could have given them an exercise in independence by letting them go through the website on their own. However, in the context of the pandemic and the urgency with which the student required support, I had to follow those steps. I looked at my email and realised that I had written down a set of instructions for the student to follow instructions!
Thanks, Tazin! This is such an illuminating account because it tells us that readability scores are only one indicator of whether a text may be suitable or not. Your account shows how essential teachers and guides are to help everyone get the information they need. After all, you can simplify a text only so far …
True indeed. I went to the NSW Government websites a few times to look for some texts for my translation classes and had to leave after 5 minutes getting lost there. I also asked myself who would be able to read or find the information they need with such a maze. Thanks for a very interesting analysis.
Thanks, Van! Agree that the information architecture is yet another, and maybe even greater, problem area.
I looked at the COVID19 Signs and Symptoms section of the SBS website. The reader is provided with an overview of the most common symptoms using easy to read sentences and dot points. I thought the text was well organised and easy to understand. My calculations showed the following:
Words: 122
Sentences: 4
Average number of syllables: 1.3
Average number of words per sentence: 30.5
This gave a a Flesch readability score of 65 which puts this text within the Plain English section of the readability chart. This matches my initial feeling that this text is relatively easy to read due to the use of simple language and a clear structure. However, accessing the text required me to have a computer, internet access, to know what the SBS is, and that they have a good website that is likely to provide basic, easy to read information about the current COVID-19 situation. I knew how to look for the information I needed because I had prior knowledge and could read everything else on the website with ease. Anyone with poor English literacy or a lack of knowledge about where to look for information would have had far more difficulty accessing this information.
Important point that accessibility is far more than readability!
The following text is from the document ‘Guidance on Preparing Workplaces for COVID-19’ of Occupational Safety and Health Act of 1970 (2020). It is a short paragraph about the symptoms of COVID-19:
“Infection with SARS-CoV-2, the virus that causes COVID-19, can cause illness ranging from mild to severe and, in some cases, can be fatal. Symptoms typically include fever, cough, and shortness of breath. Some people infected with the virus have reported experiencing other non-respiratory symptoms. Other people, referred to as asymptomatic cases, have experienced no symptoms at all. According to the CDC, symptoms of COVID-19 may appear in as few as 2 days or as long as 14 days after exposure.”
I have carried out an investigation into the readability of this text via MS Word. Here are some of the results:
– Word count: 80
– Sentences: 5
– Words per sentence: 16
– Flesch reading ease: 48.8
– Flesch-Kincaid grade level: 10.4
Based on the readability chart, it can be concluded that this text is DIFFICULT to read. Although this text is applied to all US workplaces including a wide range of audience of different levels of education (workers, employers, employees, clients, customers. etc.), the audience is required to have around 10 years of formal education to understand the text. Therefore, it seems quite challenging for those who do not have much vocabulary about medicine (non-respiratory, asymptomatic cases, exposure) to make sense of the text.
Thanks, Banie! Great job 👍🏼