A Statistical Analysis of Vietnamese High School English Test Score Data
This statistical study, perhaps the first of its kind in Vietnamese academia, has tried to understand the patterns in high school students’ pre-college national exam test scores which hold tremendous significance in students’ college or university admissions.
English belongs to one of the three mandatory subjects (the other two being Mathematics and Literature) where students must be tested as part of their overall evaluation. However, since English is a foreign language for the learners, many high school students, especially those from rural areas, struggle to attain an acceptable level of proficiency.
Meanwhile, a good proportion of students, particularly those in urban areas and with their own substantial financial resources, do attend extra tutorial classes to gain an edge in English proficiency. In this study we have analyzed the provincial mean and median test scores over a period of three years (2019, 2020 and 2021).
It has been noted that a province’s aggregate performance depends heavily on -(i) theprovince’s per capita gross domestic product (PCGDP), which measures a province’s economic prosperity; and (ii) the average number of students per high school (ANSHS) which indicates the overall competitive environment the students encounter which helps them sharpen their language proficiency.
1. Introduction
1.1. Some recent facts.
Every year, as observed over the last five years, approximately a million Vietnamese students sit the National High School Graduation Examination (NHSGE) which has a strong implication for their college and/or university admissions. NHSGE takes place in late summer, and each student must take tests in the mandatory trio -mathematics, literature and a foreign language. In the foreign language category even though the students have other options like Chinese, French, German, Japanese and Russian, it is estimated that about 99% of the students end up learning English, and subsequently take the English test during NHSGE, performance in which then becomes a part of the national dialogue, causes plenty of heartburn and soul-searching.
On a scale from 0 to 10, the national average English score in 2020 fell to 4.58, below the halfway mark, and thus causing a great deal of consternation among the students, parents as well as the educators nationwide. The 2020 results had even more depressing information to ponder over. For example, a staggering 63.1% of the examinees scored below the average score (i.e., 4.38), and a little more than 50% of the students made an average score of 3.4. Even though the 2020 average score of 4.58 was a tad above the 2019 average score, still it was the worst compared to other subject areas; such as -the average History score was 5.19, the average Math score was 6.68, the average Literature score was 6.62, and the average Civic score was an impressive 8.14.
While the 2020 average score (of 4.58) may look depressing, it also belies another profound fact,that is, the ‘urban -rural divide’. The top four areas (or provinces, in terms of highest score) were the heavily urban (and/or industrial) ones -Ho Chi Minh City (HCMC), Binh Duong, Ba Ria -Vung Tau, and Hanoi, with HCMC having the highest average score of 5.85. It is not at all surprising that the provinces which are mostly agrarian and less industrialized, such as the northern mountainous ones, likeHoa Binh, Ha Giang and Son La, achieved the lowest scores. The glaring regional differences were also observed in the most recent 2021 English test scores.
The English score distribution (see the following Figure 1 (a)) showed two distinct peaks, one around 3.80, and the other around 9.00. The lower peak perhaps represents the predominantly rural students with less resources, and the higher peak represents the more resourceful and advantageous students whose families could afford to send them to better schools or provide them with extra private tutoring which happens mostly in the urban areas. The following three figures (Figure 1 (a) – (c)) depict the overall picture of the national English test performance through the relative frequency histograms of the scores for the three most recent years.
1.2. The ground realities.
English is now a compulsory subject taught from the third grade in Vietnam. Before 2010, students were introduced to this language from the sixth grade. Obviously, since 2010, some improvements have taken place, however minimal they might be. In 2014, the Ministry of Education and Training (MOET) adopted a six-level foreign language proficiency framework compatible with the levels in the Common European Framework of Reference for Languages (CEFR). Students graduating from high school are required to achieve the A2 level so that they “can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.; deal with most situations likely to arise while traveling; produce simple connected texts on topics which are familiar or of personal interest, describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans” (Vietnam Ministry of Education and Training, 2014).
On surface, the NHSGE English test itself is believed to be not adequate to evaluate the students’ proficiency as it comprises fifty multiple-choice questions and does not evaluate a student’s speaking, listening or writing skills. Students and parents often point fingers saying that the existing curricula often followed in secondary and high schools seem to be outdated, lack a clear understanding of the fast changing globalized work environment, and under-prepare the students to meet the international standards. Further, due to the lack of investments, the rural schools mostly face the brunt of not having qualified English teachers.
Learning English, having a high proficiency and doing well in the NHSGE English test also highlight the glaring social divide between the ‘haves’ and ‘havenots’. Affluent families, especially in large urban areas, such as HCMC, Hanoi and Hai Phong, send their children to expensive private after-school tutorial centers which often hire foreign tutors from native English speaking countries. On the other hand, pupils in remote rural areas rely mostly on the school time for English learning. Not surprisingly about two-third of these students score below 5, indicating insufficient basic proficiency. If this trend is allowed to continue without any immediate interventions from the national, provincial and local policy makers, then the urban-rural divide will continue to expand which may bring more unintended social maladies. Those living in cities, and generally proficient in English, will gravitate more toward those jobs and higher education where English plays a decisive role, thereby effectively excluding those from rural areas, and hence possibly widening the aforementioned social inequalities.
1.3. The objective of the study.
Though the above two subsections provide plenty of anecdotal evidence that something is amiss in English teaching in Vietnamese schools, and there is a glaring disparity between the rural and urban students in terms of the NHSGE English test score, it is about the high time that we employ some rigorous mathematical / statistical approaches to study this aspect which touches the lives of hundreds of thousands of students. Our objective is to look at the students’ collective performance at a more granular level, that is -the provincial level, and see what factors are contributing to this performance.
1.4. The existing research gap, the research purpose, and the research questions.
(1) The purpose of this research study is to collect the existing data over a period of three years (2019 -immediately pre Covid-19 pandemic, 2020 -during the height of Covid-19 pandemic, and 2021-immediately post Covid-19 pandemic) and observe the patterns through aggregate provincial English test score data. Not only did we (the researchers) want to see the discernible trends in the data, but also figure out if these trends could be explained by some key factors. In the absence of any existing quantitative study (relevant to Vietnam), we tried a few measurable variables (or, factors) which are thought to hold some significance in explaining the aggregate provincial trends.
(2) The research questions centered around not only identifying the key factors (if any) affecting the aggregate provincial trends, but also quantifying their impacts, so that the policy makers could focus on those factors to enhance the overall students’ performance.
2. Literature review
As indicated earlier, to the best of our knowledge, there does not exist any rigorous statistical research to understand the Vietnamese high school students’ English test performance. Any substantial educational policy change and/or initiative must follow a thorough understanding of the ground realities. Otherwise, it may result not only in wasting the precious resources, but also may cause more harm than any good.
3. Materials and Methods
(1) Vietnamese National High School Examination dataset in 2019, 2020, 2021 (English score); and (2) The databases on the aforementioned variables were extracted from Vietnam Statistical Year Books (General Statistics Office of Vietnam, 2019; 2020; 2021).
To understand the high school students’ English test score better, the list of 63 provinces of Vietnam (which includes 59 designated provinces and four major cities each of which enjoys the status of a province) is presented first. Tables A.1 and A.2 in the Appendix show the list of all the provinces along with the mean (i.e, average) and the median (i.e., the central-most value) English test scores for the three recent years (i.e., year 2021, 2020 and 2019). In our quest to quantify the provinces’ overall English test performances, we have studied several possible explanatory variables data on which were readily available. Ultimately, it has been found that two variables as described below have had the most significant contributions:
(1) Per Capita Gross Domestic Product -PCGDP: monthly average income per capita (unit: 10 million Vietnamese Dong (VND)); and (2) Average Number of Students per High School (ANSHS).
Yearly, the PCGDP of a province is calculated by the province’s official GDP figure divided by the provincial population size. On the other hand, a province’s ANSHS is determined by a province’s total number of high school students (taking the English test) divided by the total number of high schools located in that province.
4. Results and Discussion
4.1.The Macro-level Big Picture: Descriptive Statistics
Remark 4.1. Note that Table A.1 and A.2 go from 1 through 64. This is to be consistent with the earlier numbering of the provinces. Also, the item-20 on those lists is missing because the province (Ha Tay) has been merged with the city (“province”) Hanoi thereby resulting into 63 current provinces.
Figures A.1 – A.3 in the Appendix show the more descriptive box-plots of all the 63 provinces for the three years mentioned above. For each province, the corresponding horizontal line shows the range of most of the students belonging to that province and the horizontal box represents the middle half (the middle 50%) of the group ranging from the first quartile to the third quartile with the median value (shown with a cut mark) in between. The few dots above and below each box, toward the end points (or boundaries), indicate the potential outliers. It is evident that the remote rural provinces are at the lower spectrum while the predominantly urban ones are at the higher end.
Remark 4.2. See the tell-tale sign of differences between what happened in 2019-2020 and in 2021. First of all, Figure 1 clearly shows that while there was a slight bump under the right tail (of 2019 and 2020) indicating the nationwide high performing students, it became more pronounced in 2021. This is because -even though the Covid-19 pandemic started affecting the rest of the world beginning January 2020, it barely touched Vietnam in the entire 2020 due to its strong initial public health measures. But Vietnam couldn’t escape the second wave of Covid-19 Delta variant that ravaged the country starting in late March 2021 which witnessed the country going into a complete lockdown and bringing the economic activities, especially in the south, to a virtual standstill. Only in late September 2021 the country started lifting the lockdown slowly, in a phased manner, to ease the economic hardship. This second wave of Covid not only caused a large number of casualties for which the country was ill-prepared, but also impacted the academic activities, which went fully in remote mode of instructions, thereby negatively and disproportionately affecting the rural students. While urban and well-to-do families weathered this pandemic storm over six months (March to September, 2021) relatively better, the rural and/or financially struggling families were not so fortunate. Notice that the first four provinces (1-4) pretty much stayed ahead of all others as a whole during the three years of the study period. In 2019 and 2020, almost all provinces had a bunch of over-achievers (marked by the dots along the vertical line, indicating their positions as potential outliers within each respective province) which vanished in 2021 (except for a handful few). In the 2021 national exam, which took place right in the middle of the heavy pandemic lockdown, the first four provinces (Hanoi, Hai Phong, Da Nang and HCMC) -the four major (and perhaps privileged) urban centers with significant populations (with almost a fifth of the country’s total population) pulled ahead of the rest of the country with their relatively better financial footing. Most likely this resulted in a clear double peak in the 2021 test score histogram (see Figure 1).
Remark 4.3. We also looked at other possible covariates such as, -the number of teachers per thousand students (essentially “teacher-student ratio”), number of teachers per high school, etc., to explain the provincial mean and/or median test score, but our subsequent data analyses didn’t find them having any significant contribution, and hence not worth pursuing.
The rest of the paper is organized as follows. In Section 4.2, we present our multiple linear regression models which establish a direct relationship between a province’s mean English test score (PM-ETS) with its PCGDP and ANSHS. The same has been done fora province’s median English test score (PMd-ETS). Our rigorous statistical model verification leads to an improved (and more technical) model which can provide a better future prediction, and this has been covered in Section 4.3. Finally, the paper ends with some comments and recommendations.
4.2. The Micro-level Granular Picture: Connecting the dots through Regression
As mentioned in the previous section, the mean (or average) English test score of a province (i.e., PM-EST) can be explained to some extent by the two explanatory variables, namelyPCGDP and ANSHS (see the paragraph before Remark 4.3). (We have done a similar modeling for the provincial median English test score, i.e., PMd-ETS, but to keep our focus fixed we have explained the matter in detail for PM-ETS.) To see the effect of each of these two explanatory variables on PM-ETS (‘Provincial Mean English Test Score’), the following figures are instrumental. Figures 2 shows the plots of PM-ETS against PCGDP as well as 𝑙𝑛(PCGDP). When a variable is widely dispersed, a natural logarithm (‘𝑙𝑛’) transformation can bring it ‘in line’ by condensing its scale, and may offer a better linearity when another (dependent) variable is plotted against it.
Remark 4.4. We believe that the student’s overall performance in a province (as expressed through PM-ETS) in a particular year is being determined by the PCGDP of the previous year. That is why Figure 2 shows the plots of ‘PM-ETS 2021’ against ‘PCGDP 2020’ as well as ‘ln(PCGDP) 2020’. On the other hand, Figures 3 shows the plots of ‘PM-ETS 2021’ against ‘ANSHS 2021’ and ‘𝑙𝑛(ANSHS) 2021’. As such, ANSHS is not expected to change drastically from year to year, but yearly PCGDP can alter greatly as we have witnessed before and during the pandemic time.
As Figures 2 – 3 show and our more detailed statistical analyses have revealed, there is not much gain in using the logarithmic transformation of the variables PCGDP and ANSHS. Therefore, for the sake of simplicity and easier comprehension, our regression models have used the original explanatory variables, i.e., based upon the plots in Figures 2 and 3.
Not being able to satisfy the normality assumption conclusively as mentioned above is a partialsetback for the regression model since all subsequent statistical inferences (such as interval estimation of the regression coefficients and/or undertaking subsequent hypothesis testing) depend heavily on this assumption. However, as far as the model fitting is concerned, the estimated parameters are obtained by the least squares method. Perhaps this setback is not surprising given the fact that the residual values do show a bit skewed form as the following Figure 4 shows.
Therefore, to improve over the model (4.2), we propose a different quantitative measure of a province’s English test performance. This is done by using the ‘Provincial Median English Test Score’ or PMd-ETS instead of PM-ETS. When a variable is asymmetrically distributed, its median is a more stable measure of the center than the mean. This is because the mean is more prone to be affected by the extreme values than the median. We repeat the model (4.1) where 𝑌now plays the role of PMd-ETS with everything else remaining the same. Again, the full model can be reduced further by dropping the insignificant interaction term.
The model assumption tests for 2021 PMd-ETS yield the following results: the normality assumption turns out to be inconclusive again as the p-values for ADT and SWT come out to be 0.20 and 0.004, respectively. However, as for the equality of variances assumption, the standard test (Modified Levene’s test (MLT)) produces a large p-value of 0.378, thereby retaining the homoscedasticity assumption.
The residual values for 2021 PMd-ETS do not seem to be normally distributed as the same pattern persists. The following Remark 4.5 and Table 1 summarizes our findings regarding PM-ETS and PMd-ETS.
Remark 4.5. For the years 2021 and 2020, the use of PMd-ETS seems to work a little better than the corresponding PM-ETS as far as the regression model fitting is concerned. Yet this is not so for all the three years as their performances tend to reverse in 2019 which we will see later. Therefore, we present the fitted regression models for both PM-ETS as well as PMd-ETS with all the details in Table 1.
Remark 4.6. For the normality assumption, it has been noted that ADT and/orSWT are yielding extremely small p-values for all the three years. Both the tests do not provide sufficiently large p-values simultaneously, not even for a single year, which could have accepted normality conclusively. This implies that the normality assumption is not tenable. However, for the equality of variances assumption, the standard test (Modified Levene’s test (MLT)) produces a large p-value, thereby retaining the assumption.
Remark 4.7. The overall micro-level trends revealed by the results presented in Table 1 is quite interesting. Note that the estimated coefficients of 𝑋1 and 𝑋2 are both positive. This means that a province can improve its mean (as well as median) English test score by improving its per capita income (i.e., enhancing the overall economic prosperity) as well as increasing the number of students per school. Both ofthese characteristics have already been seen in the top ranking four provinces (see Tables A.1 and A.2). Also, as the number of students goes up in a high school, contrary to the common fear of having stretched resources, it actually creates a more competitive academic environment which perhaps propels the students to excel more and thereby improving the overall aggregate English test score.
5. Conclusion
In this study we have tried to explain the aggregate English test performance of a province(as measured by the provincial mean English test score (PM-ETS) as well as theprovincial median English test score (PMd-ETS). Interestingly, two key factors have been found to account for nearly half of the total variability (from province to province) of that aggregate performance.This has been demonstrated by the analysis of the English test score (ETS) data over aperiod of three recent consecutive years using asuitable multiple linear regression model.The two factorswhich appear to play important rolesare essentially measuring the overall economic prosperity of a province (as measured by PCGDP), and the overall academic competitiveness faced by the students (as measured by ANSHS).Both of these two factors (or covariates) are positively associated with theprovincial aggregate English test performance. Inother words, as PCGDP and/orANSHSfor aprovince go(es) up (or down), the aggregateprovincialperformance also goes up (or down).
Purely on a technical side, our studyhas been divided in two components: one in the form of a multiple linear regression model under the usual normal error assumption (covered in sub-section 4.2), and another in a more advanced multiple linear regression under the SND error model (as shown in Subsection 4.3,which is a generalization of subsection 4.2). Both the aforementioned subsections indicate that the estimated coefficients of PCGDP as well as ANSHS are strictly positive, thereby indicating that the overall students’ aggregate performance can be enhanced by improving the province’s (that they live in) economic well-being as well as improving the overall academic competitiveness. This latter finding may have some other deeper implications, and ANSHS maybe confounded with some other factors which are yet to be identified. However, essentially it says that just reducing the number of students per high school may not be helpful as far as aggregate performance is concerned. Probably the quality of the teachersand other resources (like library / digital facilities) do play an important role in enhancing the students’overall performance (but we do not have any data yet to back up this assumption).
Future extension of this work is possibly in two potential directions asfollows. (a) Focusing on the students’performance (i.e., ETS) at a more granular level, that means,perhaps one may look at the district or ward level, and at thesame time at thecorresponding local level PCGDP andANSHS. But while the lattercovariate may be observable at alocal level, similar information on local levelPCGDP may be hard to access. (b) Bringing in otheruseful covariates into the model which can help us understand the students’aggregate performancein a more meaningful way. As mentioned earlier, the existing multiple linear regression model usingprovincial PCGDP as well as provincialANSHS(as shown in subsection 4.2) can explain about half (or a little more than half) ofthe total variability in provincial aggregate ETS. We must find other measurable covariates, whichcould be qualitative as well as quantitative, that can enhance our model’s explicability of theaggregate ETS. Educators, policy makers and sociologists may propose such useful explanatory variablesfor which provincial and/or local level data ought to be available that can strengthen the statisticalmodel.
We sincerely hope that this study will generate further interest among the educators as well as policy makers not only in Vietnam but also in other Southeast Asian nations with similar socio-economicand culturalbackgrounds. It is also hoped that the ensuing debates and discussions can help with identifying other explanatory variables (apart from PCGDP and ANSHS) which can improve the regression model further in order to study the aggregate ETS performance of the high school students.Last but notleast, this statistical analysis has been based upon the available “observational study” data. Is it possible to improve the students’aggregate ETS performance through some “experimental study”? For example, by altering the existing English learning curricula by requiring the students to watch English news channels and/or asking them to read English print media on a regular basis which can possibly help the students achieve a higher level of English proficiency? This is a more serious research question all should be concerned about.
One final word of caution: The statistical analysis undertaken in this study has been done purely as an academic research study by the authors to understand the prevailing condition of the Vietnamese high school students’performance in terms of English test score (ETS). Neither does it reflect the views of the academic institutions that the authors belong to, nor should it be construed as an indictment against any existing educational system.
– Conflict of Interest: No potential conflict of interest relevant to this article was reported.
– Acknowledgements: The second author’s research has been funded by the University of Economics and Law, Vietnam National University Ho Chi Minh City/VNU-HCM
References
[1] General Statistics Office of Vietnam (2019). Statistical Year Book. Vietnam Statistics Publishing House.
[2] General Statistics Office of Vietnam (2020). Statistical Year Book. Vietnam Statistics Publishing House.
[3] General Statistics Office of Vietnam (2021). Statistical Year Book. Vietnam Statistics Publishing House.
[4] Huynh, U., Pal, N., Truong, C., & Nguyen, M. (2021a). A Statistical Profile of Arsenic Prevalence in the Mekong Delta Region. Thailand Statistician, 19(2), 339-360.
[5] Huynh, U., Pal, N., Truong, C., & Nguyen, M. (2021b). Regression Model under Skew-Normal Errors with Applications in Predicting Groundwater Arsenic Level in the Mekong Delta Region. Environmental and Ecological Statistics, 28, 323-353.
[6] Thuithad, P., & Pal, N. (2019). Point estimation of the location parameter of a skew-normal distribution: Some fixed sample and asymptotic results. Journal of Statistical Theory and Practice, 13(2), 13-37.
[7] Vietnam Ministry of Education and Training (2014). Circular No. 01/2014/TT-BGDDT dated January 24, 2014 on promulgating the six-level Foreign Language Proficiency Framework for Vietnam.
Author: Thai Ton Duong Duong, Uyen To Huynh, Nabendu Pal
– Vietnam National University, Vietnam; University of Economics and Law, Vietnam; Faculty of Mathematics and Statistics, Ton Duc Thang University, Vietnam; Department of Mathematics, University of Louisiana at Lafayette, USA
You are viewing the article:
A Statistical Analysis of Vietnamese High School English Test Score Data
Link https://tampacific.com/vje/a-statistical-analysis-of-vietnamese-high-school-english-test-score-data.html
Keywords: analysis of variance (anova); national high school examination; vietnamese education; provincial mean test score; provincial median test score.
Next article:
* This work is licensed under a CC-BY 4.0 International License.