Text mining papers from the 2011 ICALT and ICCE conferences shows some surprising – and highly statistically-significant – differences in the topics as expressed through the terms used in the papers. Given the different organisers of these conferences - IEEE Technical Committee on Learning Technology for ICALT and the Asia-Pacific Society for Computers in Education - and the locations for the 2011 conferences, the USA and Thailand respectively, might the results point to two or more worlds of technology enhanced learning, or might there be a more simple explanation?
It is impossible to draw incontravertible conclusions but the evidence from a recent text mining study undertaken by the TELMap Project seems to show that there are indeed distinct communities with quite different focus points.
Let the Data Speak
The following two plots – which show the statistical significance and frequency of those terms which appear significantly-more in each set of conference papers – show some striking differences. A significance value of 3 equates to a 1 in 1000 probability that the difference is due to random chance and a significance of 6 equates to a 1 in a million chance; clearly some of the differences are highly unlikely to be due to random variation in the words used in the papers.
A full set of plots, data and source code is also available, including plots showing clustering, or co-occurrence, of terms.
Application vs Technology
Although there are some exceptions, ICCE papers seem to more focussed on issues of application than the ICALT papers do; there are more references to the people, the educational sectors and the activities of education. In contrast ICALT papers show a stronger focus on technology aspects.
For example, ICCE 2011 has startlingly-higher occurrence of "school" (seemingly you would never hear the word spoken at ICALT), "university", "teachers", "students" (this is omitted from the plot since it appears with a frequency of 2%), "class", "skills", "task", "knowledge", "understanding" and "participants". Interest in improving achievement is suggested by "effective", "improve", "level" and "results". Although these might sometimes be referring to improvement of aspects other than achievement, an inspection of the dominant co-occurrence (clustering) of terms suggests an achievement-oriented interpretation is warranted.
Two terms with a moderate frequency of 0.5% from the ICALT 2011 papers have such a high statistical significance that it is assumed that none of the 150 papers from ICCE contained them: "objects" and "user", two terms with a very strong computer-science/technology flavour. Other terms with a software flavour are "tools" and "application" while a theoretical perspective is indicated by "model", "concept", and "design". An inspection of the term co-occurrence graph also suggests that "interaction" is closely linked with "tool" while less-closely linked to "user", again suggesting a technology bias. In contrast, only "programming" has a strong technology sense among the dominant terms from ICCE papers.
The conclusion seems clear: ICCE is dominated by concerns of application whereas ICALT contains more of a mix between application - indicated by "course", "activities" and "performance" - and technology but with a technology bias.
A "Hot Topic" is a specific focus of interest that seems to be receiving dis-proportinate attention.
A small number of the terms are suggestive of differences in the "hot topics", although for ICCE 2011 these are less distinct with only "reading" standing out. Reading is clearly a necessary component of education from the early years but it is surprising that attention should be drawn to it. A review of the paper abstracts confirms that there is indeed a focus on literacy including English as a foreign language.
The hot topics from ICALT 2011 contain no surprises for anyone familiar with European and American Technology Enhanced Learning, being clearly indicated by the terms:
- "game"; serious games and game-based learning have risen significantly over the past few years. The term does not seem to be correlated with others on the co-occurrence graph, which suggests quite a distict topic rather than a pervasive one.
- "personal"; personal learning environments and personalisation are also well-established topics of interest.
- "content" is a perennial concern and may be experiencing a renaissance due to interest in open educational content.
Remember, though, that the terms indicating these topics are statistically more frequent in 2011 papers ICALT compared to ICCE, in some cases staggeringly or unbelievably so.
While some of the differences between application and technology might have their roots in the evolution of communities from computer science and education, it seems likely that the differences in the way the education systems work – from models of state intervention to pedagogic practice – helps to preserve distinctions.
The differences in the hot topics raise more questions than they hint at explanations. For example: why is reading neglected at ICALT when poor literacy is frequently bemoaned (e.g. UK: "One fifth of school-leavers so illiterate and innumerate they struggle to cope with challenges of everyday life").
Have your Say!
Draw your own conclusions from the data and share them by commenting.
A full set of plots, data and source code is available, including plots showing clustering of terms. All are free to use.