Danish use of internet in exams – epistemology, pedagogy, assessment…
Earlier this year I took the Google Advanced Power Searching with Google MOOC. For one of the assignments I did some research on a relatively long standing interest of mine – the use of internet in exams in Denmark, a topic on which I wrote part of my MA thesis, and which motivates some of my interest in epistemology of assessment and epistemic commitments in information seeking now. I’ve written/presented on that under the title: “What is it to ‘know’ when we search” and it’s something I hope the Edinburgh Extended Knowledge Project explores further. I’ve copied the assignment template (completed obviously!) below for other’s interest. Note there are some nice search tips in there too :-). There are also a couple of nice videos:
- Fantastic BBC summary (I’ve used this in talks before) http://news.bbc.co.uk/2/hi/8341589.stm
- Danish Education minister – Steen Lassen talking about the policy
Advanced Power Searching Assignment
Choose a search problem that’s complex enough to require at least three Advanced Power Search skills. Document how you solved the problem.
1. What is your research goal? What will you do with the information? This should be a complete sentence and/or phrased as a question.
In 2009 I was reading a lot about a Danish experiment with the use of internet access in exams, a 3 year pilot. I want to follow up on this and see whether or not the experiment is to be implemented/maintained.
2. What questions do you need to answer in order to achieve your research goal? List at least three questions that are directly related to the goal listed above.
Are there policy documents/statements in Denmark in the last year which refer to internet access in exams of the future?
Are there any “final reports” of the pilot which refer to plans for future implementation and sum up findings?
What are the properties of any future exams – i.e., what did they learn whether they’ve decided to implement or not – how have the exams changed post ‘internet access pilot’?
3. What queries did you type in during your research (either to Google or databases/sites you discovered)? List at least three queries you used when searching. These should correspond to the questions above. They should demonstrate advanced power searching skills.
Denmark exams, which was then limited to “last year” and -language (to remove irrelevant tests for immigrants). News reports gave me links to the minister in charge which allowed me to search for them.
In 2010 Steen Lassen linked me to the final report on the pilot (Undervisningsministerie (Ministry of Education), and Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education). “Endelig Rapport Fra Følgegruppen for Forsøget Med Digitale Prøver Med Adgang Til Internettet i Udvalgte Fag På Stx Og Hhx (Final Report of the Monitoring Group Experiment with Digital Samples with Access to Internet in Selected Subjects at Stx and HHX).” Undervisningsministerie (Ministry of Education), September 2010. http://www.uvm.dk/~/media/Files/Udd/Gym/PDF10/101007_it_rapport_13%20september%202010.ashx.) which I translated using google translate. I refound this in my Zotero archive (using search) removed my translations from the citation and searched google for citations of the report. None were found. Googling the URL brought up one other report (slightly later) from the site http://www.uvm.dk/Uddannelser-og-dagtilbud/Gymnasiale-uddannelser/I-fokus-paa-omraadet-gymnasiale-uddannelser/~/media/UVM/Filer/Udd/Gym/PDF10/101007_it_rapport_13%20september%202010.ashx
Unfortunately I still can’t find english language example questions; a within document search on the translated document gives some examples (mostly in readable english!) as well as a summary of findings (plus the above english document). It appears nothing new has been published recently. To check, I searched the Danish version of the Danish education site (internet=internet in danish). Their internal search doesn’t allow filtering by date, so I switched to a site:http://uvm.dk/ search, for which I did an “translated foreign pages” search https://www.google.co.uk/search?q=internet+http%3A%2F%2Fuvm.dk%2F&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a#q=internet+eksamener+http://uvm.dk/&hl=en&client=firefox-a&hs=DNS&sa=X&tbo=d&rls=org.mozilla:en-US:official&tbs=clir:1,clira:a,clirtl:de%3Bfr%3Bes%3Bit%3Bda&bav=on.2,or.r_gc.r_pw.r_cp.r_qf.&bvm=bv.42080656,d.d2k&fp=461ace6cdefb9ef0&biw=1600&bih=656
4. What resources (specific websites) did you use when gathering information? How did you know they were credible? List at least three websites and how you evaluated their credibility: verified with multiple sources, primary source, etc.
Some news reports (e.g. http://www.freshties.com/wp/?p=9812 ) led me to the government site where a search for the ministers name gave nothing. I searched for “internet examinations” (using the terms used in a report I’d already found) and was directed to an English language summary which wasn’t there when I looked a few years ago http://eng.uvm.dk/news/~/UVM-EN/Content/News/Eng/~/media/UVM/Filer/English/Fact%20sheets/091118_ICT_in_upper_secondary.ashx
Prior knowledge (from an authority) gave me a useful link on the Danish govt website (in Danish, so very hard to find without knowledge), which then let me access another report.
I did find a number of sample current assessment pages:
- http://translate.google.co.uk/translate?hl=en&langpair=da|en&u=http://www.uvm.dk/Uddannelser-og-dagtilbud/Gymnasiale-uddannelser/Proever-og-eksamen/Skriftlige-opgavesaet/Opgavesaet-for-stx (written assignment samples)
- http://translate.google.co.uk/translate?hl=en&langpair=da|en&u=http://www.uvm.dk/Uddannelser-og-dagtilbud/Folkeskolen/Afsluttende-proever/Om-afsluttende-proever/Eksempelproever Sample problems for maths (with a focus on ICT use)
- http://translate.google.co.uk/translate?hl=en&langpair=da|en&u=http://www.uvm.dk/Uddannelser-og-dagtilbud/Folkeskolen/Afsluttende-proever/Om-afsluttende-proever/Proevevejledninger Primary exam sample manuals
The google translations are fine to read for samples, and the pages are almost all on the ministry websites.
5. What was your final result? Answer the question you listed in question 1 above. Even if you did not achieve your research goal, describe why you stopped when you did.
See above. I still can’t find anything other than press releases/interviews to support a shift to this new system (e.g. no new policy document), this may be because I’m expecting to see a green paper/white paper as we’d have in the UK on this sort of thing, and that may not be necessary in Denmark.
6. What did you learn while conducting your research? List at least one interesting fact or insight that helped you further your research. What did you learn about the research process that you will do again or differently next time?
The use of “translated pages” function is fantastic, I didn’t know it allowed you to browse within the site maintaining the translation, and fortunately Danish seems to be translated rather well.
7. What Advanced Power Searching skills did you apply during this exercise? Your problem should be sufficiently complex so that you applied at least three of these.
- Use advanced search to find pages written in a particular language.
- Use search settings to set a level of filtering, turn on/off search-as-you-type, set the number of results shown, block unwanted sites, and choose whether to personalize results or not.
- Use advanced search to search websites from a specific country; use the left hand panel to search as if you were in a particular location.
- Use web history to revisit pages you have found previously.
- Use Google News and the News Archive search to locate newspaper and news blog articles.
- Use context terms to find specific types of sources.
- Use the language appropriate to the author’s social and historical context (synonyms).
- Recognize the kind of problem you are asking and match the kinds of resources to search.
- Use people (online and offline) as resources to help identify information.



Just copying some stuff from my MA for a chapter I’m writing, the below is salient for this post.
In Denmark “The aim of the examinations and tests to measure student knowledge, skills and competencies (EQFI 23 April 2008) in relation to the targets that have been created from prior instruction” (Undervisningsministerie (Ministry of Education) & Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education), 2010, p. 5) 15 In June 2009, “more than 1000 students were writing exams with Internet access” (Undervisningsministerie (Ministry of Education) & Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education), 2010, p. 22). To meet this aim, students in some subjects have been given access to most internet websites (although not those which could be used to communicate with other students).
This is a natural extension of Danish examinations which allowed access to a variety of media for some time now. For example in Danish language exams, access includes, “new types of texts made possible by the enclosed CD. These are videos (television interviews, TV news clips, webfilm, short film), audio (radio debate, rap) and webpages….The aim is that the Danish student subject to this type of tasks may seize the opportunity to work with a broad material field and not be bound to printed fiction texts. So one can also relate, for example, analysis of a political speech, an excerpt of a documentary, or as in the exam kit – a movie clip.” (Undervisningsministerie (Ministry of Education) & Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education), 2010, p. 17).
It is believed that these examinations providing a good basis for assessing student achievement of the Danish subject’s academic goals. And more than ever, this type of examination may evaluate the student’s ability to:
o Analyze and evaluate primarily non-fiction in all media
o Analyze, interpret and contextualise primarily fictional texts in all media
o Demonstrate knowledge of and relate to the modern media environment,
including the ability to analyze and evaluate texts, communicative significance
and the media’s role in communication
o Navigate and select information in screen-based texts with an academic focus
Within mathematics, “many students search on the web as an alternative to references in textbooks. The Internet is also used to check the upshot of tasks on websites that will calculate the input size, but without the necessary documentation so that they cannot form the basis for the script itself, but simply check the results obtained” (Undervisningsministerie (Ministry of Education) & Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education),
2010, p. 18).
http://eng.uvm.dk/Fact-Sheets/Upper-secondary-education/Information-and-Communication-Technologies-ICT-in-Upper-Secondary-Education
http://www.theguardian.com/commentisfree/2015/apr/30/googling-in-exams-allowed-google-searches-gcse-a-level
Nice discussion on the BBC Radio 4 Today Show on the issue http://www.bbc.co.uk/programmes/p02q9vcs with OCR chief exec Mark Dawe and Chris McGowan
[…] Knight, S. (2013) Danish use of internet in exams – epistemology, pedagogy, assessment… Finding Knowledge blog. http://sjgknight.com/finding-knowledge/2013/07/danish-use-of-internet-in-exams-epistemology-pedagogy… […]