Tuesday, 15 July 2014

Vivas - examiners' perspectives

Back in May, we drew up a list of questions to be put to Professor Elaine Toms when she attended the discussion group in June.  By a stroke of good fortune, the iSchool was being visited by Professor Wildemuth, Associate Dean of the iSchool at the University of North Carolina, Chapel Hill.  She was able to bring another perspective to the questions

What training is available at the iSchool to help students prepare for their vivas?
Currently the iSchool does not offer formal training, though  there have been discussions about introducing more formal training.

How does an upgrade viva compare with a PhD viva?
The two are very similar and both provide an opportunity for the candidate to demonstrate knowledge and understanding of their chosen research topic.

Candidates should provide evidence of systematic study, and.should satisfy examiners that they have made an addition to knowledge.  This is important at both upgrade and PhD level.  

What sort of scene setting is there at the start of a viva?
Both Elaine and Barbara agreed that it was important for students to set the scene by giving the background to their research.  In some places this is done through a formal presentation, though this is not general practice in Sheffield.

To help with scene setting candidates were recommended to take a well labelled copy of the thesis so that they could quickly and easily find parts they wished to refer to.

How long should a viva last?
Recent experiences from Angharad Roberts, Joanne Bates, Rita Wan-chik and Robinah Namuleme all took around two hours, but all of these were relatively trouble free.  If there are serious issues, vivas may last longer.

Who sits on a viva panel? What are their roles?
A PhD viva has an external and an internal examiner on the panel.  Both are vetted by the university.  A key part of their job is to make sure the student is comfortable: they are not there to catch him/her out.  The two examiners should reach agreement, but it is the job of the internal examiner to ensure that university rules are followed.  The supervisor may sit on the panel but should not say anything

An upgrade viva also has a chairman to keep the process on track and to ensure that the student’s knowledge has been properly tested.

What are the possible outcomes of a viva?
PhD viva - Possible results are:
  • No amendments (rare);
  • Minor amendments, no resubmission needed - 3 months;
  • Resubmission without viva - 12 months;
  • Resubmission with viva - 12 months;
  • Re-examination without amendment to the thesis (very rare - only where the thesis is AOK but the performance at viva was unsatisfactory.);
  • Award of MPhil without amendments or re-examination (very rare);
  • Degree not awarded - re-submit for MPhil with/without re-examination;
  • Degree not awarded.
Upgrade viva - Possible results are :
  • Confirmation of PhD status
  • Deferral (6 months to resubmit)
  • 2nd attempt (pass / fail)
How often do people fail their viva?  What sort of things cause them to fail?
Very rarely.  Failure may be for the following reasons:
  • Poor time management
  • Not taking advice from supervisor
  • Not enough evidence of reading
  • Poor organisation of information (eg, bibliography).
Can you give an example of something that impressed you in a viva?
  • Clearly expressed research questions, well explored.
  • Enthusiams
Elaine and Barbara suggested some questions to prepare for:
  • What is your hypothesis?
  • You say your main finding was X.  So what?
  • Why was your research worth the money?
  • Why didn’t you use a particular research method? (Often the examiner's favourite method!)

Tuesday, 13 May 2014

Ask an expert

Discussions for May and June will be on the subject of vivas and upgrade reports / theses.  In preparation therefore, we gave some thought to the questions we would like to discuss.


  1. What training is available at the iSchool to help students prepare for their vivas?
  2. How does an upgrade viva compare with a PhD viva?
  3. What sort of scene setting is there at the start of a viva?
  4. How long should a viva last?
  5. Who sits on a viva panel?
  6. What are their roles?
  7. What are the possible outcomes of a viva?
  8. How often do people fail their viva?
  9. What sort of things cause students to fail?
  10. What preparation do you look for? 
  11. Can you give an example of something that impressed you in a viva?


  1. What guidelines are available for upgrade reports and theses?
  2. Where are they?
  3. What is the process for upgrades?
  4. What is required in a Doctoral Development Programme portfolio?
  5. How is a transfer report assessed?
  6. What are the possible outcomes?
  7. What is the difference between a transfer report and a thesis?
  8. How long should a thesis be?
  9. What makes a good thesis?
  10. What makes a good Methods chapter?
  11. What difference does it make if a student has already published from her/his thesis?
  12. Can you suggest anything that all disciplines would look for in a thesis?

Monday, 17 March 2014

Back to the iFuture (by J.E.A. Wallace)

Those of you who were around last year may remember iFutures. For those who need a reminder, however, iFutures is a one-day conference for PhD researchers in the information science community, organised by students in the Information School at the University of Sheffield. Last year, the conference was held to mark the 50th anniversary of the founding of the University’s Information Studies department, but this time around, we’re looking into the theme of "Research into Practice".

The plan is to interpret this theme in two ways. The first is to try and provide an opportunity for students to introduce their work and the ways in which it can potentially impact on society; while the second is to offer a forum for discussions about the practical steps young researchers can take to maximise the impact of their work, and to engage effectively with the world beyond academia.

To support this second strand we plan to run two parallel interactive workshops. 
  1. "Research Beyond Academia" (led by Sheila Webber) - will look at how to engage with industry as part of the research process, with a focus on how to identify research questions that are relevant and interesting to non-academic institutions, how to determine appropriate potential non-academic partners for grant proposals, and how to approach them and "sell" your new research ideas. 
  2. "Disseminating your Research to Maximise Impact" (led by Dr. Paul Clough) - will explore interesting and innovative ways in which "new" researchers can disseminate their research beyond academia, and what strategies we can use to maximise the impact of our research on practice. 
I will be focussing on these workshops in the Researcher’s Discussion Group. Their format has not yet been fully decided, and the committee is interested in your views as to the sort of issues you would like to see covered.

As for the rest of the schedule, the programme will include keynote talks by two distinguished speakers - David Bawden (Professor of Information Science, City University) and Mounia Lalmas (Principal Research Scientist, Yahoo! Labs). There will also be papers presented by PhD students, and a lunchtime poster session. Finally the programme includes a repeat of last year's highly successful Pecha Kucha session. While nerve-wracking for presenters (just ask Dan and Simon), we hope the unconventional format (twenty slides each displayed for twenty seconds) will both entertain and stimulate discussion.
We welcome submissions from doctoral students at any stage of their research, and full details of the submissions process can be found on our site, http://ifutures.group.shef.ac.uk/

Wednesday, 5 March 2014

What do we want? Inductive models! When do we want them? When we have an evidence base!

After a researchers' discussion, I usually adjourn to the pub with others from the group.  Following the last meeting however, I rushed off to a talk by the Executive Director of Financial Stability at the Bank of England.

I sat next to a Chinese PhD student who introduced himself and asked my name.  "Andrew" I said. He looked at the ticket he was holding.  "Ah!" he replied, "The same as the speaker.  An important name."

We had come to hear Andrew Haldane in conversation.  The academic conversing with him was introduced as Professor Andrew Gamble.  "No" I observed, "just common".

Andrew Haldane may have a commonplace name, but for someone embedded at the core of the establishment, his views are not commonplace.  He has attracted attention in the past for speaking in favour of the Occupy Movement.  In the course of his conversation with Professor Gamble, he criticized economists' over-reliance on deductive models and expressed the view that greater efforts should be made to develop more pragmatic inductive models.  

Amongst the questions asked by members of the audience was one from an economics student at Sheffield University.  She bemoaned the fact that she was learning the type of model that Haldane had criticized.  What, she asked, should she and her fellow students do about it?  "Protest!" was his succinct response.

Haldane’s objection to many of the standard economic models was that they were devised to evaluate risk rather than to address uncertainties.  The difference, explained Prof Gamble, was the difference between known unknowns and unknown unknowns.  Risk analysis identifies a spectrum of outcomes and assigns probabilities to each, then uses those probabilities to help prioritise actions. So for example, if an unbiased coin is tossed, probabilities can be assigned to whether it will, on coming to earth, be seen to display heads or tails.  If it falls down a crack in the floor, those calculations become invalid.  

Although the probability of a coin disappearing in this way is small, there are numerous other improbable things that could happen to the coin, each of which could affect knowledge of whether it displays heads or tails (or lands on its edge!).  There is a high probability therefore, that at least one improbable event will occur, with an impact that has not been considered in the risk analysis.

Because deductive models tend to focus on known unknowns, these get over-emphasised.  By contrast, pragmatic models based on rules of thumb, implicitly adjust for the accumulated improbables.

Friday, 28 February 2014

Post-viva questionnaire - responses from Angharad Roberts

What is the title of your thesis?
Conceptualising the library collection for the digital world: a case study of social enterprise

Can you provide an abstract (for inclusion in this blog)?
This collaborative research project, supported by the British Library, used a case study of the library collection for social enterprise to develop a conceptual approach to the library collection in the digital world, exploring stakeholder perceptions of collections, terminology and collection development and management processes. A mixed-methods multiphase case study design was used to address the research questions. Three strands of data collection are described: a case study of the British Library’s collections and content for social enterprise, searches for relevant material on 88 publicly accessible UK library catalogues, and an exploratory sequential study involving stakeholder interviews (19 interviews with 18 people) followed by two surveys of a larger stakeholder population (149 completed responses in total). Findings from each strand are described and three core concepts of collection are identified: “collection-as-thing”, “collection-as-process” and “collection-as-access”. Conventional views of library collections may tend to focus more on the idea of “collection-as-thing”; this research emphasises the importance of taking a more dynamic view of collection. Three models of collection are described: a revised collection development hierarchy which suggests links to different levels of strategic management; a model of interrelationships between the three concepts of collection; and a model which examines how collection adds value to content by providing context. This research demonstrates that the concept of collection remains highly relevant in the digital world, although the onus is on libraries to embrace all dimensions of these three concepts of collection if they wish to add maximum value to the content they identify, select, hold, make accessible and to which they connect.

How long did you spend preparing for your viva?
I re-read my thesis over the course of a week or so before, and dipped into some chapters again (methodology, discussion and conclusion) on my way to the viva. I also met with my supervisor a couple of days before the viva to talk about it, which was very helpful.

How long did your viva take?
2 hours, followed by fifteen minutes waiting for the examiners to call me back in to give their decision.

Is there anything you wish you had done differently?
I spent quite a bit of time in the days and weeks preceding the viva scaring myself by imagining really tough questions! It's useful to think through possible answers to broad general questions which are highly likely to be asked (such as "why this topic?" or "why this methodology?"). However, you can never prepare for every question, and spending time dwelling on very specific and unlikely possible questions can take time away from more useful forms of preparation (such as re-reading the thesis).

Did the examiners concentrate on any particular section of your thesis? If so, which?
We mainly discussed the literature review, methodology and discussion chapters, with some discussion of practical recommendations and recommendations for future research from the conclusion.

Can you describe any part of your viva where you were pleased with your performance?
Early on I was asked about whether any of the strands in my methodology were weaker than others. I was candid about the limitations of one of my strands and that was received well by the examiners.

What was it you did that pleased you?
That strongly reinforced the idea that the viva is about being honest, reflective and critical about your own work, rather than being about “dressing things up” or trying to show everything in a positive light.

Can you describe any part of your viva where you were dissatisfied with your performance?
I was asked about whether I agreed with a specific quotation which I had included in my literature review. I floundered a bit when trying to answer, and I think my perspective changed quite a bit following the exchange with the examiners.

Please give an example of a question that you found hard.
I was asked about the order of concepts in a revised collection development hierarchy which I had developed.

Why was it hard?
I needed to think about my model in quite an abstract way.

What was the outcome of your viva?
Passed subject to minor corrections.

Please give some examples of the sort of corrections you need to make (if any).
Clarification of my use of the word “for” in the phrase “library collection for social enterprise” from my research aims and objectives.

Minor adjustments to one of my conceptual diagrams.

Corrections to typos, formatting (eg avoiding splitting tables over multiple pages).

Do you have any tips for looking and feeling confident in front of the examiners?
Wear smart comfortable clothes. Try to get a good night’s sleep the day before.

Can you think of any good advice that you would give to students who are preparing for their viva?
  • Try to meet your supervisor in the run up to your viva to talk over any issues and for reassurance.
  • You can ask your supervisor to sit in as an observer. I did and found it helpful to have someone else’s perspective on how things had gone.
  • If you submitted an electronic copy of the thesis on CD, you can go and collect it from Research and Innovation Services straight after the viva.
  • Have something nice to look forward to. I didn’t get a holiday last summer so a long weekend away after the viva was just what I needed.
After the viva
Bear in mind that the viva is only a part of a longer process (although obviously it’s an absolutely vital part of that process!) First you submit the thesis for examination, then you have the viva, then you are likely to need to make corrections and your internal examiner needs to confirm whether the corrections made are acceptable. Finally you submit the library copy of the approved amended thesis and, if you started your course after 2008, upload the final version to White Rose eTheses. Completion of each of these stages inspires varying degrees of euphoria!

The following point doesn’t quite relate to the question above (about preparing for the viva), but prior to submission it is worth thinking about the practicalities of how you will get your thesis printed and bound. It does cost quite a lot to get this all done by Print and Design Solutions. For the initial submission I printed two copies (for the examiners) and only went to Print and Design Solutions to get them bound. For my final submission, I got Print and Design Solutions to print and bind three copies – one for the University Library, one for an organisation which supported my research and one for myself.

It's also useful to be aware that you may not receive any formal communication following the viva - my internal examiner emailed me the list of minor corrections. Once the University Library copy of the final thesis had been submitted, I was sent copies of the more detailed examiners' reports, which were really interesting and helpful to read.

Wednesday, 15 January 2014

The problem of identifying research with impact

The deadline for submissions to the Research Excellence Framework (REF) from UK higher education institutions passed on 29 November 2013.  Academics around the country breathed a big sigh of relief and returned to normal life (i.e., preparing for the next exercise in research evaluation).

Research evaluations of this sort began in the UK in 1986 and play a major part in the allocation of research funds.  Up until the REF they relied very heavily on citations (eg, Norris & Oppenheim, 2003).  When the Research Assessment Exercise (RAE) was replaced with REF, the new evaluation attempted to shift the emphasis away from citations to other forms of impact.

The obvious advantage of citation-based measures of impact is that they are relatively simple to calculate and to organise, making comparisons possible.  Amongst the disadvantages is the fact that publications that are highly cited in the few years following their publication are often cited because they are convenient summaries of existing research and say little that is original.  Conversely, truly ground-breaking ideas often have no detectable impact in their first few years because their implications are not understood.

A classic example of such a case is that of Gregor Mendel's paper on hybridisation, which is generally regarded as laying the foundation for genetics.  It was published in 1865 but remained almost uncited throughout the remainder of the century ("about three times" according to Wikipedia 06/01/14, and twice according to Google Scholar).

Boris Belousov suffered similar lack of recognition but took it more to heart.  He tried for several years throughout the 1950s to publish a description of an interesting reaction he had observed, but reviewers considered it to be impossible.  It is now recognised as a discovery which forced "a change of perspective and emphasis". (Winfree, 1984)

Winfree, Arthur T. (1984), The Prehistory of the Belousov-Zhabotinsky Oscillator, Journalof Chemical Education 61(8) 661-663.

Thursday, 2 January 2014

Summoning of demons: an inappropriate response to criticism

Life sometimes gets in the way of blogging, so I haven't written anything here for a while.  The last entry (by Mary Anne Kennan) discussed the assumptions that underpin research methodologies.  The group discussion that followed its publication touched on some of the problems that arise when such assumptions are not properly examined.  One example that I proffered was the issue of research undertaken according to one methodology being assessed by reviewers following a different methodology.

A few months ago I published a blog entry in which I included some reviews of a paper that was subsequently published. The research being reported on was interpretivist, but it was reviewed as though it were positivist.  Consequently, the reviews were hostile.

Amongst the things that have got in the way of blogging recently has been Christmas, with its abundance of distractions.  One of the smaller distractions was Mark Gatiss' production of the M.R. James story "The Tractate Middoth".  Much of it took place in a library, and the vital clue to a missing will was an accession number.

As a child, I read quite a few M.R. James stories but did not remember this one.  I set out to discover and rediscover some.  One that I discovered was James' story, Casting of the Runes, which the inspiration for a 1950s horror film called Night of the Demon (renamed 'Curse of the Demon' in the USA).  In the film, the leader of a Satanic cult uses his knowledge of the occult to summon up demons to destroy his enemies.  I had seen the film but not read the story.  I did so and found that, in the original, the occult powers were used in response to bad reviews of the author's work.