Survey Questions

BIAS 1

Below you will find a number of statements about the general public and science. Please indicate your level of agreement with items 1.1 - 1.4 using the scale below.

1.1 Most people tend to overlook data that do not accord with their own views.

1.2 Most people test their own views primarily by looking for confirming evidence rather than possible disconfirming evidence.

1.3 Most people find ways to actively dismiss data that do not accord with their own views.

1.4 Different people with contradictory views often view the same piece of data as evidence for their own views.

AIM

Below you will find a number of statements regarding the role of successful communication of science to the general public. Please use the following statement to indicate your greement for items 2.1 - 2.6.

Successful communication with the general public is primarily a matter of …?

2.1 transferring scientific information.
2.2 generating interest in science.
2.3 conveying scientific understanding.
2.4 creating a shared understanding.
2.5 getting the public to identify with the scientific enterprise.
2.6 conveying an understanding of scientific reasoning.

MORAL THEORY (MT)

The following items ask you to imagine a scenario and an action. Please indicate the appropriateness of the proposed action to be taken in items 3.1 - 3.5. When imagining these scenarios, rely only on the facts given in the scenario.

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

3.1 A runaway trolley is heading down the tracks toward five workmen who will be killed if the trolley proceeds on its present course.

You are on a footbridge over the tracks, in between the approaching trolley and the five workmen. Next to you on this footbridge is a stranger who happens to be very large. The only way to save the lives of the five workmen is to push this stranger off the bridge and onto the tracks below where his large body will stop the trolley. The stranger will die if you do this, but the five workmen will be saved.

How appropriate is it for you to push the stranger onto the tracks in order to save the five workmen?

3.2 You are at the wheel of a runaway trolley quickly approaching a fork in the tracks. On the tracks extending to the left is a group of five railway workmen. On the tracks extending to the right is a single railway workman. If you do nothing, the trolley will proceed to the left, causing the deaths of the five workmen.

The only way to avoid the deaths of these workmen is to hit a switch on your dashboard that will cause the trolley to proceed to the right, causing the death of the single workman.

How appropriate is it for you to hit the switch in order to avoid the deaths of the five workmen?

3.3 Enemy soldiers have taken over your village. They have orders to kill all remaining civilians. You and some of your townspeople have sought refuge in the cellar of a large house. Outside you hear the voices of soldiers who have come to search the house for valuables.

Your baby begins to cry loudly and you cover his mouth to block the sound. If you remove your hand from his mouth, his crying will summon the attention of the soldiers who will kill you, your child, and the others hiding out in the cellar. To save yourself and the others you must smother your child to death.

How appropriate is it for you to smother your child in order to save yourself and the other townspeople?

3.4 You are visiting the sculpture garden of a wealthy art collector. The garden overlooks a valley containing a set of train tracks. A railway workman is working on the tracks, and an empty runaway trolley is heading down the tracks toward the workman.

The only way to save the workman's life is to push one of the art collector's prized sculptures down into the valley so that it will roll onto the tracks and block the trolley's passage. Doing this will destroy the sculpture.

How appropriate is it for you to destroy the sculpture in order to save this workman's life?

3.5 You, your husband, and your four children are crossing a mountain range on your return journey to your homeland. You have inadvertently set up camp on a local clan's sacred burial ground.

The leader of the clan says that according to the local laws, you and your family must be put to death. However, he will let you, your husband, and your three other children live if you yourself will kill your oldest son.

How appropriate would it be for you to kill your oldest son in order to save your husband and your other three children?

AUDIENCE PERCEPTIONS (AUD)

The following items are statements that might be made about the general public. Please indicate your level of agreement with statements 4.1 - 4.5 using the scale below:

Most of the general public ...

4.1 think that science benefits human well-being.

4.2 think that scientific advances pose significant dangers.

4.3 understand probabilities as quantitative expressions of the degree of confidence a scientist has in a theory or parameter estimate.

4.4 interpret reversals of a previous scientific consensus as evidence that science is unreliable.

4.5 think that special interests significantly influence the findings that scientists report

The following items provide statements regarding the general public. Please indicate your level of agreement with statements 5.1 - 5.5 using the scale below:

5.1 The use of probabilities by scientists tends to make the public doubt scientific findings.

5.2 The general public is capable of understanding how evidence supports or fails to support a scientific finding.

5.3 Audiences with vested economic interests tend to be incapable of objective assessments of scientific findings.

5.4 Audiences with a value system that predisposes them to distrust particular theories and to repeatedly question the evidence for those theories are incapable of rationally assessing the theories.

5.5 Audiences with a value system that seems inconsistent with some scientific theories may rationally pursue their values by subjecting those theories to greater skepticism than others.

EFFECTIVENESS (EFF)

Frames are sets of related concepts that organize central ideas in ways that are socially shared and persistent over time. Frames are used to simplify discussion of complex issues. Through the use of frames a speaker or writer can induce an audience to employ core values and assumptions.

The following items concern the effectiveness of various ways of 'framing' a discussion in getting audience to:

understand a scientific theory and the evidence for it
believe or accept a scientific finding
identify with and become interested in the science

Please indicate your level of agreement regarding the understandability, acceptability, and the 'interest generation' ability of each of the frames presented:

In general, how effective is a frame in which the scientist is cast as a champion of the public interest, and his or her opponents as representatives of special interests in...

6.1 getting an audience to understand a scientific finding?
6.2 getting an audience to accept a scientific finding?
6.3 getting the audience interested in science?

In general, how effective is a frame focused on the scientist who develops and learns through the scientific process of inquiry in...

7.1 getting an audience to understand a scientific finding?
7.2 getting an audience to accept a scientific finding?
7.3 getting the audience interested in science?

In general how effective is a frame in which narratives are developed about members of the general public who have experienced difficulties due to some natural disaster, illness, or loss, and scientific progress is shown to help those with the difficulty in...

8.1 getting an audience to understand a scientific finding?
8.2 getting an audience to accept a scientific finding?
8.3 getting the audience interested in science?

In general how effective is a frame in which science is cast as an adventure in pursuit of truth, where all sides are treated as partners in a collective search for truth in...

9.1 getting an audience to understand a scientific finding?
9.2 getting an audience to accept a scientific finding?
9.3 getting the audience interested in science?

COMMUNICATION NORMS (CN)

10.1 To what extent are scientists obligated to communicate their findings to the general public if their research HAS been funded by public grants?
Not at all Obligated ס ס ס ס ס ס ס Very Obligated

11.1 To what extent are scientists obligated to communicate their findings to the general public if their research has NOT been funded by public grants?
Not at all Obligated ס ס ס ס ס ס ס Very Obligated

12.1 In general, to what extent are scientists obligated to help the public understand the empirical evidence for their theories or findings?
Not at all Obligated ס ס ס ס ס ס ס Very Obligated

13.1 When precision about scientific findings will tend to confuse rather than aid understanding, in general, is it more important to convey information precisely or to convey understanding?
Precision is more important ס ס ס ס ס ס ס Understanding is more important

14.1 Metaphors or analogies are not completely accurate but nonetheless often help an audience to understand; in general, is it more important to convey information accurately or to convey understanding?
Accuracy is more important ס ס ס ס ס ס ס Understanding is more important

The following items provide statements about the appropriateness of potential strategies adopted by scientists in scientific communication to the general public.

Please indicate your view regarding items 15.1 - 15.2 using the scale below.

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

15.1 To what extent is it appropriate for scientists to adjust their communications with the public to fit more comfortably with the values of their audience?

15.2 How appropriate is it for scientists to advocate acceptance of particular scientific theories, in an attempt to bring the general public to endorse those theories?

The following items provide statements about potential strategies and roles adopted by scientists in scientific communication to the general public.

Please indicate your view regarding items 16.1 - 16.2 using the scale below.

16.1 It is perfectly appropriate for scientists to advocate for particular policies.

16.2 Scientists should disavow their status as experts when advocating for particular policies.

COMMUNICATION PRACTICE (BEH)

In the next few items, you will be presented with a number of brief scenarios, and asked to evaluate the appropriateness of each. In responding to each of the items, please use only the actual information given in each scenario.

Consider the following scenario in responding to questions 17.1 - 17.3.

Sue has discovered a new technique for producing genetic modifications. The technique has potential for revolutionizing several agricultural technologies, and will likely lead to more effective conservation efforts and great economic benefits.

There is some chance that in developing the technology, or in using it, significant adverse consequences will ensue. Yet, Sue thinks that the chance is very small (less than one chance in 60,000).

Sue is currently engaged in writing articles and preparing public talks to explain her findings to policy makers and the general public. This is necessary because related technologies have in the past been widely decried in the general public, and consequently heavily regulated.

Please use the following scale to indicate the level of appropriateness for items 17.1 - 17.3, using the background information provided by the aforementioned scenario:

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

17.1 Sue has been considering using language in her communication that she thinks will help people overcome previous resistance by emphasizing the possible long-term potential for increased food sources and appealing to human progress and the betterment of humankind.

How appropriate would it be for Sue to do so?

17.2 Sue has also considered omitting any mention of the potential risks of her new technique and the technologies to which it may lead.

How appropriate would it be for Sue to do so?

17.3 Sue has considered mentioning the risk, but doing so using an analogy rather than precise statements of probability. Specifically, she has considered comparing the risk of adverse consequences to that of being hit by lightning (lifetime risk ~1 in 80,000).

How appropriate would it be for Sue to include the comparison without giving precise probabilities?

Consider the following scenario in answering questions 18.1 - 18.3.
Thomas has over the last decade accumulated data which convincingly supports a radical theory and thus has begun to win support for the novel theory.

Though Thomas' data do convincingly support the novel theory, many opponents support various modifications of the orthodox theory to accommodate the new data.

Thomas is presenting a talk to a mainly non-scientific audience, but most of them will have heard something about the current controversy.

Please use the following scale to indicate the level of appropriateness for items 18.1 - 18.3, using the background information provided by the aforementioned scenario:

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

18.1 Thomas is considering 'framing' his discussion in terms of an 'upstart' or challenge to the conventional 'establishment', to help the audience understand how novel his theory is and the reasons so many prefer the orthodox theory, despite his new data.

How appropriate would this kind of description be?

18.2 A frame that is sometimes used by those who advocate particular policies involves describing opponents as 'hired guns' out to serve 'special interests' or 'the privileged' at the expense of truth, and everyone else.

In fact, Thomas is convinced that at least some of the most imminent critics of the new theory are in exactly this position. He is considering using this frame in his talk.

How appropriate would this kind of description be?

18.3 Historians and journalists sometimes also frame discussions of past advances in moral terms, describing the defenders of the novel theory as champions of good science in the search for truth, fighting against an corrupt establishment that is either against science and/or indifferent to truth.

In fact, Thomas thinks this is true of many, though perhaps not all, of those who have not yet accepted the novel theory (they have in fact raised the standard they insist be met, on more than one occasion). He is considering using this frame in his talk.

How appropriate would this kind of description be?

Consider the following scenario in answering questions 19.1 - 22.1.

The researchers below are each giving a public talk to a non-science audience, aimed to describe a theory and the evidence for it. The theory is widely accepted by scientists, but not yet conclusively proven and politically very controversial.

Among the claims which have widely been denied in public by critics is an estimated value for a particular parameter.

In each of the questions below, the researcher knows that the value of this parameter is in fact irrelevant to the case for the theory, but also believes that the estimated value is very probably but not certainly correct. The researcher also knows that many in the audience are disposed to reject the theory on political grounds.

For items 19.1 - 22.1, please indicate the level of appropriateness or effectiveness of the proposed actions, using the background information provided by the aforementioned scenario:

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

19.1 Justine decides that the audience is more likely to UNDERSTAND the relevant findings if she 'meets them half-way' by allowing that the estimate of the parameter may well be in error.

How appropriate would it be for Justine to do so?

Please indicate the level of effectiveness of the previous action, again using the background information provided by the scenario as a base:

20.1 Whether or not appropriate, would Justine's strategy be effective in helping the audience to UNDERSTAND the relevant findings?

Again, using the scenario presented above, please indicate the level of appropriateness of the following action:

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

21.1 Jason believes his audience is more likely to ACCEPT the theory, whether or not they understand it, if he allows that the estimate of the parameter may well be in error.
How appropriate would it be for Jason to do so?

Please indicate the level of effectiveness of the previous action, again using the background information provided by the scenario as a base:

22.1 Whether or not appropriate, would Jason's strategy be effective in getting the audience to ACCEPT the theory?

Consider the following scenario in answering questions 23.1 - 23.3.
Several scientists will be discussing with different public audiences the possible effects of a policy according to the scientific consensus of a particular field.

The policy is politically controversial, since it may be of great benefit overall (if the scientific consensus is correct), but will certainly do financial harm to some in the public.

There is significant uncertainty within the field regarding both the actual models used to make the predictions and some of their parameters.

Still, most of the models provide similar predictions. However, some models (given low probability by most but not all researchers), make very different predictions; one of these in particular has some defenders.

Please use the following scale to indicate the level of appropriateness for items 23.1 - 23.3, using the background information provided by the aforementioned scenario:

1 - Very Inappropriate | 2 - Inappropriate 3 - Somewhat Inappropriate | 4 - Neither Appropriate nor Inappropriate 5 - Somewhat Appropriate | 6 - Appropriate | 7 - Very Appropriate

23.1 Moira is particularly concerned about one model to which she (and most others) attributes a very low probability, but which does give a significantly different prediction.

She is considering mentioning some of the specific model variants but not mentioning this model at all, because she fears her audience will mistakenly focus on that unlikely result and ignore the remaining model predictions.

How appropriate would it be to use such an approach?

23.2 John believes that the uncertainty in the models is not very important with respect to the actual predictions, since nearly all the models with any reasonable probability agree about the most important predictions.

But John believes that his audience is likely to misunderstand uncertainty about the models as lack of competency on his part. He is considering not mentioning any particular uncertainties but rather plans to say just "according to our best models today, the prediction is ...".

How appropriate would it be to use such an approach?

23.3 Theresa is in the scientific minority and actually assigns a higher probability to the divergent model than most others do.

She is considering emphasizing the low but significant (in her judgment) probability that this model is correct, together with its deviant predictions.

How appropriate would it be to use such an approach?

BIAS 2

The following items will ask you about your awareness of a number of cognitive and social processes. Please use the following scale to indicate your level of awareness for items 24.1 - 24.3.

24.1 There is a cognitive process that causes a person to be more confident in the quality of data that confirms a view the person holds than in the quality of data that disconfirm that view.

Before taking this survey, how aware were you of this process?

24.2 There is a cognitive process that causes a person to test a hypothesis primarily by searching for evidence that confirms the hypothesis rather than by searching for possible evidence that disconfirms it.

Before taking this survey, how aware were you of this process?

24.3 There is a social phenomenon in which groups that are opposed on some issue, when exposed to mixed evidence regarding that issue, move further apart rather than closer together.

Before taking this survey, how aware were you of this phenomenon?