Theory Mapping Pilot test: Causes of climate change

f
Exciting news! The first pilot debate using Theory Mapping is going to be carried out in conjunction with Austhink and the Open University. The subject is the causes of climate change, and the pilot should be finished by the end of April 2009. Since it will be a pilot test, the results will only be released if the debating parties agree. However, it is the start of a process that will resolve the controversial debate over the causes of climate change by demonstrating which theory can really provide the most coherent explanation of the facts - is it due to human activity as the establishment claims, or is it more complicated than that, as the climate sceptics argue? If you feel strongly about climate change one way or the other, then why not join one of the debating teams? Please contact me for more information.


What?

The details of the debate are as follows:

Topic: 'What is the cause of climate change since 1950?'.

Objectives: the main objectives of the debate are:

• Agreement on the facts: a table of facts about climate change will be drawn up that both sides agree to. This will not only pave the way for the public Theory Mapping debate (see below), but can be used by anyone as a genuinely neutral account of how the world's climate is changing and what is already known for sure about what is behind it.
  
• Develop the first Theory Maps of each side: each Map will provide the clearest and most comprehensive account of each sides argument, and will feed into a public Theory Mapping debate (see below) that will identify which theory can provide the most coherent explanation of the facts. A very early draft of a Theory Map of the anthropogenic case (with some possible Agreed Facts) is here. This is based on a quick reading of the IPCC AR4 report. Objections in red flag up areas that need more work.
• Evaluation of the Theory Mapping method: an independent evaluation will be carried out by Anna De Liddo from the Knowledge Media Institute (KMI) of the UK's Open University. For some of the key issues that will be evaluated, see this draft evaluation framework.

Participants: this would be between a proponent of the view that human activities are the main cause (anthropogenic theory) and a sceptic, possibly with teams providing them with support.

Facilitator: Strahan Spencer, from whatbeliefs.com.

Public access: given that the debate is a pilot, the maps and relative scores will only be publicly released if both parties agree.

Sponsorship: this debate is being sponsored by Austhink, creators of bCisive, currently the best commercial-grade argument mapping software in the world.

Timing: the debate will run during March and April 2009.


Why?

I think that climate change is a good topic to pilot test Theory Mapping for the following reasons:

1. ESSENCE initiative:
The global internet dialogue and debate experiment ESSENCE (E-Science/Sensemaking/Climate Change) is testing different computerised methods for facilitating debate about climate change. This can provide a good framework for identifying the value of Theory Mapping.

2. Problems with the debate:
From an official perspective the debate about to what extent global warming is occurring and is caused by human activities is now over, and the focus should be on deciding on an appropriate international policy response. However, there would still seem to be significant pockets of climate scepticism. For instance, the Climate Sceptics political party was etablished in Australia in February 2009, strongly denying that global warming is caused by human activities. Some of the reasons for this continued difference of opinion include:

• Lack of an objective method for selecting theories: Theory Mapping is based on the view that in areas where controlled experiment is not possible (and so theories cannot be falsified using the scientific method), society currently lacks a method for selecting the best theories. This applies to climate change since, as the IPCC AR4 2007 report points out, a controlled experiment cannot be done on the world's climate! (AR4 Chap 9, p668). Theory Mapping provides such a method, in which the public Theory Mapping debate will result in a clear ranking of which theories provide the most coherent explanation of the facts about climate change. This is therefore an opportunity for the anthropogenic theory to conclusively demonstrate that it is the best, and hence bring closure to the issue. But it is also an opportunity for the sceptics to show that their preferred explanation for global warming is equally if not more coherent, and so greatly increase their credibility.
• Different views over levels of scientific proof: it is argued by some sceptics that the degree of certainty is not high enough to warrant action. A key benefit of Theory Mapping is to uncover different underlying assumptions and require that they be coherently justified, so a Theory Mapping Debate may be able to resolve this issue.
• Disagreement over the 'facts': each side quote facts to support their view that the other side disagrees on e.g. sceptics often challenge conventional scientific estimates of how important carbon dioxide is compared to other greenhouse gases. Theory Mapping requires that for a fact to be given as evidence for a theory, it must be first agreed between both sides. If they cannot reach agreement, they then unpack the 'fact' into its different sources until they hit the 'raw' facts that are publicly verifiable and so cannot be disagreed over e.g. that scientific study A using method S concluded levels of carbon dioxide at Y, while study B using method T concluded levels of X. Each side then needs to come up with a coherent interpretation of those studies that fits their overall case e.g. if the sceptic disagrees with the conclusion of study B, he/she must mount a coherent attack on the method T that they used.
• Different interests: it may be that both sides of the debate have different vested interests, which makes agreement harder. In scoring theories in a completely objective way, Theory Mapping can rise above this.

3. High-profile: as a high profile issue it will be easier to find people interested in participating, and easier to get funding for the public debate (see below).

4. Academic: much of the debate over climate change is between qualified scientists, and so this is a good test of how well Theory Mapping can be used to improve the efficiency of academic debates.


When can the debate be made public?

If the first pilot is successful, I would seek funding for a full public Theory Mapping Debate with high profile scientists on both sides, ideally before the United Nations Climate Change conference in Dec 2009. This would later be opened up for public participation as a Theory Mapping Competition to find the most coherent explanation of the facts (for the difference between a Theory Mapping Debate and Competition, see the post Modes of Application). As a result of this process, it will be possible to resolve the debate over the causes of climate change so that all attention can be focused on the policy implications.

f

Problems that Theory Mapping solves

f
The value of Theory Mapping is in solving two key problems that plague any debate between people holding to different theories:

• Lack of clear communication: much time is wasted in debates because neither side fully appreciates the position of the other. This may be broken down into three aspects. Firstly, each side basing their arguments on implicit assumptions that even they may not be consciously aware of. Theories are like icebergs, in which most of the structure lies below the surface in the form of implicit assumptions. If these assumptions are not exposed there is no hope of properly understanding each theory and resolving the debate. Secondly, lack of clear expression from each side of their ideas and arguments. And thirdly, neither side paying enough attention to really understand the other's position. Theory Mapping addressing all of these aspects. The first two are dealt with by using Argument Maps which ensures complete clarity of the arguments and that all assumptions are uncovered. The third is dealt with by providing a process which requires each side to study the other's position in detail.
• Lack of an agreed truth test: we would say that it was crazy to have a race without an agreed course and finishing line. Everyone would either run in different directions and claim themselves to be the winner, or simply not bother competing. Yet amazingly that is the situation we are in when it comes to competitions between theories, particularly outside the physical sciences where controlled experiments are generally not possible. Beyond a vague requirement that theories fit the facts, there is no clear procedure to be followed to ensure that theories are trying to explain the same thing and can be easily compared (the course) and no objective measure of which theory fits the facts best and is hence most likely to be true (the finishing line). As in the race analogy, this not only results in massive inefficiency as people's incentives are not aligned with the objective of identifying the most accurate theories, but also reduces people's participation in debates since their accepted worldview is never objectively challenged. Theory Mapping therefore provides rules to ensure the comparability of theories, and an objective quantitative truth test based on the latest thinking in Epistemology.

f

Theory Mapping FAQs

f
The following questions are answered:

• What is Theory Mapping?
• Are there example Theory Maps I can look at?
• What is the vision for Theory Mapping?
  
• Why Theory Mapping?
• Where can Theory Mapping be used?
• What are the key objections to Theory Mapping?
• Has Theory Mapping been tested?
• How to implement Theory Mapping?
• How can I help?
• Why the change in name?
  

What is Theory Mapping?

Theory Mapping is a new and potentially revolutionary method to improve the quality of theories that society uses. It may be described in terms of its Features, Process and Modes of Application:

• Features: it consists of the unique combination of five features: 1) Argument Mapping; 2) Ability to map complex theories; 3) Comparability of Theory Maps; 4) Quantitative test of the truth; and 5) Web-based.

• Process: it consists of the following steps: 1) State the Research Question; 2) Collection of Facts; 3) Draft Theory Map to fit the facts; 4) Measurement of coherence; 5) Iteration of previous stages to increase coherence.
  
• Modes of application: the two main modes for which it can be most effectively used are: 1) Theory Mapping Debate, in which two or more sides together agree on the facts to be explained and try to compete to provide the most coherent explanation of the facts; and 2) Theory Mapping Competition, in which one organisation sponsors the competition and specifies the facts to be explained, and anyone is free to enter Maps providing the most coherent explanation of them.

Are there example Theory Maps I can look at?

Yes, the following two maps were written using the bCisive argument mapping software. They are zip files, so you will need to extract them first. Then open the html file to see the map. This gives read-only access. To edit you need to buy the software or get a free trial.

• Example Theory Map: Lone Gunman Theory for 'Who shot JFK?' (most advanced);
  
• Example Theory Map: Anthropogenic Theory for 'What is the cause of climate change since 1950?' (very draft stage)
  

They are only drafts to help to stimulate thought in preparation for the pilot debates. The red Objection boxes highlight the key areas of incoherence. The blue boxes contain the titles of Agreed Facts, the details of which can be found in the Agreed Facts excel spreadsheet (accessed by clicking on the link in the Situation box at the top).

Alternatively, if you just want simple Pdfs without the hassle of zip files (but minus the linked spreadsheets), click here: JFK map Pdf; Climate change map Pdf.


What is the vision for Theory Mapping?

The vision is to end up with a website containing league tables of competing theories in every area of human thought. Visitors to the website can either build up their belief system by choosing the highest scoring theories, or get involved in online Theory Mapping Debates or Competitions to try to raise the ranking of their own preferred theories. These Debates and Competitions would be sponsored by various organisations, including government research funding agencies in recognition of the efficiency of the method in effectively harnessing the world's collective intelligence.


Why Theory Mapping?

For those areas of inquiry which are not amenable to controlled experimentation, Theory Mapping enables the generation, communication, critique, refinement and selection of theories better than other means of debate e.g. academic journals, face-to-face debate, reports, discussion boards, blogs etc. It does this by solving two key problems that plague debates between different theories: lack of clear communication; and lack of an agreed test of the truth. The analogy is of blind people running a race without a agreed course and an agreed finishing line - the result is pretty inefficient! Theory Mapping provides the glasses (Theory Maps), the course (a clear process) and the finishing line (agreed truth test).

For more information see the following:

• Argument Map of the case for Theory Mapping: lays the case out very logically! Click the link to download, unzip and open the html file to see the map.
  
• Benefits of its features: each of its five features brings particular benefits to the table. See the FAQ What is Theory Mapping? for links to posts on each of the features, each of which includes an explanation of the benefits.

Where can Theory Mapping be used?

Theory Mapping can be applied in the following areas (click on the titles below for more information):

• World views/belief systems/ideologies: allows the possibility of a genuine debate between world views that all sides can sign up to, allows people to decide between world views on a rational basis, and improves mutual respect between world views;
• Academic research: enabling anyone in the world to contribute their ideas and knowledge to academic debates;
• Intelligence analysis: combining the strengths of both Analysis of Competing Hypotheses and Hypothesis Mapping for developing reliable theories out of a mass of evidence.
• Theology: a tool to allow the theologians of any religion to identify the most coherent interpretation of their holy books and traditions.
• Education: Theory Mapping can be used to assist in the teaching of any theory.
• Epistemology: researching the results of Theory Mapping can help to resolve debate about the structure of knowledge between Foundationalism and Coherentism.

What are the key objections to Theory Mapping?

Sounds too good to be true? For my responses to some possible objections see:

• Objection: People not interested in rational argument;
  
• Objection: Test of the truth is too weak;
  
• Objection: High risk of deadlock;
• Objection: Too complicated;
  
• Objection: Bad communication tool.
  

Has Theory Mapping been tested?

Not yet, but I am currently working on the following pilot tests:

• Pilot test: Causes of climate change: a way to test how well Theory Mapping can improve the efficiency of debates within academia. In association with the Open University and sponsored by Austhink. Should be completed by April 2009.
  
• Pilot test: JFK assassination: a way to test how well Theory Mapping can improve the efficiency of debates between different world views.
  

How to implement Theory Mapping?

Some of the issues involved in acheiving the vision for Theory Mapping are covered in the following posts:

• Organisation and Funding: what kind of organisation can best promote Theory Mapping, and how can it be financed?
• Software: what software development is needed to support Theory Mapping?
• Website: what kind of website is needed to support Theory Mapping?

How can I help?

• Comment: I really welcome your feedback both positive and negative. Please add your comments to the most relevant post and I will do my best to reply and/or incorporate your views.
• Participate: if you have strong views on either climate change or JFK's assassination then you are very welcome to get involved in the pilot tests as a participant. Or if you would like to help set up a debate on another topic let me know.
  
• Support: if you have good IT skills I would appreciate help in setting up the Theory Mapping website!
• Spread the word! if you think that Theory Mapping is an exciting idea, then let others know about this site.
  

I can be contacted at strahanspencerATgmail.com.


Why the change in name?

In previous posts I have called the method Belief Mapping, but now believe that Theory Mapping is a more accurate description, since what distinguishes it from other visual argumentation approaches (such as Argument Mapping, Concept Mapping, Issue Mapping etc.) is that it is specifically about the visualization and evaluation of theories.
f
f

Objection: People not interested in rational argument

f
Objection:

People are generally not interested in deciding on their beliefs rationally, and so would not be interested in participating in Theory Mapping, and would not be influenced by the results. This may be for various reasons:

• Meaning and value: their existing beliefs provide meaning to their lives. For instance, I have heard it argued that conspiracy theorists gain a sense of meaning from their mission to uncover conspiracies, and so would strongly resist any evidence to the contrary.
• Perceived self-interest: their existing beliefs may be more convenient. For instance, I have heard it argued that one reason for scepticism about global warming is because it is an 'inconvenient truth' which would require making sacrifices. Cognitive dissonance theory also argues that we have inertia against changing our behaviour, so if information comes in which is inconsistent with our behaviour our first response is to try to dismiss the information (cognitive dissonance theory is about psychological discomfort that is caused by inconsistency within our beliefs and actions).
  

Responses:

1. Number of irrational people overestimated
From talking to people on both sides of a debate, I find that both like to portray the other side as not open to rational debate. For instance, atheists see Christians as irrational, while Christians see atheists as irrational! This is because up until now there has not been a method to allow debates to be conducted efficiently, and so people get frustrated when trying to engage others in dialogue. Theory Mapping can solve this problem.

2. Theory Mapping would increase people's incentive to think rationally
Theory Mapping will provide objective evidence about the incoherency of particular theories and world views, which will be hard to ignore. This will create much higher cognitive dissonance than other methods of debate, and so will force people to either change their beliefs or engage in the debate to defend them.

3. Those who genuinely are not interested in rational argument are not important
Yes, there will always be some people who are not interested in logical argument. But by not being willing to engage in a rational debate (or if they do engage, not being willing to play by the rules) they would be shown for who they are and hence marginalised.
f

Objection: Poor communication

f
Objection:

Well-written prose is a better way of communicating complex theories than Theory Mapping, for the following reasons:

• Visual element adds confusion: people have different levels of visual-spatial ability and so many people will find Theory Maps difficult to process. One friend has talked to me of his mind 'shutting down' when he comes across a complex diagram. Although it is true that in some cases "a picture can paint a thousand words", this is not the case with Argument Mapping since the space on the page does not correspond to a conceptual dimension, unlike with a graph for instance.
  
• Theory Maps too big and complicated: a Theory Map of a complex theory would be be very large and unweildy. Even to get an overview of the key reasons at the top would require a lot of navigation around.

To improve the communication of theories it would therefore be more effective to focus on improving how well prose is written.

Responses:

1. Prose is complementary not a substitute
At the beginning of the post Introducing Theory Mapping, it is stated that one of the problems that Theory Mapping is trying to solve is 'Lack of clear communication' between sides in a debate. However, this is broken down into three particular aspects: a) not communicating assumptions; b) lack of clear expression of ideas and arguments; and c) people not paying enough attention. Whereas the objection seems to be focusing on a fourth aspect (d)), which is purely about the means of communication. Points a), b) and d) can all be given as advantages that Argument Mapping has over ordinary prose (see the post on Argument Mapping), but the case for Theory Mapping only needs to rest on a) and b). This is because prose may be seen as complementary to Theory Mapping e.g. you construct the Theory Map first to uncover the assumptions and gain greater clarity over the ideas and arguments, which then enables you to write a summary in prose to communicate it in a user-friendly way.

2. Argument Maps may be a clearer means of communication than prose
Since Argument Maps can lay out the structure of an argument in a much clearer way than prose and can ensure that rhetoric and extraneous information is removed, it can be argued that they can be clearer than prose under the following conditions:

• Good software: this can make them much easier and fun to read e.g. see aMaps for an example of how to present them in a visually arresting way.
  
• Argument Map literacy: it has been pointed out by Simon Buckingham Shum (here) that with regard to argument visualization tools we need “a new literacy in being able to read and write in the new medium, and a new fluency in using these conversational tools in appropriate ways in different contexts.” As argument mapping becomes more widely used, so this literacy will develop.

f

Theory Mapping Truth Test

f
What?

A test is applied to the Theory Maps to assess which theory has the highest probability of being true. This test is the coherent explanations of facts.

Why?

There are a number of benefits to having a test of the truth:

• Ease of theory selection: Theory Mapping only deals with theories concerned with matters of truth not value, where truth is defined as the accurate prediction of sense experiences (see the post Truth - what is it? for a justification of this) i.e. it is only concerned about what 'is' rather than what 'ought' to be. Theories must therefore be selected primarily according to their relatively probability of being true. The theory which scores the best is not proven to be true, but is the most rational to act on given our current knowledge of the facts and the range of theories available.
  
• Correct incentives: if you have an effective test of the truth that everyone can agree to, and there are rewards gained from a theory scoring well (whether in terms of social esteem or money), you have a very powerful focusing of human intellectual energy since everyone's incentives are aligned with finding the truth. The scientific community has this as a result of the scientific method, but many other areas of intellectual endeavour do not. As a result, incentives are distorted, with people focusing on gaining esteem or money directly in ways that either waste time (e.g. academics focusing on writing papers to impress or trying to ruin each others reputations), or are downright damaging (e.g. misinformation and fraud). For how this applies to particular areas, see Theory Mapping and World Views, and Theory Mapping and Academic Research.
  

In terms of why I am suggesting the coherent explanation of facts as the truth test in Theory Mapping, see the following posts: Truth – how to find it?, Coherent explanation of the facts, and Theory Mapping and Epistemology.

How?

There are three key parts involved in the coherence test (to understand this you should first read about Stages 1, 2 and 3 in the Process of Theory Mapping):

1. Explanatory power
This is how well the theory is able to explain and fit the Agreed Facts. The primary purpose of a theory in Theory Mapping is to give an account of reality that can explain and fit our observations of it as represented by the Agreed Facts. I would suggest that the best way to measure this, is to require that every Agreed Fact be ‘explained’ by the theory in at least one of three ways:

• Provide an answer to the Research Question, which will explain a large number of the facts automatically;
• Use the Agreed Fact as evidence in support of an argument, since the argument will also explain the Fact;
• Provide a separate explanation for any Agreed Facts that have not been covered by the other two methods. This is most likely to be the case for Agreed Facts that support competing theories e.g. in the JFK case, the evidence for conspiracy.

The measure is then simply to go through the table of Agreed Facts that has been constructed in Step 2 and count how many that are not explained in any of these three ways.

If some Agreed Facts are considered to be more important than others (e.g. for the JFK case if physical evidence is considered more reliable than eyewitness evidence), a weighted sum could be taken, with the weights varying according to the level of importance.

In the map of the Lone Gunman Theory, 3 facts are highlighted that do not have adequate explanations, which are some of the facts put forward in support of a conspiracy (I am sure that some people have come up with explanations, but I have not put them in for illustrative purposes).

2. Epistemic justification
This is how well justified the theory is as being true, in terms of logically valid arguments grounded in either Agreed Facts (which are directly related to the Research Question) or Background Facts (which are not).

This would be analysed by looking at the Theory Map and highlighting:

• Explanations or Reasons/premises that were not supported by logically valid arguments;
• Reasons/premises that were not supported by Agreed Facts or Background Facts.
  

The number of incoherencies would then be added up to get the measure. For instance, for the map of the Lone Gunman Theory, 8 premises are identified as not being adequately supported, with Objections attached to them (again, I am sure that stronger justifications do exist, this is just for illustration).

If some reasons/premises are considered to be more important than others because the theory is more dependent upon them, a weighted sum could be taken, with the weights varying according to the number of arguments that each premise supports.

3. Internal consistency
This is whether there are any explanations, reasons/premises that are logically inconsistent with each other. This would again be analysed by looking at the Theory Map, highlighting and counting the inconsistencies.

No areas of inconsistency are identified in the map of the Lone Gunman Theory.

Incoherence Score
The overall measure (or Incoherence Score) of the Theory Map is then presented as a combination of the three numbers. For instance, with the JFK Theory Map for the Lone Gunman Theory, the incoherence score would be 3:8:0 i.e. 3 facts not adequately explained, 8 premises without adequate justification, and no inconsistencies.

The Incoherence Scores of the competing theories would then be compared, with the lowest scoring theory seen to be the one with the highest probability of being true. This does not amount to proof that it is true, just that it would be the most rational theory to act on given our current knowledge of the facts and the range of theories available.

Whether these three numbers could be combined into an overall number is something that would need to be worked on. Any measure should ensure that it does not create perverse incentives, and can be agreed upon by all participants. For instance, I was previously thinking of taking the total percentage of coherent beliefs in the system as the measure (which I called the Coherence Quotient, CQ), since it could combine all three scores into one. However, this would be very susceptible to manipulation, since you could reduce the impact of a given number of incoherencies on the CQ measure simply by inflating the total number of beliefs in the system.
f

Comparability of Theory Maps

f
What?

Theory Maps about competing theories can be easily compared.

Why?

Two reasons:

• Ease of communication: by following common conventions it easy to read and understand them.
  
• Ease of selection: comparability allows a common test for the truth to be applied to allow one to select theory that has the greatest probability of being true.

How?

Theory Maps for competing theories have to follow common rules, which include:

• Layout conventions: see the Process of Theory Mapping.
  
• Stick to the facts: evidence to support arguments can only come from an agreed set of facts, with rules concerning what constitutes a fact and what information should be provided about the fact (see Stage 2: Collection of Facts in Process of Theory Mapping) .
  
• Explaining counter-evidence: each theory must provide explanations of the facts used to support competing theories (see Stage 3: Draft Theory Map to fit the facts in Process of Theory Mapping).

f