# Uncertainty Elicitation Tool

Welcome to the MATCH Uncertainty Elicitation Tool. Descriptions of the buttons in the toolbar at the top of the screen are given below. Instructions and further help are available by clicking on the 'Help?' button.

Total number of placed chips = 0
Median =
Lower Quartile =
Upper Quartile =
P1: P(MIN < X < ) =
P2: P( < X < MAX ) =
P0: P(MIN < X < ) =
Median =
Lower Tertile =
Upper Tertile =
Median =
P1: P( ) =
P2: P( ) =
Distribution
Auto-select best fit
 rd percentile = percentile =
0:00

## Elicitation Options

1. Set the range for the uncertain quantity by specifying a lower and upper limit. This range will be used in various plots throughout the elicitation tool. The smaller the range, the better, but you should be certain that the uncertain quantity will not lie outside this range.
2. Select the input mode you want to use, to elicit your probability judgements.
3. Click Help at any time for instructions.

## Roulette Method

• Click on the grid to allocate chips to the marked bins. The elicited probability of the uncertain quantity lying in any particular bin is represented by the proportion of chips allocated to that bin.
• You may use any number of chips in total. We suggest 10 or 20.
• You can change the number of bins used, and the height of the grid, by specifying values in the roulette options boxes.
• When you have finished allocating chips, click on Fitting & Feedback

## Quartile Method

1. Specify your median value, either by typing it in the Median box, or by clicking and dragging in the Median rectangle, using the mouse. Your probability of the uncertain quantity lying in the green section should be the same as your probability of the uncertain quantity lying in the blue section.
2. Specify your lower quartile, either by typing it in the Lower quartile box, or by clicking and dragging in the Lower quartile rectangle. Your probability of the uncertain quantity lying in the light green section should be the same as your probability of the uncertain quantity lying in the dark green section.
3. Specify your upper quartile, either by typing it in the Upper quartile box, or by clicking and dragging in the Upper quartile rectangle. Your probability of the uncertain quantity lying in the light blue section should be the same as your probability of the uncertain quantity lying in the dark blue section.
4. When you have finished, click on Fitting & Feedback on the toolbar.

## Tertile Method

1. Specify your median value, either by typing it in the Median box, or by clicking and dragging in the Median rectangle, using the mouse. Your probability of the uncertain quantity lying in the green section should be the same as your probability of the uncertain quantity lying in the blue section.
2. Specify your lower and upper turtles, either by typing them in the appropriate boxes, or by clicking and dragging in the rectangle to change the green/orange and orange/blue divides. The green, orange, and blue sections should all have the same probability of containing the uncertain quantity.
3. When you have finished, click on Fitting & Feedback on the toolbar.

## Probability Method

1. You are asked to specify three probabilities. You can type these in, or you can click and drag in the rectangles.
2. You can change which probabilities you specify, by changing the limits in the text boxes.
3. Your probabilities will need to be coherent. For example, using the default limits, P1 cannot be less than P2.
4. When you have finished, click on Fitting & Feedback on the toolbar.

## Hybrid Method

1. Specify your median value, either by typing it in the Median box, or by clicking and dragging in the Median rectangle, using the mouse. Your probability of the uncertain quantity lying in the green section should be the same as your probability of the uncertain quantity lying in the blue section.
2. You are asked to specify two probabilities. You can type these in, or you can click and drag in the rectangles.
3. Your probabilities will need to be coherent. Neither, P1 nor P2 can be greater than 0.5.
4. When you have finished, click on Fitting & Feedback on the toolbar.

## Fitting & Feedback

• Click on any distribution name to fit that distribution to the elicited judgements.
• Clicking on Auto-select best fit will show the best fitting distribution.
• You can see any two percentiles of the currently displayed distribution by clicking and dragging the green and blue sliders, or by typing the desired percentiles in the text boxes.

## Distributions

### Student-t

Note that the degrees of freedom, ν, is fixed at 3 in the elicitation tool.

### Scaled beta

Note that L and U are the lower and upper limits in your chosen range.

### Log Student-t

Note that the degrees of freedom, ν, is fixed at 3 in the elicitation tool.

## Toolbar Legend

 Help Options Display/hide the Help window Display/hide the Elicition Options window Display/hide the Elicited distribution and the Fitting and Feedback window Display/hide the Fitting parameters of the Elicited distribution Share this elicition with another user over the Internet. Replay this elicition session (only available once the elicitation session has been complete) Complete this elicitation session Restart the elicitation session from the beginning Display/hide information about us, the tool, and methods to contact us Citations If you use this tool in any published research, please could you cite it as: David E. Morris, Jeremy E. Oakley, John A. Crowe, A web-based tool for eliciting probability distributions from experts, Environmental Modelling & Software, Volume 52, February 2014, Pages 1-4, ISSN 1364-8152, http://dx.doi.org/10.1016/j.envsoft.2013.10.010.

This tool was produced by Ed Morris and Jeremy Oakley as part of the MATCH project, and you are welcome to contact us with comments or questions.

The tool is based on the freely available SHELF package of elicitation templates and software, and further guidance on elicitation is available within the SHELF package. A recommended textbook on elicitation methods is

O' Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. E., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006). Uncertain Judgements: Eliciting Expert Probabilities. Chichester: Wiley.

## Citations

If you use this tool in any published research, please could you cite it as:

David E. Morris, Jeremy E. Oakley, John A. Crowe, A web-based tool for eliciting probability distributions from experts, Environmental Modelling & Software, Volume 52, February 2014, Pages 1-4, ISSN 1364-8152, http://dx.doi.org/10.1016/j.envsoft.2013.10.010.

Ranges
 Lower Limit Upper Limit

Input Mode
 Roulette  • Quartile  • Tertile  • Probability  • Hybrid  •
Roulette Options
 Number of Bins Height of Grid

Are you sure that you want to complete this eliciation session?

No more changes can be made after the session is completed and this process cannot be undone.

You can only open the Fitting & Feedback window once you have selected an input mode.

Are you sure that you want to restart the eliciation session?

The current session will be completed and no more changes can be made. This process cannot be undone.

You can only open the Text Output window once you have selected an input mode.