Evaluating Model Performance

We have reached the end of this introduction to the use of R as a Geographic Information System. You have (hopefully) learned a bit about the nature of spatial data, how to manipulate and visualize spatial data, and finally, how to build predictive models based on spatial data. In this final lecture, we’ll look at how to evaluate whether the models we fit are up to the tasks we have for them. This is, necessarily, a brief introduction to a topic that could take an entire course; however, upon completion you should now be able to take a spatial analysis from start to finish without ever having to leave R.

Resources

• A review of methods for the assessment of prediction errors in conservation presence/absence models by (Fielding and Bell 1997) has an accessible overview for many of the model evaluation methods implemented in the dismo package.

• Chapters 4 and 5 in the second edition of An Introduction to Statistical Learning by (James et al. 2021) provide excellent overviews of the general problem of classification and introductions to resampling-based techniques for assessing their fit.

• Model selection using information criteria, but is the “best” model any good? by (Mac Nally et al. 2018) is an excellent reminder of the differences between a good model and the best of a group of bad models. A must-read for those hoping to use multi-model inference to learn something about their study system.

Slides

View all slides in new window Download PDF of all slides

• Evaluating model performance
• Fit and prediction
• Threshold-based techniques
• Resampling-based techniques