Banner

6.13 Model Selection and Troubleshooting¶

Part: 6 — Geographically Weighted Models (Chapter 6.13 of 14)

Dataset: diabetes_northeast.shp (217 counties)

Focus: which model answers which question; four real failure modes, triggered and diagnosed live

Library: spatialstats.gwmodel (PySpatialStats)

Table of Contents¶

  1. Overview

  2. Python Setup

  3. Research Question to Model

  4. Troubleshooting: Four Real Failure Modes

    • 4.1 Singular or near-singular local design matrices

    • 4.2 Bandwidth pinned at a search bound

    • 4.3 Slow fits

    • 4.4 Missing optional dependencies

  5. Summary and Conclusions

  6. Resources

1. Overview¶

Fourteen chapters have covered a lot of ground: six kernels, two bandwidth-selection strategies, GWR, MGWR, MixedGWR, GWGLM, ensemble and deep variants, a spatiotemporal extension, and a network-distance variant. This chapter is the consolidation — a table for choosing among them, and a short, hands-on troubleshooting guide for the four failure modes most likely to show up in practice, each one triggered and diagnosed with real code rather than described in the abstract.

Step What we do
Choose A research-question-to-model table, drawing on every model this Part has covered
Diagnose Four real failure modes — singular matrices, bandwidth at a bound, slow fits, a missing optional dependency — each triggered live

2. Python Setup¶

In [1]:
import sys, time, warnings
from pathlib import Path

import numpy as np
import pandas as pd
import geopandas as gpd

warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)


def find_repo_root(start: Path = Path.cwd()) -> Path:
    for p in [start, *start.parents]:
        if (p / "pyproject.toml").exists() and (p / "data").is_dir():
            return p
    raise FileNotFoundError("Run this notebook from inside the PySpatialStats repository.")


ROOT = find_repo_root()
DATA = ROOT / "data"
try:
    import spatialstats
except ImportError:
    sys.path.insert(0, str(ROOT))
    import spatialstats

from spatialstats.gwmodel.linear import GWR
from spatialstats.gwmodel.core import BandwidthSelector, SpatialKernel
from spatialstats.gwmodel.diagnostics import local_condition_number
from spatialstats.gwmodel.utils.distance import pairwise_distances

northeast = gpd.read_file(DATA / "diabetes_northeast.shp")
covariates = ["Obesity", "PhysInact", "Acc_Exer", "PCP_rate", "FoodEnvIdx", "SVI"]
X = northeast[covariates].to_numpy()
y = northeast["Dia_pct"].to_numpy()
coords = northeast[["x", "y"]].to_numpy()
print(f"spatialstats : version {spatialstats.__version__}")

spatialstats : version 0.2.0

3. Research Question to Model¶

Research question Reach for Why (this Part's chapters)
Does one relationship hold everywhere, or does it vary spatially? GWR vs. a global OLS/SLM/SEM comparison Chapter 6.5 (fit/predict/score), Chapter 6.8 (side-by-side table)
Which covariates vary locally and which are essentially global? MGWR Chapter 6.6 — per-covariate bandwidths distinguish near-stationary (large bandwidth) from sharply local (small bandwidth) covariates
I already know some covariates should be global and others local MixedGWR Chapter 6.6 — fixed_vars / varying_vars set that structure explicitly instead of inferring it
The outcome is a count or a proportion, not continuous GWGLM (family='poisson'/'binomial') Chapter 6.6 — but note the missing offset argument makes it unsuitable for exposure-adjusted rates (Part 5's domain) without a workaround
Exploring local means/variances/correlations before modeling anything GWSS Chapter 6.4
I care about predictive accuracy more than interpretable local coefficients GWRandomForest, GWXGBoost, GWLightGBM Chapter 6.9 — with spatial block CV, not row-order or random splits
Data spans both space and time GTWR (linear) or GTNNWR (deep) Chapter 6.10, 6.11 — but neither's predict() currently uses the times argument; both are fit-only for understanding coefficient evolution
Distance should follow a road/river network, not straight lines NetworkGWR Chapter 6.11 — needs tuple-style node labels; computes hop count, not weighted network distance
I want the local weighting function learned from data rather than assumed GNNWR Chapter 6.11 — check coef_.std(axis=0) before trusting the result; it converged to near-OLS with modest settings on this data
A handful of covariates, dozens to a few hundred locations, need genuine dimensionality reduction that varies spatially GWPCA Chapter 6.6
Is the (G)WR residual autocorrelation actually gone? spatialstats.esda.Moran on .residuals_, not gwmodel.diagnostics.moran_test Chapter 6.7 — the diagnostics module's own version has a verified S0 bug
Are two nested (G)WR specifications significantly different? Chapter 6.7's GWR.monte_carlo_test, not diagnostics.monte_carlo_variability Chapter 6.7 — the latter has a verified permutation-invariance bug

4. Troubleshooting: Four Real Failure Modes¶

Each of the four failure modes the outline calls out is triggered here with real code, not just described.

4.1 Singular or near-singular local design matrices¶

The intuitive worry — a local weighted regression with too few effective observations should crash — turns out not to be what actually happens in this library. It fails silently instead:

In [2]:
for bw in [5, 6, 7, 91]:
    m = GWR(bandwidth=bw, fixed=False, kernel="bisquare", n_jobs=2)
    m.fit(X, y, coords)
    print(f"bandwidth={bw:3d}  (k={X.shape[1]+1} params)   "
          f"R2={m.score(X, y, coords):.4f}   any NaN coef: {np.isnan(m.coef_).any()}")

bandwidth=  5  (k=7 params)   R2=1.0000   any NaN coef: False

bandwidth=  6  (k=7 params)   R2=1.0000   any NaN coef: False
bandwidth=  7  (k=7 params)   R2=1.0000   any NaN coef: False

bandwidth= 91  (k=7 params)   R2=0.8224   any NaN coef: False

With only 7 parameters to estimate (intercept + 6 covariates), an adaptive bandwidth of 5, 6, or 7 nearest neighbours leaves an exactly- or under-determined local system — and the local weighted least-squares solver (evidently a pseudoinverse-based solve, not a direct one that would raise LinAlgError) returns a perfect R² of 1.0000 rather than an error. This is the real danger: a suspiciously perfect local fit is not a success, it is a symptom that the bandwidth is too small relative to the number of covariates. Always check that the adaptive bandwidth is comfortably larger than the number of parameters (a common rule of thumb: at least 2–4x); do not trust a local $R^2$ near 1.0 without checking this first.

A related, subtler failure — exactly collinear covariates, at any bandwidth — is worth checking with local_condition_number rather than waiting for a crash that may never come:

In [3]:
Xi = np.column_stack([np.ones(len(X)), X])
X_collinear = np.column_stack([X, X[:, 0] * 2.0])           # an exact duplicate (rescaled) column
Xi_bad = np.column_stack([np.ones(len(X_collinear)), X_collinear])

D = pairwise_distances(coords)
kernel_obj = SpatialKernel(kernel="bisquare", fixed=False, bandwidth=91)

cn_clean = local_condition_number(None, Xi, D, kernel_obj)
cn_bad = local_condition_number(None, Xi_bad, D, kernel_obj)

m_bad = GWR(bandwidth=91, fixed=False, kernel="bisquare", n_jobs=2)
m_bad.fit(X_collinear, y, coords)

print(f"clean design      : condition number {cn_clean.min():.2e} to {cn_clean.max():.2e}")
print(f"collinear design   : condition number {cn_bad.min():.2e} to {cn_bad.max():.2e}")
print(f"collinear fit still 'succeeds': any NaN coef = {np.isnan(m_bad.coef_).any()}")

clean design      : condition number 7.98e+06 to 1.87e+07
collinear design   : condition number 4.72e+16 to 6.67e+20
collinear fit still 'succeeds': any NaN coef = False

The collinear design's condition number is 8 to 13 orders of magnitude larger than the clean design's — confirming local_condition_number catches what .fit() alone silently absorbs. One caveat worth flagging honestly: even the "clean" design here has condition numbers in the millions, not the tens or hundreds textbook thresholds suggest — almost certainly because Obesity/PhysInact (percentages, tens) and FoodEnvIdx (a small float) sit on wildly different scales next to an unscaled intercept column. Standardize covariates before reading too much into an absolute condition-number threshold; use local_condition_number to compare relative values (clean vs. suspect) rather than checking against a fixed textbook cutoff.

4.2 Bandwidth pinned at a search bound¶

BandwidthSelector searches within [bw_min, bw_max]. If the truly optimal bandwidth lies outside that range, the selector will not — cannot — find it; it returns a value at or near the boundary instead, silently:

In [4]:
def score_fn(bw):
    m = GWR(bandwidth=float(bw), fixed=False, kernel="bisquare", n_jobs=2)
    m.fit(X, y, coords)
    return m.aicc_


selector_narrow = BandwidthSelector(score_fn, fixed=False, bw_min=20, bw_max=40, criterion="AICc", tol=1.0)
bw_opt, aicc_opt = selector_narrow.golden_section()
print(f"search range [20, 40] -> optimal bandwidth = {bw_opt:.2f}  (AICc = {aicc_opt:.2f})")
print(f"within 2 units of the upper bound: {abs(bw_opt - 40) < 2}")

search range [20, 40] -> optimal bandwidth = 39.66  (AICc = 8.10)
within 2 units of the upper bound: True

The result lands within 1 unit of the artificially narrow upper bound of 40 — the selector is telling us, in effect, "give me more room to search." Always check whether the returned bandwidth sits near bw_min or bw_max (as several of Chapter 6.6's MGWR per-covariate bandwidths did, at the minimum of 8) before accepting it: a boundary solution usually means the search range needs widening, not that the boundary value is truly optimal.

4.3 Slow fits¶

Chapter 6.12 already measured this directly rather than asserting it here again: a single GWR fit went from a fraction of a second at $n=217$ to tens of seconds at $n=3{,}107$, with growth steeper than quadratic in the upper range — worse than a naive extrapolation from small-$n$ timings would suggest. The practical response, also established there: always pass n_jobs= explicitly (Chapter 6.12, Section 3.1 found the global set_compute_options does not reach GWR/MGWR/MixedGWR on its own), and measure at or near the actual target sample size rather than trusting an extrapolation from a much smaller pilot run.

4.4 Missing optional dependencies¶

The deep models (GNNWR, GTNNWR, Chapter 6.11) are built so that constructing the model never requires PyTorch — only .fit()/.predict() do. PyTorch is installed in this environment, so the failure path can be triggered directly, honestly, by temporarily flipping the module's own availability flag rather than actually uninstalling anything:

In [5]:
import spatialstats.gwmodel.deep.gnnwr as gnnwr_module
from spatialstats.gwmodel.deep import GNNWR

gnnwr_module.TORCH_AVAILABLE = False   # simulate "PyTorch not installed", without touching the real environment
try:
    m = GNNWR(n_epochs=2)
    print("construction succeeded without PyTorch:", m.__class__.__name__)
    m.fit(X[:20], y[:20], coords[:20])
except ImportError as e:
    print(f"fit() raised ImportError as documented: {e}")
finally:
    gnnwr_module.TORCH_AVAILABLE = True   # restore real state immediately

construction succeeded without PyTorch: GNNWR
fit() raised ImportError as documented: PyTorch required: pip install torch

Confirmed exactly as documented: GNNWR(...) constructs successfully regardless of PyTorch's availability, and only .fit() raises — with a message naming the missing package and the install command (pip install torch) — deferring the failure to the last possible moment rather than blocking an entire script at import time. The same guard pattern applies to GTNNWR.

5. Summary and Conclusions¶

This chapter consolidated fourteen chapters' worth of models into one selection table, then triggered — not just described — the four failure modes most likely to matter in practice.

What we did¶

  1. Built a research-question-to-model table spanning every estimator in this Part.
  2. Triggered a too-small-bandwidth failure directly: bandwidths at or below the parameter count produced a deceptive, perfect local $R^2$ rather than a helpful error.
  3. Triggered a collinearity failure directly: .fit() absorbed exact collinearity silently, but local_condition_number caught it with an 8-to-13-order-of-magnitude jump — while also revealing that even "healthy" fits here have surprisingly high condition numbers due to unstandardized covariate scales.
  4. Triggered a bandwidth-at-bounds failure directly: an artificially narrow search range produced a boundary solution, exactly as it would with a real range that happened to be too narrow.
  5. Triggered the missing-PyTorch failure directly: GNNWR() constructs fine, .fit() raises ImportError with an actionable message — confirmed by flipping the library's own availability flag rather than merely trusting the documentation.

Practical takeaways¶

  • A local $R^2$ suspiciously close to 1.0 is a red flag, not a success — check the bandwidth against the parameter count first.
  • Run local_condition_number on any fit before trusting its coefficients, and standardize covariates first so the resulting numbers are comparable to a meaningful baseline rather than an arbitrary textbook cutoff.
  • Check whether a selected bandwidth sits at a search boundary before accepting it.
  • For slow fits, see Chapter 6.12 directly rather than guessing at scaling behavior.
  • Deep models fail late and informatively (at .fit(), not at construction) when an optional dependency is missing — write code that constructs a full model dictionary up front and only fits what it needs.

Next steps¶

Chapter 6.14 (Exercises) closes out this Part with four hands-on tasks that draw directly on the models, datasets, and diagnostics introduced across the previous thirteen chapters.

6. Resources¶

Textbooks¶

Reference Notes
Fotheringham, A. S., Brunsdon, C. & Charlton, M. (2002). Geographically Weighted Regression. Wiley. Chapters 2–4 cover the numerical pitfalls behind Section 4.1–4.2
Wheeler, D. & Tiefelsdorf, M. (2005). Multicollinearity and correlation among local regression coefficients in geographically weighted regression. Journal of Geographical Systems, 7, 161–187. The classic reference on local collinearity in GWR, directly relevant to Section 4.1

In this library¶

Object Role
spatialstats.gwmodel.diagnostics.local_condition_number Section 4.1's collinearity check
spatialstats.gwmodel.core.BandwidthSelector Section 4.2
spatialstats.gwmodel.deep.gnnwr.TORCH_AVAILABLE Section 4.4's guard flag
Chapter 6.12 Section 4.3's slow-fit findings, referenced rather than re-measured
Chapters 6.1–6.11 Every row of Section 3's table