Banner

4.1 Concepts and Overview¶

Part: 4 — Spatial Cluster Analysis (Chapter 4.1 of 8)

Dataset: diabetes_northeast.shp (217 counties)

Focus: The four distinct meanings of "cluster" in spatial analysis, and which tool answers which question

Library: spatialstats.cluster (PySpatialStats)

Table of Contents¶

  1. Overview

  2. Python Setup

  3. Four Questions, Four Families

  4. A First Look at Each Family on the Same Data

  5. Summary and Conclusions

  6. Resources

1. Overview¶

"Cluster" is one of the most overloaded words in spatial analysis. Part 2's LISA already found HH/LL counties in Dia_pct; Part 1's second-order functions already asked whether crime events attract each other. Part 4 is not a repeat of either — it is the systematic map of every distinct question "is this clustered?" can mean, and the specific, validated tool this library offers for each one.

Step What we do
Four questions State precisely what each of "hot spot", "excess cases", "dense events", and "similar contiguous region" means, and why they are not interchangeable
One table Match each question to its family, its function, and its implementation status in this library
First look Run one representative tool from each family on the same 217-county dataset, side by side
Connections Show exactly how this Part extends, rather than repeats, Parts 1 and 2

Chapters 4.2–4.6 each take one family and develop it fully; this chapter is the map that says which chapter to read for which question.

2. Python Setup¶

In [1]:
import sys, warnings
from pathlib import Path

import numpy as np
import pandas as pd
import geopandas as gpd
import matplotlib.pyplot as plt

warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)


def find_repo_root(start: Path = Path.cwd()) -> Path:
    for p in [start, *start.parents]:
        if (p / "pyproject.toml").exists() and (p / "data").is_dir():
            return p
    raise FileNotFoundError("Run this notebook from inside the PySpatialStats repository.")


ROOT = find_repo_root()
DATA = ROOT / "data"
try:
    import spatialstats
except ImportError:
    sys.path.insert(0, str(ROOT))
    import spatialstats

from spatialstats.weights import Queen
from spatialstats.cluster import getis_ord_gi, spatial_scan, skater

plt.rcParams.update({
    "figure.dpi": 100, "savefig.dpi": 100, "font.size": 10, "axes.titlesize": 11,
    "axes.titleweight": "bold", "axes.spines.top": False, "axes.spines.right": False,
    "axes.grid": True, "grid.alpha": 0.25, "legend.frameon": False,
})
C = {"blue": "#2b6cb0", "red": "#c0392b", "green": "#2f855a", "orange": "#dd6b20",
     "grey": "#718096", "purple": "#6b46c1", "teal": "#2c7a7b"}

counties = gpd.read_file(DATA / "diabetes_northeast.shp")
w = Queen(counties)
print(f"spatialstats : version {spatialstats.__version__}")
print(f"counties     : {counties.shape[0]}")

spatialstats : version 0.2.0
counties     : 217

3. Four Questions, Four Families¶

Question Family Tools in this library Status
Are high values near each other? Where? Statistical hot spots getis_ord_gi, local_moran_clusters (compare esda.G_Local, Moran_Local from Part 2) ✅
Do cases exceed what population implies? Scan statistic spatial_scan (Poisson, circular windows) ✅
Where are events densely packed? Density-based dbscan_clusters, hdbscan_clusters, k_distance (compare Part 1's KDE) ✅
Which areas form similar contiguous regions? Regionalisation skater, agglomerative, evaluate_regions ✅ (max-p regions not implemented)

Each row is a genuinely different statistical object, answering a question the others cannot:

  • A hot spot (Gi) says a value and its neighbours are unusually high *together — it says nothing about whether that excess is more than population size would predict, and nothing about point-level density.
  • The scan statistic explicitly compares observed cases to expected cases given a population at risk — the tool for "is this rate too high", not just "is this value high".
  • Density-based clustering operates on raw event locations (points), not on a value attached to pre-defined areas — it is Part 1's territory (point patterns), applied with a clustering rather than a hypothesis-testing goal.
  • Regionalisation does not test significance at all — it partitions space into new areal units that are internally similar, a construction task rather than a detection task.

4. A First Look at Each Family on the Same Data¶

One representative call from three of the four families, all on Dia_pct (or its supporting variables) for the same 217 counties — full development of each is Chapters 4.2–4.5; this is only a preview of what each family's output looks like.

In [2]:
hotspots = getis_ord_gi(counties["Dia_pct"], w, star=True, inference="analytic", correction="fdr")
print(f"Gi* hot-spot analysis : {hotspots.hot.sum()} hot spots, {hotspots.cold.sum()} cold spots "
      f"(of {len(counties)} counties)")

Gi* hot-spot analysis : 7 hot spots, 3 cold spots (of 217 counties)
In [3]:
coords = np.c_[counties.geometry.centroid.x, counties.geometry.centroid.y]
scan = spatial_scan(coords, counties["Dia_count"], population=counties["POP_Total"],
                    permutations=999, seed=1, n_clusters=1)
top = scan.clusters.iloc[0]
print(f"Scan statistic : most likely cluster of {int(top.n_locations)} counties, "
      f"relative risk {top.rr:.2f}, p={top.p_value:.3f}")

Scan statistic : most likely cluster of 3 counties, relative risk 1.17, p=0.001
In [4]:
regions = skater(counties[["Dia_pct", "Obesity", "SVI"]], w, n_clusters=6)
print(f"SKATER regionalisation : {regions._stats['n_regions']} regions, R²={regions.r2:.3f}, "
      f"sizes={sorted(regions.sizes, reverse=True)}")

SKATER regionalisation : 6 regions, R²=0.272, sizes=[np.int64(139), np.int64(46), np.int64(20), np.int64(6), np.int64(5), np.int64(1)]

Three outputs, three different shapes of answer: a per-county significance label, a single reported cluster with a relative-risk number, and a full repartition of the map into six new zones. None of these three numbers is directly comparable to either of the others — which is exactly the point of treating them as separate families rather than interchangeable synonyms for "cluster".

5. How Part 4 Connects to Parts 1 and 2¶

  • Part 1 (point patterns) asked whether crime events themselves attract or repel each other, using distance-based summary functions (K, L, pair correlation) on raw point locations. Chapter 4.3's density-based clustering revisits exactly the same crime data, but with a constructive goal (find and delineate the dense regions) rather than Part 1's hypothesis-testing goal (is the pattern different from CSR?).
  • Part 2 (spatial autocorrelation) introduced Local Moran's I (LISA) with a careful treatment of two competing permutation schemes and the FDR resolution-limit trap. Chapter 4.2 gives the companion Getis-Ord Gi/Gi statistic the same full, calibrated treatment Part 2 gave LISA — including running Part 2's exact resolution-limit story again, this time for Gi.
  • Part 3 (spatial regression) modelled Dia_pct as a function of covariates with one global relationship. Chapter 4.5's regionalisation offers a related but distinct idea: instead of modelling variation, partition the map into zones that are each internally homogeneous — useful preparation for the small-area stabilisation problem Part 5 addresses next.

6. Summary and Conclusions¶

This chapter mapped the territory Part 4 covers before diving into any one family in depth.

What we did¶

  1. Distinguished four genuinely different senses of "cluster": statistical hot spots, excess-case scan statistics, density-based point clusters, and spatially constrained regionalisation.
  2. Tabulated which function in spatialstats.cluster answers each question, and flagged the one missing piece (max-p regions).
  3. Ran one representative call from three families on the same 217-county dataset, showing their outputs take fundamentally different forms.
  4. Connected each family back to where it extends Parts 1–3.

Practical takeaways¶

  • Before reaching for any tool in this Part, name which of the four questions is actually being asked — "is this clustered" is not specific enough to pick a method.
  • A hot spot, a scan-statistic cluster, and a regionalisation region are not the same kind of object and their outputs should not be conflated in a report.

Next steps¶

Chapter 4.2 (Hot-Spot Detection and Local Clustering) develops the first family in full: Getis-Ord Gi and Gi*, their relationship to Part 2's Local Moran's I, and the same multiple-testing hazards Part 2 uncovered, now for a different statistic.

7. Resources¶

Textbooks¶

Reference Notes
Fischer, M. M. & Getis, A. (Eds.) (2010). Handbook of Applied Spatial Analysis. Springer. Comprehensive coverage of all four cluster families
Kulldorff, M. & Nagarwalla, N. (1995). Spatial disease clusters: detection and inference. Statistics in Medicine, 14, 799–810. Foundational framing of the "is this clustered" question in public health

Key papers¶

Paper Contribution
Getis, A. & Ord, J. K. (1992). The analysis of spatial association by use of distance statistics. Geographical Analysis, 24, 189–206. Origin of the Gi/Gi* family, Chapter 4.2
Duque, J. C., Church, R. L. & Middleton, R. S. (2011). The p-regions problem. Geographical Analysis, 43, 104–126. The max-p regions problem this library does not implement (Chapter 4.5)

In this library¶

Object Role
spatialstats.cluster.getis_ord_gi Statistical hot spots — Chapter 4.2
spatialstats.cluster.spatial_scan Excess-case scan statistic — Chapter 4.4
spatialstats.cluster.dbscan_clusters, hdbscan_clusters Density-based point clustering — Chapter 4.3
spatialstats.cluster.skater, agglomerative Regionalisation — Chapter 4.5