Lost and Found: How Self-Supervised Learning Helps GPS Coordinates Find Their Way

ACML Accepted Paper.

Link: https://proceedings.mlr.press/v222/bougie24a/bougie24a.pdf

 

Summary

To capture the rich underlying semantics of GPS coordinates

• auxiliary tasks including geo **prediction
• high-level reconstruction
• intermediate clustering.

1. each GPS coordinate is projected onto a map image centered on the input coordinate.
2. Then, a student and teacher networks receive two different augmented versions of the same map image.
   • random rotations, zooming, and crops

Proposed Methods

 

 

From GPS Coordinates to Maps

• use an entropy-based cleaning strategy.
  • Low (semantic meaning) Entropy: ocean or mountain
  • Hight (low semantic meaning) Entropy: roads or buildings

Self-Supervised Learning via Student-Teacher Distillation

• DINO
• $P_s(m)^{(i)} = \frac{\exp(g_{\theta_s}(m)^{(i)} / \tau_s)}{\sum_{k=1}^{K} \exp(g_{\theta_s}(m)^{(k)} / \tau_s)}$
• $\min_{\theta_s} \mathcal{H}(P_t(m), P_s(\alpha(m)))$
• Overall process:
  • $\min_{\theta_s} \sum_{m \in \{m_1^g, m_2^g\}} \sum_{m' \in V \wedge m' \neq m} \mathcal{H}(P_t(m), P_s(m'))$

Geo-predictive Tasks

• a land price prediction task
  • $L_{lp} = \frac{1}{N} \sum_{j=1}^{N} (y_j - \hat{y}j)^2 = \frac{1}{N} \sum{j=1}^{N} (y_j - f_p(z^j))^2$
• a population density clustering task
  • $L_c = \frac{1}{N} \sum_{j=1}^{N} \sum_{i=1}^{\mathcal{K}} - p(c_i = k) \log (\hat{p}(c_i = k | f_c(z^j)))$
• a latitude/longitude prediction task
  • $L_l = \frac{1}{N} \sum_{i=1}^{N} \left[ (lat_i - \hat{lat}_i)^2 + (lon_i - \hat{lon}_i)^2 \right]$
• Total loss term:
  • $\arg\min_{\theta_s} L_{geo} = \frac{1}{3} L_{lp} + \frac{1}{3} L_c + \frac{1}{3} L_l$

High-level Reconstruction Task

• To encourage the student network to learn “local-to-global” correspondences
• $L_{rec} = \frac{1}{N} \sum_{j=1}^{N} \| (m_j^* - f_r(z^j, m_j^{**})) \odot \hat{M} \|_2^2$
• $p_i^s = \frac{\exp \left( (f_c^s(y_c^s) - p_{div}) \cdot c_i / \tau_c' \right)}{\sum_{i=1}^{m} \exp \left( (f_c^s(y_c^s) - p_{div}) \cdot c_i / \tau_c' \right)}$
• $p_i^t = \frac{\exp \left( (f_c^t(y_c^t) - p_{div}) \cdot c_i / \tau_c' \right)}{\sum_{i=1}^{m} \exp \left( (f_c^t(y_c^t) - p_{div}) \cdot c_i / \tau_c' \right)}$

Intermediate Clustering Task

• learns a clustering matching between the student and teacher networks.
• $L_{clust}(m, m') = - \sum_{i=1}^{m} p_i^t \log (p_i^s)$

Final Loss Term

• $L_{total} = L_{cl} + \alpha L_{geo} + \beta L_{rec} + \gamma L_{clust}$

Experimental Results