BEVFormer

BEV Queries

$$Q∈R^{H×W×C}$$

where H, W are the spatial shape of the BEV plane.

Each grid cell in the BEV plane corresponds to a real-world size of s meters.

The center of BEV features corresponds to the position of the ego car by default.

Spatial Cross-Attention

$$\mathrm{S C A} ( Q_{p}, F_{t} )=\frac{1} {| \mathcal{V}_{\mathrm{h i t}} |} \sum_{i \in\mathcal{V}_{\mathrm{h i t}}} \sum_{j=1}^{N_{\mathrm{h i t}}} \mathrm{D e f o r m A t t n} ( Q_{p}, \mathcal{P} ( p, i, j ), F_{t}^{i} ),$$ $$x^{\prime} \!=\! ( x \!-\! {\frac{W} {2}} ) \! \times\! s ; \quad y^{\prime} \!=\! ( y \!-\! {\frac{H} {2}} ) \! \times\! s,$$ $$\mathcal{P} ( p, i, j )=( x_{i j}, y_{i j} )$$ $$z_{i j} \cdot\left[ \begin{matrix} x_{i j} & y_{i j} & 1 \\ \end{matrix} \right]^{T}=T_{i} \cdot\left[ \begin{matrix} x^{\prime} & y^{\prime} & z_{j}^{\prime} & 1 \\ \end{matrix} \right]^{T}.$$

Temporal Self-Attention

$$\mathrm{T S A} ( Q_{p}, \{Q, B_{t-1}^{\prime} \} )=\sum_{V \in\{Q, B_{t-1}^{\prime} \}} \mathrm{D e f o r m A t tn} ( Q_{p}, p, V ),$$