# v0.47.0 — Complete Embedding Programme: Sparse Vectors, Hybrid Search & Embedding API

> **Full technical details:** [v0.47.0.md-full.md](v0.47.0.md-full.md)

**Status: Planned** | **Scope: Large**

> Sparse and half-precision vector aggregates, reactive neighbour-alert
> subscriptions, hybrid-search corpus patterns, ergonomic
> `embedding_stream_table()` API, per-tenant ANN indexing, outbox-emitted
> embedding events, and a vector benchmark suite. Completes the AI/RAG feature
> set built on the hardening arc.

---

## What is this?

v0.46.0 completes embedding pipeline infrastructure and ANN maintenance.
v0.47.0 delivers the remaining production AI/RAG features in a single release:
storage-tiered sparse and half-precision vector aggregates, reactive
distance-predicate subscriptions, hybrid-search corpus patterns, and an
ergonomic `embedding_stream_table()` API that sets up common RAG corpora in one
call. This release also covers multi-tenant ANN patterns, outbox-emitted
embedding events, and a comprehensive vector benchmark suite.

---

## Sparse and half-precision vector aggregates

pgvector 0.7+ ships `halfvec` and `sparsevec` types for storage-efficient
embeddings. v0.47.0 adds:

- `sparsevec_avg` / `sparsevec_sum` — element-wise mean/sum over the union of
  sparse dimensions, treating absent entries as 0.
- `halfvec_avg` / `halfvec_sum` — running state upcast to `vector(d)` for
  numerical stability; returns `halfvec(d)` on read.

A stream table can maintain tiered storage: raw `vector(1536)` →
`halfvec(1536)` → `sparsevec(1536)` (for binary-quantised re-ranking), each
layer incrementally maintained and independently indexed.

---

## Reactive vector-similarity subscriptions

Extending reactive subscriptions (v0.35), v0.47.0 adds support for reactive
queries over distance predicates. A subscription fires whenever a *new* row
enters or exits the similarity neighbourhood, using differential semantics
(only predicate-truth changes fire — no spurious re-emits on every refresh).

---

## Hybrid-search corpus pattern

Combine vector similarity with BM25 full-text search and metadata filters on a
single flat stream table. v0.47.0 publishes
`docs/tutorials/HYBRID_SEARCH_PATTERNS.md` with flat denormalised corpora,
RLS-scoped corpora, tiered search patterns, and performance-tuning notes.

---

## `embedding_stream_table()` ergonomic API

Today, users still write the full denormalization query by hand. v0.47.0 adds a
higher-level helper that auto-generates the query, creates the stream table,
provisions indexes, configures post-refresh actions, and returns a dry-run
preview when `dry_run => true`. The API covers common single-source-plus-joins
RAG patterns without removing SQL control from power users.

---

## Also in v0.47.0

- Per-tenant ANN indexing patterns: RLS-scoped embedding corpora with security
  review and explicit trust boundaries
- Outbox-emitted embedding events: downstream agents or search services can
  react to embedding churn without polling
- Vector benchmark suite measuring differential refresh throughput, aggregate
  reducer cost, drift-detection overhead, and storage savings across `vector`,
  `halfvec`, and `sparsevec`
- Materialised k-NN graph research spike (non-blocking): trade-off analysis of
  pre-computed neighbour relationships for fixed-pivot retrieval
- Public starter repository and best-effort ecosystem examples for the
  post-hardening AI stack (never a release gate)

---

## Scope

v0.47.0 is a large release that consolidates the entire embedding feature set.
The k-NN graph work is explicitly research — it does not block the release. The
`embedding_stream_table()` API hides real query-shape and indexing complexity,
so `dry_run` support and index-inference documentation are required before ship.

---

*Previous: [v0.46.0 — Embedding Pipeline Infrastructure & ANN Maintenance](v0.46.0.md)*
*Next: [v1.0.0 — Stable Release](v1.0.0.md-full.md)*
