[C API] Expose leanvec ood

[ethanglaser]

Summary

Exposes LeanVec out-of-distribution (OOD) training to the C API. Callers can train dimensionality-reduction matrices from a data sample plus a sample of queries, then build an index that uses those matrices instead of computing PCA matrices at build time.

API

// Train matrices. Pass num_queries=0 / x_q=NULL for in-distribution (PCA).
svs_leanvec_training_data_h svs_leanvec_training_data_build(
    size_t dim, size_t num_vectors, const float* x,
    size_t num_queries, const float* x_q,
    size_t leanvec_dims, svs_error_h out_err);

void svs_leanvec_training_data_free(svs_leanvec_training_data_h);

// LeanVec storage using pre-trained matrices; leanvec_dims comes from the
// training data. Storage holds its own reference, so the handle may be freed
// as soon as this returns.
svs_storage_h svs_storage_create_leanvec_trained(
    svs_leanvec_training_data_h training_data,
    svs_data_type_t primary, svs_data_type_t secondary,
    svs_error_h out_err);

Training data is attached at storage construction rather than via an index-builder setter (per review): leanvec_dims has a single source of truth, storage configs stay immutable and safely shareable across builders, and misuse on a non-LeanVec
storage is a compile-time error instead of a runtime one.

Tests

c_api_index.cpp: OOD build + search, in-distribution pre-trained build + search (freeing the training data before build, to exercise shared ownership of the matrices), and NULL-training-data rejection. All sections no-op gracefully on builds or hardware without LeanVec support.

Notes

• Renamed internal StorageLeanVec::lenavec_dims → leanvec_dims.
• LeanVec paths are compile- and ABI-verified locally but not runtime-verified: the C bindings' pinned SVS_URL predates MemoryBreakdown (the C++ bindings were bumped in Update SVS_URL to nightly with get_memory_breakdown #357, the C ones weren't), and the only nightly that has it is LTO-built and won't link against local GCC 11.4. Relying on CI with private sources here.

[rfsaliev]

API concerns

[rfsaliev]

It seems like the "training data" belongs to LeanVec storage kind only.
The proper API call should be connected to LeanVec storage only:
Preffered:

SVS_API svs_storage_h svs_storage_create_leanvec_pre_trained(
    svs_leanvec_training_data_h training_data, // leanvec_dims to be extracted from there
    svs_data_type_t primary,
    svs_data_type_t secondary,
    svs_error_h out_err /*=NULL*/
);

alternative:

SVS_API bool svs_storage_set_leanvec_training_data(
    svs_storage_h leanvec_storage,
    svs_leanvec_training_data_h training_data, // leanvec_dims to be extracted from there and compared with previously defined
    svs_error_h out_err /*=NULL*/
);

[ethanglaser]

Updated, but named it svs_storage_create_leanvec_trained

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request