Idea/Soft Hashes for Lazy Vector Indexing

• Soft Hashes for Lazy Vector Indexing

  • Let agents decide at retrieval time whether a corpus needs vector search, instead of precomputing embeddings for every possible use. Lazy indexing avoids spending compute and disk on retrieval patterns no agent may need, and avoids baking those assumptions into the corpus in advance.
  • Give each document or chunk a stable address, in the spirit of IPFS, and compute an inexpensive soft hash when the chunk is first indexed. The soft hash would act as a fast proxy for whether the stored vector still represents the current text.
  • On a later vector retrieval, compare the current chunk's soft hash with the one associated with its vector. If the change is significant, return the available result with a freshness signal and queue a re-embedding asynchronously; if it is still a close enough proxy, reuse the vector.
  • A shared store for a team's corpus could let machines reuse existing chunk identities, freshness metadata, and vectors instead of independently embedding the same content. Agents could request vector indexing only when useful, while later users benefit from the work already done.
  • The hard part is defining the soft hash and its threshold: it must be cheap to compute, sensitive to meaning-changing edits, and tolerant of changes that do not make the old embedding misleading. It may relate to the embedding itself, but should be cheaper than recomputing the embedding.
  • Open questions: How should chunk identity survive edits or changed chunk boundaries? How should freshness be represented across embedding models and versions? Can the store coordinate deduplication and asynchronous refresh across machines without making a central service mandatory?
  • This connects to AI/Embedding and the question of when AI/Agents should choose indexed retrieval over direct search.