If I wanted to build a rag around a generative transformer, how does the retrieved vector plug into the transformer? Is it cross attention from an encoder or a prefix on the decoder, or sonething else? Does it plug in directly as the retrieved vector or is it somehow re-embedded?

Okay! I have a question, more like an observation! (Sorry I am maybe not your target audience, but to get things started)

Many buzzwords used and after reading your project page, I requested the help of ChatGPT to make any sense out of it!

Is what Chatgpt tells me about your interesting project right?

This is what it has to say:

Chroma's serverless system is designed to help developers build AI applications without managing servers. It allows you to efficiently store and retrieve data using vectors (for AI models) and text queries, all without needing to handle infrastructure. The system is scalable, cost-effective, and optimized for AI workloads, so developers can focus on their projects rather than backend management.

Thanks ChatGPT!

Yes - that's a great explanation.

It seems like a really cool idea.

Is this (or will it be) part of the oss version of chroma? And would it work with the embedded version of chroma? From the article, it looks like it might require more server-side components now vs chromadb today

it’s fully open source, apache 2.0 and in the mono repo today.

a distributed database is naturally has more complexity, but we’ve put a lot of effort in to make it as easy as possible to run.

what do you think about ColBERT? doing embedding on every token feels wrong in ways i can't articulate - is there a time to use it and not use it, or is it the one retrieval technique to rule them all?

Colbert is great! - Check out https://github.com/AnswerDotAI/RAGatouille by the excellent https://x.com/bclavie

Relatedly ColPali (https://arxiv.org/abs/2407.01449) is gaining a ton of steam in the IR community.

In general we are skeptical of silver bullets in retrieval.