← All case studies Pinecone → Qdrant

Revelry copied thousands of Pinecone vectors into Qdrant in about three minutes

Revelry's Elixir app already had a module per vector database. On 20 March 2024 they copied thousands of Pinecone vectors to Qdrant in batches of 100, and they still support both.

On 20 March 2024 the Revelry team wrote up a Pinecone to Qdrant copy. The post is theirs. It names no author beyond the team, and it gives no dollar figure.

They use a vector database for retrieval-augmented generation. Text becomes vectors. Similarity search pulls the chunks that belong in the prompt, so the model answers from their data instead of only from training.

They picked Pinecone first because the team was small and they did not want to run that subsystem. Pinecone is closed source and hosted. They sent vectors over HTTP and read them back for search.

Qdrant looked better once the first version worked. It is open source, it can run on premises, and it also has a managed cloud. They wanted the speed and the storage savings. LLM calls are already slow. Any time saved before the prompt is sent shows up in the product.

The problem was the data already stored. Thousands of vectors sat in Pinecone. New vectors could appear during a long export. They needed a copy that finished fast, and they needed the app to talk to both databases during the switch.

A module per provider

The backend is Elixir. Each vector database has its own module. A migration task copies from one provider to another. Adding a third provider means a new module and a copy from a database that is already current.

The Pinecone to Qdrant run took about three minutes. Most of that was HTTP. Reshaping a Pinecone vector into a Qdrant vector in Elixir was nearly instant. They sent batches of 100. That cut the number of requests without pushing a batch so large that it timed out.

They say three minutes was short enough that users did not see a real delay. They still support Pinecone and Qdrant. The move was not a cutover that deleted the old index the same day.

The post does not list failed batches, a vector count more precise than “thousands”, or a before-and-after latency table.

What I would copy

Put the provider behind a module before the index is large. Revelry’s copy is a task between two modules. A rewrite that knows Pinecone’s client in every call site would have taken longer than three minutes.

Batch the HTTP. They used 100. If a batch times out, lower it. The slow part was the network, not the reshape.

Keep the old provider readable until the new one has the same vectors and a query you trust. The Pinecone → Qdrant playbook warns that a finished migration can hide missing vectors, and that both systems may still be live. Revelry left both live on purpose. That is safe only if you know which one a request is reading.