Qdrant is also a good default choice, since it can work in-memory for development, with a hard drive for small deployments and also for "web scale" workloads.
As a principal eng, side-stepping a migration and having a good local dev experience is too good of a deal to pass up.
That being said, turbopuffer looks interesting. I will check it out. Hopefully their local dev experience is good
Qdrant is one of the few vendors I actively steer people away from. Look at the GitHub issues, look at what their CEO says, look at their fake “advancements” that they pay for publicity on…
The number of people I know who’ve had unrecoverable shard failures on Qdrant is too high to take it seriously.
I’m curious about this. Could you please point to some things the CEO has said, or reports of shard failures?
The bit about paying for publicity doesn’t bother me.
Edit: I haven’t found anything egregious that the CEO has said, or anything really sketchy. The shard failure warnings look serious, but the issues look closed
There used to be a benchmarking issue with a founder that was particularly egregious but I can’t find it anymore.
The sharding and consensus issues were from around a year and a half ago, so maybe it’s gotten better.
There are just so many options in the space, I don’t know why you’d go with one of the least correct vendors (whether or not the correctness is deception is a different question that I can’t answer)
For local dev + testing, we recommend just hitting the production turbopuffer service directly, but with a separate test org/API key: https://turbopuffer.com/docs/testing
Works well for the vast majority of our customers (although we get the very occasional complaint about wanting a dev environment that works offline). The dataset sizes for local dev are usually so small that the cost rounds to free.
> although we get the very occasional complaint about wanting a dev environment that works offline
It's only occasional because the people who care about dev environments that work offline are most likely to just skip you and move on.
For actual developer experience, as well as a number of use cases like customers with security and privacy concerns, being able to host locally is essential.
Fair enough if you don't care about those segments of the market, but don't confuse a small number of people asking about it with a small number of people wanting it.
As someone who works for a competitor, they are probably right holding off on that segment for a while. Supporting both cloud and local deployments is somewhere between 20% harder and 300% harder depending on the day.
I'm watching them with excitement. We all learn from each other. There's so much to do.
Yep, we're well aware of the selection bias effects in product feedback. As we grow we're thinking about how to make our product more accessible to small orgs / hobby projects. Introducing a local dev environment may be part of that.
Note that we already have a in-your-own-VPC offering for large orgs with strict security/privacy/regulatory controls.
having a local simulator (DynamoDB, Spanner, others) helps me a lot for offline/local development and CI. when a vendor doesn't off this I have often end up mocking it out (one way or another) and have to wait for integration or e2e tests for feedback that could have been pushed further to the left.
in many CI environments unit tests don't have network access, it's not purely a price consideration.
(not a turbopuffer customer but I have been looking at it)
> in many CI environments unit tests don't have network access, it's not purely a price consideration.
I've never seen a hard block on network access (how do you install packages/pull images?) but I am sympathetic to wanting to enforce that unit tests run quickly by minimizing/eliminating RTT to networked services.
We've considered the possibility of a local simulator before. Let me know if it winds up being a blocker for your use case.
Look into Bazel, a very standard build system used at many large tech companies. It splits fetches from build/test actions and allows blocking network for build/test actions with a single CLI flag. No hassle at all.
The fact that you haven't come across this kind of setup suggests that your hundreds of CI systems are not representative of the industry as a whole.
I agree our sample may not be representative but we try to stay focused on the current and next crop of tpuf customers rather than the software industry as a whole. So far "CI prohibits network access during tests" just hasn't come up as a pain point for any of them, but as I mentioned in another comment [0], we're definitely keeping an open mind about introducing an offline dev experience.
(I am familiar with Bazel, but I'll have to save the war stories for another thread. It's not a build tool we see our particular customers using.)
you pull packages from a trusted package repository, not from the internet. this is not rare in my experience (financial services, security) and will become increasingly common due to software supply chain issues.
So I can note this down on our roadmap, what's the root of your requirement here? Supporting local dev without internet (airplanes, coffee shops, etc.)? Unit test speed? Something else?
In comparison, "Google Docs" is simpler and not rage inducing.
365 is a terrible name. Could you explain to your grandma what a 365 is? Likely not, and I’m not looking forward to that conversation. This is disappointing stuff
> Could you explain to your grandma what a 365 is?
Microsoft Office - Okay, this is a bundle of software relating to office work, released by Microsoft. Makes sense!
Microsoft Office 365 - This is also office software by Microsoft, but the 365 indicates it's a subscription service, because you're paying for it 365 days a year or something. This is contrasted to normal non-365 Office.
Microsoft 365 - So... This is something by Microsoft, that is also a subscription service? Okay...
Microsoft 365 Copilot - Oh! Copilot! Everyone knows Copilot, right? It's like, their one and only remaining important trademark, everyone has got to know what it means with no further explanation. So, it just makes sense, this is a Microsoft subscription service where you pay for this "Copilot", right?
I can recommend tailscale for creating private networks. It has a generous free tier and would reduce the attack surface considerably compared to ngrok
Better yet would be setting up your own wireguard instance and not relying on free lunches. But as far as free lunches go tailscale would be my preferred option
What it provides is a opinionated configuration management - which is admittedly great which is why I use it as well, but it's nonsensical to say tailscale works in places where wireguard is blocked.
You're likely just noticing the preconfigured nat traversal which tailscale provides and never set one up yourself, as you'd need a static IP for that and it's unconfigured by default.
> it's nonsensical to say tailscale works in places where wireguard is blocked
I have two machines on my desk, I configure a wg service on both. I also configure tailscale on both. Everything works.
I move one machine to another network, at a friend's place.
Wg does not work anymore. Tailscale works. So this is very much sensible to say what GP said.
Now, you can have all kinds of explanations about why wg dos not work and ts does, you know STUN, DERP, ts using wg under the hood, and whatnot but the facts are cruel: I cannot wg to my machine, but I can ts.
Right, it’s that specific person’s Wireguard configuration, which is likely a typical one as a result of Wireguard‘s defaults. Tailscale‘s defaults work better, hence the surface-level impression that plain Wireguard does not work in cases in which Tailscale does.
As I said above - how would you set up plain Wireguard in a place without the possibility of exposing a port, or even that does not have a public IP - and initiate the connection from outside that place? I would love to learn something. Without rebuilding tailscale (or whatever other solutions with STUN or whatnot).
i think youre not hearing what - at least i - was saying.
I never said that running the same connectivity and NAT traversal via 2 nodes which are both inside of a NAT is possible.
Neither did I ever claim you dont need a static public IP which _isnt_ behind a NAT / has an open port.
With Tailscale, these are being provided to you by them.
Without them, you would have to maintain that yourself.
This is a significant maintenance burder, which is why I - as in my very first comment you yourself responded to - pointed out that the service theyre providing is great and that i use it myself for that as well.
Nonetheless, _if wireguard was blocked, tailscaile wouldn't work either_
But its not blocked. Hence tailscale works. Just like wireguard would work, if you configured NAT traversal in some way.
To get that working, you have multiple options, one of these being the STUN server.
Another being an active participants in the VPN which facilitates the connection (not just the initiation, which the STUN server would be doing). easier to configure and maintain, but less performant.
Tailscale themselves actually have an incredibly indepth article on how they've implemented it on their end, its a little aged at this point, but I suspect they havent changed much (if any) since
> i think youre not hearing what - at least i - was saying.
You said " it's nonsensical to say tailscale works in places where wireguard is blocked".
If by "blocked" you mean "blocked at the firewall level through some kind of adaptive block that will recognize a wireguard connection based on its behaviour/nature of packets/whatever" → then yes, of course tailscale will not work either as it uses wg under the hood.
If the OP message "tailscale has a much better chance to work when you need it most. WireGuard is blocked by too much stuff" means "I installed wireguard and it does not work (because whatever) but tailscale consistently delivers" → then it is not nonsensical at all. It is the right tool to start with.
> Tailscale themselves actually have an incredibly indepth article on how they've implemented it on their end
This is an excellent documentation to which I refer people as well.
> even if he's proud of being such, as you seem to be
Of course on Internet nobody knows you are a dog. But hey, I may be someone who wrote a part of the Linux kernel in 1994, ran IT operations for a company that was big (big!) and then almost vanished (not my fault :)) and produces open source that you may have even used if you are "technical" as you say.
And set up WG in so many places, including a frontend that unfortunately did not get the worldwide success it should have :)
With this modest introduction - tailscale works where wireguard does not. I am not sure why my example was not obvious. You can reach the machine at my friend's with tailscale, not with plain wireguard.
Of course if you open ports in the right places then yes! And check a few more things.
Now - how would you set up plain Wireguard in a place without the possibility of exposing a port, or even that does not have a public IP - and initiate the connection from outside that place? I would love to learn something. Without rebuilding tailscale (or whatever other solutions with STUN or whatnot).
many times in public/hotel wifis. it's usually places which blanket ban UDP and allow TCP 80 and 443 exclusively. tailscale somehow manages to get a connection.
Venture Capital subsidized gig economy apps until the dinosaurs of the previous economy (taxis, delivery services if any existed?) went extinct. So what now? Wait for regulation to catch up in an opaque at-will-employment industry with no unions?
They do offer more utility and convenience than the previous generation of services, which is why they’ve gained market share. Increasing accountability on the current big players is better
It kind of is, though. You use some input material to produce the weights via some process, even if the weights might not become exactly the same every time you reproduce the process; the production of the weights isn't done by working with the weights, but with the training material and the process to convert them into weights. The analogy to source code and the resulting binaries is there.
Training data and the weights produced are not source code, just as access to the resulting binaries are not a requirement for open source.
Open source does not require full working implementations. There's no requirement that a code snippet that I release be fully working and identical to a complete solution.
So we are on agreement that "weights" are not source code. Training data might not also be actual "code", but it is source. After all, the model trained using that data tries to estimate its training data. It is the ground truth for the model.
About the access of binaries or providing working implementations, where did those come from? I don't think this thread was discussing those at all.
Indeed I would be willing to call something an "open source model" if it came without weights, but did come with the training data and with a documented process (preferably executable); and a release with just the training data could be called "open dataset" while the software to run the training would be just plain old open source software.
And, of course, a model with only the model data distributed with an open license is relatively commonly called "open weights", this being pretty self-explanatory term.
I mean no, you don't need to be open source at all. Just don't release the data and call the release "open weights". Or do release the data, and the training process, and call yourself "open source".
Though, I do think it's still acceptable if you just point how to get the data (i.e. if it was the offline version of Wikipedia and then URL to that) if actually providing the source data is overwhelming. Offering to provide a copy at cost would be quite acceptable (i.e. I deliver the media to you to make a copy).
But if there's no way another person can acquire that data, even in theory, then I think it's pretty clear the source was not open. Just use the more appropriate term and everyone is on the level what the release is about.
Query expansion and re ranking can and often do coexist
Roughly, first there is the query analysis/manipulation phase where you might have NER, spell check, query expansion/relaxation etc
Then there is the selection phase, where you retrieve all items that are relevant. Sometimes people will bring in results from both text and vector based indices. Perhaps and additional layer to group results
Then finally you have the reranking layer using a cross encoder model which might even have some personalisation in the mix
Also, with vector search you might not need query expansion necessarily since semantic similarity does loose association. But every domain is unique and there’s only one way to find out
Models like bge are small and quantized versions will fit in browser or on a tiny machine. Not sure why everyone reaches for an API as their first choice
As a principal eng, side-stepping a migration and having a good local dev experience is too good of a deal to pass up.
That being said, turbopuffer looks interesting. I will check it out. Hopefully their local dev experience is good