Not much different from Workflowy. Slowly improving day by day. You can self-host it on the Cloudflare architecture! (I need to create better docs for that)
Interesting idea! What are you suggesting to do with relevance? I feel like one MEANS request returns just one best matching result while a user may want to have more. We decided to express it in SQL logic like what you see below but I'm really curious how the MEANS operator could be applied.
-----
SELECT id
FROM index_name
ORDER BY emb <-> $query_vector
LIMIT k;
-----
$query_vector may come from anywhere including being transformed on the fly from 'some text'.
Yeah I’d say sorting by relevance is similar but MEANS is strictly a boolean operator only allowed in WHERE clauses at the moment.
The way it works is:
1. Prune rows based on semantic index
2. Run surviving rows through the LLM which answer yes or no to the given predicate.
3. Collect final result.
Order by relevance is on the roadmap but I haven’t given it much thought.
The use case I’ve been working with is querying meeting transcripts. May I ask what your use cases are and how you’d like order by relevance to work?
A-ha! So you have re-ranking discreetly baked into the pipeline. I'd argue that this could be better for a user to decide but it depends on the use-case really. Speaking of 'order by relevance', the cases that I had in mind could be connected to the re-ranking use case. For example, what if I was looking for a photo of a chihuahua and then wanted to take the best 20 and run them through a re-ranker to come up with the best 5 (or best 1 even).
Are you doing this to unlock natural human-to-model conversations? I was speaking to an AI-demo-assistant yesterday and I think I get whay you are talking about when you say the delay has to be really small.
Yes, you can see an early demo video in the page, of a very early version (I decided not to post the final wip yet).
I’m building an AI gaming companion. Essentially an AI NPC (a speaking dog, think dogmeat of fallout on steroids) that is grounded in the game world, can perceive what’s happening, perform actions, follow complex multi-step instructions, and strategize with the player. I want it to feel natural, so I 'm aiming for real audio conversationality (no push to talk & wait), so audio is important, but also... every millisecond trimmed from audio can be dedicated to heavier AI systems (caching helps heavily there, but extracting player's intent before it even reaches an LLM is probably the biggest trick).
Budgets are pretty tight, ~400ms for everything. I have optimized qwen-3-asr and added streaming detection to it, as well as pocket-tts (this) and qwen-3-tts, with a goal of ~70ms for audio for both detection/creation and then ~350ms for the planning.
It's interesting that gpt-live was just announced today, planning to make more things open source as well as the learnings while building this.
It is an interesting idea. What troubles me is the read/write rate in a real production environment. If it is bound by S3, a buffer will eventually be needed and then it all become very complicated. At least it is how it looks on the surface
You are right, the time of a single write/read is Achilles' heel. But if we are talking about parallel processing it can keeps several hundreds milliseconds delay and provides enormous throughput.
Great project! I was thinking of building something similar with not only search but analytics as well. Could you hint at where the dataset comes from? I'd really like to have a look
Thank you. This has been in my mind for past 1 year. Wanted to do it using vector embedding similarity match, but due to costs and compute requirements, had to resort to keyword based.
I have a dual monitor setup so I usually have CTOP running in a separate terminal in the bottom left corner of my setup. It's a good place to glance to see which of my agent sessions are pending instructions / blocked / running. Also a good birds eye view to see which ones are about to run out of context.
One thing that I've been doing a bit of recently - there's a CTOP CLI which lets one my 'supervisor agents' automatically drive prompts into sessions that need unblocking
Yeah and much more (think all in one), it's a companion app — good for quick temporary drops, but also for keeping important files together and easy to reach throughout. I wrote my thoughts here: https://hachipoo.com/best-mac-file-shelf-dockside