Even if they had, doesn't mean it's a good idea to flood the product with changes, nor it's necessary that the market has the capacity to asborb and expand your sales, or that the existing customer base will appreciate it.
Well I did say "good", which excludes things that don't fit in the context of the wider business.
I think startups have an easier time for 2 reasons:
They are innovating on the product side already, and probably were already capacity constrained for work on the top ideas.
They also have a high proportion of tech debt which never bubbled to the top of effort vs reward (competing against product work) before but would still be valuable to fix. E.g. https://posthog.com/blog/sql-parser was on the HN frontpage a few months ago, and this work would never have happened in the before times
Even if the idea fits the context of the wider business, there's such a thing as too much change "just because you can", which alienates users and can make the end product an unfocused mess.
If it's easier for startups, it's for the same reason doing a "pivot" is easier: they usually don't have much of a product or revenue or customer base to begin with, they throw things out there and see what sticks.
It doesn't seem so easy, because if you can easily dismiss a collection of digital neurons then why is it so hard to dismiss a collection of biological neurons as being conscious?
The best way I have found to think about this is that consciousness is a property of a collection, like temperature. One atom does not have a temperature but if you have enough of them together then it's a useful enough property to talk about. Similarly, one human neuron does not have consciousness but if you put enough of them together then they do.
While I'm here - I recently optimized our PR review skill to make better use of the KV cache.
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.
The layout, as well as all of the paragraphs are written by me manually, although I generally draft a completely broken English one very rapidly and ask it to paraphrase, I do apologize if the editorial pass made it harder to read, but it really isn't an LLM's opinion :)
I thought KV cache not being as important was obvious, as a big part of the benchmrks were focused on pricing, but in retrospect I can see how I could've made that more clear.
In the end cost is what matters most and as seen from the results the KV cache miss on the cheap model doesn't really matter as much.
What is there to talk about the KV Cache, they’re handing off to a different model, I thought that you can’t reuse KV cache between entirely different models?
Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides".
I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it?
Edit: this specific title has been deleted from the article. That was not my point! Please put in more effort into writing things that you want others to read! Rather than putting in low effort but being better at hiding it.
Yeah this is an extremely poorly written article. They didn’t even bother to add a “rewrite this article to make it sound less than AI”.
It’s also using a bazillion words to make a point that could be summed up in a single paragraph: there’s a huge variance in the number of tokens required to encode the same content, with code leading the charts.
To be fair, most of this was already known, and Anthropic communicated very clearly about the different tokenizer they started using.
Their compute is also mostly 1:1 correlated to the number of tokens, so I don’t believe in the conspiracy that this is just to inflate prices.
Yes it's worth commenting about. Not everyone, but many want to know that signal just like you do. It not only provides a useful heuristic about the article, but about whatever product or service they're advertising/selling.
A problem is AI by default is not very good at anything. It’s pretty mediocre. With a good harness and a lot of prompting/context - you can get it to spit something out that’s pretty good. Coders have been learning and fighting this fight for a couple of years now.
The issue is that it’s not just code - they suck at writing. Really bad. Unreadable, incoherent, messy.
Humans are also bad at judging the quality of things they themselves aren’t very good at. So a senior swe sees what claude spits out and says “This is trash.” And spends x amount of time getting it to not be trash. And Jr dev thinks “this is magic!” And pushes it to a PR.
So my theory is the people “writing” this AI slop think its great! But actually just aren’t very good at writing copy and don’t have the skill to recognize it and prompt their way out of it.
Or they don’t care. That’s an option as well.
PS for anyone reading, next time AI does something that you aren’t super familiar with that looks pretty good… maybe find an expert to review it.
Well, criticizing is, of course, great. But the reality is that English is not my native language and I dictated most of it with my voice, then processed it with the help of AI, translated, added, corrected, and converted.
It is actually a big result of work, a lot of research and attempts. And to just say that "oh, this is AI-slop," I consider unfair, but that is your choice.
There is a difference:
- There are people who do,
- And there are those who criticize.
Instead of getting offended by a fair criticism you should learn from it. In your articles consider adding a disclaimer that says exactly what you just said here in your comment here that you post-processed your voice and thoughts through LLM.
LLM speak is like the new corporate speak. Enterprise writing is fulll of fluff and nothings and they all read the same. That sameness is what most readers here are sick of.
(Your comment here that I replied to is also written by AI which is even more sad :| )
Bro, this was the only reason I opened the comments, and I'm so happy to see someone else noticed. Even if they removed that, the utter sloppiness of the prose is unbelievable. Offensive, even.
Thank you for the criticism. I heard you. I added a TLDR. I cleaned up many AI constructions. By the way, I tweaked it a bit, compressed it.
One way or another, I want to note that yes, this text was made in collaboration with AI. My English is non-native. It helps me translate, helps me structure better. Yes, there is a downside, it can bloat the text with unnecessary words. But that, unfortunately, is the price.
But the key thing is that I tried very hard to share my many years of experience, or rather a part of it, which I acquired, with all of you. And I am very glad that this information turned out to be useful to you.
The key here is:
* The information that is written in the article.
* Not how it is written, but what I was trying to convey to you.
A family member stayed in a hotel in Dubai recently and on returning said how incredible the staff were and willing they were to help them, and my response was "no shit"
I went once. It feels like Disney Land but for the wealthy. I'll never go back, and I actually regret going to be honest. It's disgusting and it's right out in the open.
Has it? This is from that repo, "PostHog FOSS is a read-only mirror of PostHog, with all proprietary code removed." Seems they have some proprietary code in the non-open-source version.
Nothing has changed, just OP found the mirror of the main repo with the ee/ folder removed. That repo has been up for most of the lifetime of the company.
That's valid criticism, I kinda hand-waved the "months" part. I read everything I could about parsers while building this (I have a CS background but hadn't thought about parsers in a long time) and came across this blog post https://lakesail.com/blog/sql-parser-in-one-week/ which talked about building a toy parser in a week, so I scaled that up to months for a production one.
That requires having enough good new ideas for everyone to work on, and sadly a lot of companies don't have those