-> I think teaching a child to trust an LLM from a formative age is horrifically irresponsible.
I largely agree, although with some nuance. One of the most important skills a young child today is going to need as they grow up is the ability to understand what's AI and what isn't, and how to interact with things that are. We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.
If part of the comment was on the accuracy of LLMs; our system of multiple harnessed LLMs running in parallel does deliver pedagogically solid and accurate content and experiences. There's a few comments in this thread about LLM hallucinations - We have curriculum specialists constantly evaluating the interactions that Ello delivers, and it just isn't a major factor right now.
So we need to ensure that we're delivering a high-quality, accurate experience for a child, while at the same time making sure that they understand what's reality.
-> Actually, better, don't put an LLM in front of children. At all.
I've responded above to your reason against using AI in education. But to judge whether it's right to avoid using AI with children entirely, we also need to weigh the positive side. So many children get a substandard education that isn't responsive to them and doesn't set them up for success - whether that's a child stuck in a class of 30 being treated as average, or one of the 270 million kids who don't have access to any education at all. Saying "don't put an LLM in front of children" misses the reality of what education actually looks like for so many of them IMO. If your kid gets everything they need from their school, their home, their private tutor and so on, then great.
So our answer is to use the benefits of AI to build technology that teaches - that understands where a child is and gives them the next learning experience they need to keep growing - while making sure they understand they're dealing with a computer program, and without trying to replace the critical role of the people in their lives.
> We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.
The problem is not talking to a computer program. Kids have been using computers in school long before LLMs.
The problem is teaching a child to trust what the non-deterministic system is guessing at them. That will carry forward in their life.
You’re teaching them that an LLM is right, not just what it is.
As an aside, I’m also not sure about the certainty of what “making sure” a 4-year old knows is real or not is.
> If your kid gets everything they need from their school, their home, their private tutor and so on, then great.
That’s a powerful emotional point made to shut down criticism. Except it has nothing to do with putting a kid in front of LLMs.
I expressly mentioned this in my comment. You could create a mountain of educational material, apps, and tools with LLM assistance and verify it. Same result.
-> You could create a mountain of educational material, apps, and tools with LLM assistance and verify it. Same result.
I very strongly disagree with this. Teaching is not the same as a bunch of educational material and apps. The use of AI in education does not replace teachers, but it can get much closer to the responsive, learner centered, high-touch learning experiences that teachers provide, and that are proven to deliver the best outcomes.
-> That’s a powerful emotional point made to shut down criticism.
You made a blanket statement - without any substantive points - that we shouldn't put LLMs in front of kids. I was arguing that there are advantages that we should take into account, not trying to shut down criticism.
-> not sure about the certainty of what “making sure” a 4-year old knows is real or not is.
I agree. We're being very cautious here. My co-founder, who is a clinical child psychologist, has a blog post on exactly this you can find it in the footer of our website.)
We’re going to disagree about the quality of LLM learning experiences.
As an educator I’ve used them a lot for myself and in teaching others. Good for one-off topics if you check facts and a lot of human educators have built good quality content that it rips off.
LLMs are rather terrible at an extended progressive curriculum.
That said, the core thesis was simple: it’s dangerous to teach children to trust LLMs.
Not what they are. That they’re right. Which they aren’t. A lot.
You said you largely agreed with that. So, what’s the approach within this service to mitigate that?
I appreciate you responding but you’re throwing a lot of “but X” when the question is “what about Y”.
On the first Q: that popup is for the website. We're a consumer company and use paid marketing to reach parents. There are no advertisements in the product, nor is any data in the product sold or used for any purpose apart from delivering the experience.
On the second, we absolutely do not just tell an LLM to be safe and hope for the best. Every output of the LLM that is going to be presented to a child runs through a three-layer safety system. But - as importantly - everything a child says to the LLM goes through that same system.
1. Programmatic, anything that could easily be flagged with hardcoded rules.
2. An LLM evaluator that can interpret meaning to catch anything that the programmatic rules miss.
3. A human in the loop that reviews everything that could be slightly concerning.
We then, in turn, have a multi-tiered response system, including giving parents full access to everything that's happening so they're informed, flagging in a clear, but non-alarmist way, anything somewhat concerning, and finally direct outreach to the parents and/or authorities for anything critical.
Thanks for signing up but even more, thanks for the wonderful feedback.
- We actually find the opposite. Without the promo code people are reluctant to try it. I think because it's new and - to you next point - not cheap.
- This is a real concern for us and one of our primary product develop tracks it to change this. Our mission contains the phrase 'regardless of resources' and as a $25 DTC product we are not doing that yet. Hopefully much more to come on this soon.
- Interesting. We kinda have to know the grade level to send the right books. About 10% of parents know their child's reading level but 90% the only info we get is grade level and ahead/ behind etc... I haven't heard that as a privacy concern. We could do more to explain why we want this.
- We can explain how to add siblings better up front. It is there, but it's small and easy to miss.
- Agree. Right now as we are getting started we have made it binary - our main goal was that Ello would work for those who really wanted no data sharing, and to give parents that option very easily and clearly. As we start to build out more product features that rely on processing audio we should make privacy modular rather than binary.
- Again agree. We're working on that. We definitely need to get parents/kids to the 'aha' moment sooner and not have a 5 day shipping delay. What do you mean by it feeling like a growth hack? It feels like the opposite to me - a growth killer!
Would love any more thoughts as you get the box and they start using it. Thanks you!
> Without the promo code people are reluctant to try it.
Yeah, I mean I used the promo code, was just complementing that you hit on something I was jumping into regardless. Feels like most subscription kids apps are $9 a month max (I lost track of all we pay for), but I didn't blink at you being way above.
> We kinda have to know the grade level to send the right books... I haven't heard that as a privacy concern.
I might have slightly misphrased this. Basically some parents would be super involved with an app like this (we tend to be) and want to self-serve a little bit more. As a more extreme example - and not your beachhead customer for sure - there are plenty of adults who read at a low level and would feel more comfortable with there being an "other" option in there.
But I get the opposite side too. It is a lot simpler to have it tied to their school grade. I'm just sort of thinking out loud about removing barriers because I think something like this might be super useful for kids
> What do you mean by it feeling like a growth hack?
Might be a silly point of mine. But I mean by adding anticipation/delays into the mix, even though they are enforced by real-world shipping factors. There are plenty of public domain books - many of which families already own (or can print pdfs of) - that can be there to seed the collection. So the blank library on day 1 is silly.
Also, maybe I am projecting my own anticipation of the box arriving.
Another note, it seems that the requested features (privacy levels and interests) are not available for edit after registering.
"Sold a Story" looks great, thanks for that. I will give the episodes a listen.
Regarding Ello, how does it adapt to a child's reading level? Say my child is in 2nd grade, but is reading at a 1st grade level. Will Ello learn that over time and choose/recommend books that are more appropriate for their level? Appreciate any color you can provide on how that works. Thanks.
Right now we work closely with parents (checking in with them each month before the books ship out) to help guide them in making the decision around book choice. We also have a form parents can send to the teachers. In the future, we will have a tighter feedback loop to support this decision making that comes directly from the way the child interacts with the app (if we have the permission to analyze that data from the parent).
We are actually adding another language soon! (for somewhat random reasons) - we hope eventually to expand much more.
And it really depends on how strict you want it - 'strict' is a bit subjective. What we cannot do yet, but want to, is put those controls in the hands of the caregiver.
I'd definitely subscribe if it can teach my kids the proper native accent (preferably posh British) because that's something I can't and it would really complement my own knowledge and that will really help my kids get a job in the future so it makes financial sense to subscribe.
This would also widen the target audience, my kid is one year and kids that age are already starting to categorise sounds.
Becoming a "poor reader" is something I think most parents on Hacker News aren't going to worry about. So my tip is that you might want to change the marketing material a bit. Also I'd like to have the current age range more front and center.
If you try it, I'd love your feedback. If youve been thinking about building this yourself I'm sure you'll have some great thoughts on what we've done well and what we could do better with.
- Five is definitely insufficient for some children (and too much for kids who really struggle and want to perfect books). We're working on giving parents the ability to get more; either we ship more, or once you have read the physical books you can unlock digital books. Lots of considerations there of course. We also want you to be able to use books from e.g. the school library. Q for you - for the 1 to 2 readers a day your child reads, where do those books come from?
- That's a great one. I haven't had that question before but I could certainly see the challenge there. Let me ask Elizabeth and Lauren (the experts) what they think on that.
- It goes one of two ways; both which we're happy with. The first way is the child builds their confidence and doesn't need Ello anymore. Not the best for business but a great outcome :) The other is that the child continues to value the 'safe space' to practice reading, and does it in conjunction with reading elsewhere.
- Research pretty strongly shows that the most important skill - to begin with - is decoding (learning the sounds words makes.). Comprehension comes after that. What has been fairly strongly disproven is that context in reading is sufficient (and some would even say helpful) in learning the fundamentals of reading. Having said that, comprehension is a crucial latter part of reading - we're excited for some of the potential for ChatGPT here in allowing us to go beyond basic questions.
Great questions for us to think about as to what would be in the back of parents minds. And Ill get back to you on the second!
Thanks for the detailed reply! Great answers and fills me with hope that perhaps Ello will work for us when our second or third starts reading.
We get the readers either from school or the library. Generally there's 1 reader a day from school, and maybe a little support book too. Then we just hire some from the library each week and use them to fill in the gaps, especially on the weekend. We have a wealth of other books, and hire picture books etc. for 'shared reading' times when we read to the kids after dinner etc.
"Comprehension comes after that" - makes total sense. Can see Ello being great at the decoding step, and yeah, some potential in ChatGPT and other new tech to assist more with comprehension.
-> I think teaching a child to trust an LLM from a formative age is horrifically irresponsible.
I largely agree, although with some nuance. One of the most important skills a young child today is going to need as they grow up is the ability to understand what's AI and what isn't, and how to interact with things that are. We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.
If part of the comment was on the accuracy of LLMs; our system of multiple harnessed LLMs running in parallel does deliver pedagogically solid and accurate content and experiences. There's a few comments in this thread about LLM hallucinations - We have curriculum specialists constantly evaluating the interactions that Ello delivers, and it just isn't a major factor right now.
So we need to ensure that we're delivering a high-quality, accurate experience for a child, while at the same time making sure that they understand what's reality.
-> Actually, better, don't put an LLM in front of children. At all.
I've responded above to your reason against using AI in education. But to judge whether it's right to avoid using AI with children entirely, we also need to weigh the positive side. So many children get a substandard education that isn't responsive to them and doesn't set them up for success - whether that's a child stuck in a class of 30 being treated as average, or one of the 270 million kids who don't have access to any education at all. Saying "don't put an LLM in front of children" misses the reality of what education actually looks like for so many of them IMO. If your kid gets everything they need from their school, their home, their private tutor and so on, then great.
So our answer is to use the benefits of AI to build technology that teaches - that understands where a child is and gives them the next learning experience they need to keep growing - while making sure they understand they're dealing with a computer program, and without trying to replace the critical role of the people in their lives.