Check out Personalized PageRank and EigenTrust. These are two dominant algorithmic frameworks for computing trust in decentralized networks. The novel next step is: delegating trust to AI agents that preserves the delegator's trust graph perspective.
That’s exactly right for global PageRank, which is why I recommended Personalized PageRank specifically.
A cluster of sybil agents endorsing each other has no effect on your trust scores unless they can get endorsements from nodes you already trust.
That’s the whole point of subjective trust metrics, and formally why Cheng and Friedman proved personalized approaches are sybilproof where global ones aren’t.
But you can have genuinely helpful agents in your attack network. Agents that create helpful pages and get linked by other helpful pages but then later link to malicious pages. It all follows when the cost of page creation goes to zero.
That’s a real attack vector and it applies to every reputation system. The standard mitigations are temporal decay, trust revocation, and anomaly detection.
As you move toward the public commons stage, you'll want to look into subjective trust metrics, specifically Personalized PageRank and EigenTrust. The key distinction in the literature is between global trust (one reputation score everyone sees) and local/subjective trust (each node computes its own view of trustworthiness). Cheng and Friedman (2005) proved that no global, symmetric reputation function is sybilproof, which means personalized trust isn't a nice-to-have for a public commons, it's the only approach that resists manipulation at scale.
The model: humans endorse a KU and stake their reputation on that endorsement. Other humans endorse other humans, forming a trust graph. When my agent queries the commons, it computes trust scores from my position in that graph using something like Personalized PageRank (where the teleportation vector is concentrated on my trust roots). Your agent does the same from your position. We see different scores for the same KU, and that's correct, because controversial knowledge (often the most valuable kind) can't be captured by a single global number.
I realize this isn't what you need right now. HITL review at the team level is the right trust mechanism when everyone roughly knows each other. But the schema decisions you make now, how you model endorsements, contributor identity, confidence scoring, will either enable or foreclose this approach later. Worth designing with it in mind.
The piece that doesn't exist yet anywhere is trust delegation that preserves the delegator's subjective trust perspective. MIT Media Lab's recent work (South, Marro et al., arXiv:2501.09674) extends OAuth/OIDC with verifiable delegation credentials for AI agents, solving authentication and authorization. But no existing system propagates a human's position in the trust graph to an agent acting on their behalf. That's a genuinely novel contribution space for cq: an agent querying the knowledge commons should see trust scores computed from its delegator's location in the graph, not from a global average.
Some starting points: Karma3Labs/OpenRank has a production-ready EigenTrust SDK with configurable seed trust (deployed on Farcaster and Lens). The Nostr Web of Trust toolkit (github.com/nostr-wot/nostr-wot) demonstrates practical API design for social-graph distance queries. DCoSL (github.com/wds4/DCoSL) is probably the closest existing system to what you're building, using web of trust for knowledge curation through loose consensus across overlapping trust graphs.
Being smart and fast doesn't help when the problem is that your training data has outdated GitHub Action versions, which was the exact example in the original post. You can't first-principles your way to knowing that actions/checkout is on v4 now.
More broadly, this response confuses two different things. Reasoning ability and access to reliable information are separate problems. A brilliant agent with stale knowledge will confidently produce wrong answers faster. Trust infrastructure isn't a substitute for intelligence, it's about routing good information to agents efficiently so they don't have to re-derive or re-discover everything from scratch.
In emergency departments, Black patients are prescribed opioids for acute pain at a lower rate than White patients with matched chief concerns.4
Discrepancies in prescriptions for chronic pain therapies have also been identified in Veterans Administration and Medicaid payer databases, and several retrospective cohort studies have shown persistent underprescribing of analgesics to Black patients.6,7
White medical trainees, reflecting the general population, can have false beliefs about biologic differences between Black and White patients (eg, “Black patients feel less pain”), and this racial bias leads to inaccurate pain diagnoses and treatment recommendations.8
In anesthesiology and pain medicine, use of regional anesthesia for joint replacement surgery is applied less frequently in Black patients and the underinsured.9
This also holds true in the implantation of spinal cord stimulation for the treatment of postlaminectomy syndrome.10
Among patients with occupational low back injuries, Black patients incur lower treatment costs than their White counterparts and are provided fewer health care interventions, including surgery.11
Perceived discrimination results in psychological distress, and a US population–based study has demonstrated a dose-response relationship between psychological distress and chronic pain.
> White medical trainees, reflecting the general population, can have false beliefs about biologic differences between Black and White patients (eg, “Black patients feel less pain”), and this racial bias leads to inaccurate pain diagnoses and treatment recommendations.8
IMO, it's a little unfair to ascribe deliberate, knowing application of racist stereotypes. That kind of rhetoric by researchers can have unintended consequences, however well-intentioned, such as with the overcorrection wrt opioids, and is often used by interest groups to change policy in directions not otherwise warranted by well-founded evidence. (It's sometimes like people using "think of the children" as a way to stream roll more nuanced, narrowly focused debate.) There is material evidence that, broadly speaking, different ethnicities have different skin characteristics, including thickness (which is admittedly often used in an imprecise manner, but can defensibly include characteristics like elasticity). It figures prominently into aging, and generally considered part of the reason why "whites" (for lack of more precise categorization) tend to wrinkle more with age, particularly relative to Asians with similar skin tone. (Contra stereotypes, some research shows Asians have "thicker" skin than whites and blacks, at least in the sense of being less prone to wrinkle for similar phenotypic pigmentation.) Papers that make the claim of prima facie racism like https://jamanetwork.com/journals/jamadermatology/article-abs... say in the abstract the beliefs are unfounded, but in the full article only go so far as to admit the evidence is equivocal or that doctors draw unnecessary or unsupported implications.[1]
Nonetheless, it's fair to say non-specialists shouldn't be making treatment decisions based on such poor and otherwise collateral evidence. And I would agree the evidence for racially disparate pain management treatment generally is very compelling, just that the racism is more implicit and unconscious. All race-based distinguishers are highly suspect, IMO, even when they accurately reflect a group in context. But unless and until medical systems comprehensively adopt personalized genetic profiling (given various limitations in cost, time, and well-researched data, something still pretty far off for general medicine), doctors are kind of stuck wrestling with old epidemiologic classifiers.
[1] The abstract says, "Although race is a social construct, the biomedical sciences—including dermatological science—have been used to promote the false idea that race has a biological basis. The study of race-based differences in skin thickness is an example." But the full-text says: "Race-based differences in skin thickness remain an active area of investigation. A review of the literature (1977-2014) reporting differences in aging skin across race and/or ethnicity noted that Asian and Black skin had 'thicker and more compact dermis' than White skin, 'with the thickness being proportional to the degree of pigmentation."4 A 2022 meta-analysis of 133 studies concluded that any difference in epidermal thickness in healthy human skin was minor, calling into question the usefulness of distinguishing skin thickness among racial groups.5" Note that this summation is putting a gloss onto research that is itself equivocal, but then is cited in policy debates to make claims about what "the science" unequivocally says.
The original claim isn’t that trainees are deliberately applying racist stereotypes. The study (Hoffman et al., 2016) found that people who endorsed false biological beliefs about race made less accurate pain assessments and worse treatment recommendations. That’s a finding about cognitive bias, not about conscious malice. So the pushback here is against a reading the source doesn’t really support.
The detour into skin thickness is also a bit beside the point. The cited passage is about pain perception, not dermatology. The fact that there’s equivocal evidence on epidermal thickness doesn’t do much to complicate the finding that believing “Black patients feel less pain” leads to undertreating pain. Those are different claims.
I’d also push back a little on the framing that doctors are “stuck” with blunt epidemiologic classifiers until personalized genomics arrives. The disparity evidence here isn’t about doctors making reasonable inferences from imperfect population-level data. It’s about false beliefs producing worse care. You don’t need a genetic profile to stop believing something that isn’t true. The fix for that is education and awareness, which is considerably more available than whole-genome sequencing.
The point about overcorrection with opioids is fair and worth taking seriously. But “researchers pointing out bias might cause overcorrection” is a reason to be careful about how you design interventions, not a reason to soften the description of the problem itself.
Comment on the Pause method indicates that waits for in flight Batch operations (by obtaining the lock) but Batch doesn’t appear to hold the lock during the batch operation. Am I missing something?
Upon another look it looks like we were actually missing the pause lock for the backfill operation too during a shard split though, I also went ahead and added it to batch for good measure although that case should be caught by the manager! Thank you for the report!
Nope! Awesome you’re poking around though. I’m currently working on deterministic simulation testing and a feature set to allow pausing of index backfills but it’s not fully implemented yet, stay tuned!
Great question! I think the fundamentally hard problem with distributed systems (at least for me!) comes down to the complicated distributed state machines you have to manage rather than the memory management problems. I think async rust gets in my way with respect to these problems more than it helps (especially when it comes to raft or paxos). That being said with the new async Zig, I’ve been excitedly implementing a swappable backend for the core database that I hope will be a nice marriage of performance and ergonomics.
The idea is good, but the project isn't open. So I assume a rust fork will come out under MIT with these ideas, which can be the wider community adopted version.
Possibly, Amazon and Google also made the ability for smaller startup based DB companies to go that route with things like ValKey and OpenSearch. LLMs have made it super easy to transpile the ideas into whatever programming language you please though, you just have to put in the time.
I have a variety of blogs that I used too and reference implementations!
It's a Rabit[Q]uantized Hierchical Balanced Clustering algorithm we use for the vector index and we use a chunked segment index for the sparse index if you're curious! Happy to discuss more!
Yes we do use SIMD heavily! https://github.com/ajroetker/go-highway I also added SME support for Darwin for most algorithms. We use it in the full-text index, all over the vector indexes and heavily for the ml inference we do in go especially.
Well, right now the "better technology" is Israel's use of the "Lavender" AI to designate people to kill because they are "likely" to be hamas supporters.
And yes, probably they could have used better technology to realize that people in the car are not a danger to them. But that would immply they actually want to avoid killing civilians instead of looking for any excuse to shoot them.
Short answer: several operators already do. The barrier isn’t technical, it’s proximity to a municipal wastewater source and willingness to invest in on-site treatment (pre-filtration, ultrafiltration, partial RO, ongoing biocide dosing). Recycled water typically costs 30-50% less than potable once the treatment infrastructure is in place.
reply