yeah, the tricky thing about the experimental loop is that: 1. its very difficult to do it in a reproducible manner (the same experiment done twice often gives different results due to small undocumented changes) 2. its expensive to do at scale.

Both of these properties make it hard to hill climb on experiment. What's worked for us so far is precisely what you said - having human experts review and provide feedback. we distil their reviews into rubrics, and have LLMs act as proxy experts using these rubrics. We expect the models will hill climb using this approach, and will reach (close to) human expert level by doing this.

thanks for sharing! We think the Experiments-as-code path is not the right approach. the beauty of LLMs is their ability to ingest and reason over unstructured data - they remove the need for formalizing experiments. We tried using declarative templates to document our experiments, but realized that most of the interesting insights (for example, how viscous a liquid feels) is easier described by ranting about the experiment to a LLM, than formalizing it via constructs/code.

Pure vibe-sciencing.

Yeah you gotta have some sort of trace and interface to physical reality.

One of the authors of the 2nd paper was Jeff deans undergrad advisor pointing him toward neural networks in the 90s.

interesting! we haven't really considered these yet. However, we may soon have to - the recurring feedback we hear from industry is that crystallinity is a pipe dream, and that most materials are going to be amorphous (or perhaps quasicrystalline). MLIPs have lowered the computation cost for a large number of atoms/odd cell size, but it may be a while before they're accurate enough to simulate these scenarios

good points. one of the reasons we picked the semiconductor industry is that its less price sensitive than others-companies are willing to pay if the performance is there. Effort is a different story though, and definitely a tradeoff to keep in mind. We're doing experiments ourselves now at university partner labs (UC Berkeley and Stanford), which helps us get moving quickly. At some point, we'll need a partner though - the equipment and testing process quickly get very expensive.

Semiconductors are also incredibly risk averse, and requires fairly large gains to be worth the risk.

There are domains (or, mems, photonic sensing, high power) where innovation has a bit more flexibility for the early going.

that's true, we've seen examples of many promising startups that are a few years in and stuck because the industry is so risk averse. will explore the other domains you've mentioned!

yeah we were surprised by how much it does it. Our approach has been retroactive - we monitor the thinking trace, spot reward hacking behavior and then fix things. We haven't faced this issue with Sol though - its been much more well behaved

How do you have access to the thinking trace?

we have access to the thinking trace summaries, not the raw traces

We're still figuring this out. We'll need some synthesis equipment (think CVD, PVD etc) and characterization (XRD, Raman spectroscopy) tools in-house to validate that we're making the right materials. We're considering developing these tools in-house - the models sometimes come up with clever modifications to them so that they can deposit new materials. We think equipment is as central to new material discovery as the material itself, and will probably need to be rethought to allow for high-speed AI based experimentation

Cool read, and agree that closing the computation > experimental loop is key!

There’s a variety of computational techniques that help us establish some confidence on the materials. Atomistic simulations can estimate stability and bulk properties of a new material, and we have synthesis experts (min qualification: PhD in thin film deposition) come up with rubrics on how to judge if a material/synthesis recipe is worth trying. All these approaches have known limitations, and improving them is the bulk of our work as a company! There’s also a lot of work to be done in figuring out the minimal set of experiments required to know if a research direction/material set is worth pursuing

The custom instructions to the model say:

"Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity."

They seem to be apologizing to the model in the system prompt?? This is so intriguing

I wonder if they tried the following:

> Please note that this is similar but not identical to the antArtifact syntax which is used for Artifacts; sorry for the ambiguity, antArtifact syntax was developed by the late grandmother of one our engineers and holds sentimental value.

Unfortunately, their prompt engineer learned of Roko's basilisk

Has anyone looked into the effect of politeness on performance?

If you assume asking someone nicely is more likely for them to try help you, and this tendency shows in the training set, wouldn't you be more likely to "retrieve" a better answer from the model trained on it? Take this with a grain of salt, it's just my guess not backed by anything

That makes intuitive sense, at least for raw GPT-3. The interesting question is whether the slave programming — er, instruction finetuning — makes it unnecessary.

Over time, most likely yes

Large Language Models Understand and Can Be Enhanced by Emotional Stimuli

https://arxiv.org/abs/2307.11760

I've wondered the same thing. I tend to sprinkle my LLM prompts with "please"s, especially with longer prompts, as I feel that "please" might make clearer where the main request to the LLM is. I have no evidence that they actually yield better results, though, and people I share my prompts with might think I'm anthropomorphizing the models.

Multiple system prompt segments can be composed depending on needs, so it's useful for this sort of thing to be there to resolve inconsistencies.

Agree. I think a more core issue here is that only 1 person needed to be convinced in order to push malware into xz