I've been watching Dagger with great interest, although have not moved production workloads to it (nor, admittedly, even committed an afternoon to setting up any workflows/graphs).
Passive comment readers should be aware that ^shykes here cofounded Docker (my gratitude), so it's really worth a look.
Can anyone comment on the ergonomics of Dagger after using it for a while?
I was just looking at the docs earlier this week to consider a migration but got confused by the AI sections...
You're not the only one... At some point last year, we discovered that CI/CD workflows and so-called "AI agent workflows" have a lot in common, and Dagger can in theory be used as an execution engine for both. We attempted to explain this - "great for CI/CD and for Agents!". But the feedback was mostly negative - it came across as confusing and lacking focus. So, we are rolling back this messaging and refocusing on CI/CD again. If you haven't checked our docs in the latest 12 hours, it's worth checking again: you'll see clear signs of this refocusing (although we are not done).
In doubt, I recommend joining our public discord server (https://discord.com/invite/dagger-io) it is basically a support group for CI/CD nerds who believe that a better way is possible, and want to discuss it with like-minded people.
Yes, the whole thing. The only thing in the entire repo I wrote was the CLAUDE.md, which was the only file at the beginning.
I just fire up `claude` and say "Have fun!".
The only other nudge I've given it is once encouraging it to not be afraid to be ambitious, and that if it wanted me to distribute anything I would (and would provide it with user feedback if it remembers to ask me about it).
Interval trees or ordered segment sets are the "correct" answer but sometimes you can accept constraints in your domain that radically simplify– for example, with equipment availability and time slots, you may have a fixed window and allow for slots to only fall on 5 minute marks. With those 2 constraints in place, a simple bitmask becomes a viable way to represent a schedule. Each bit represents 5 minutes, and 1=avail 0=unavail
I tried a bitmask/bitset/bitvec but found the performance of searches to be a worse than the ordered map I'm using. Intuitively I expected it to perform slightly better for searches because of the cache efficiency, but I guess the way my ranges were distributed made it spend a lot of time searching within each element of the set. I'd like to revisit it eventually though because I think this direction is promising.
- To find runs of 15+ 1s you can find bytes with the value 255 using SIMD eq
- For windows of width N you could AND the bitmap with itself shifted by N to find spots where there's an opening at index i AND index i+N, then check that those openings are actually contiguous
- You can use count-trailing-zeros (generally very fast on modern CPUs) to quickly find the positions of set bits. For more than 1 searched bit in a word, you can do x = x & (x-1) to unset the last bit and find the next one. This would require you search for 1's and keep the bitmap reversed. You can of course skip any words that are fully 0.
Thanks for the ideas! I did try lzcnt/tzcnt with bitshifts to find contiguous intervals across elements, but I found that it tended to be more expensive than searching the map depending on the density. I think it's a really promising approach though, and I've like to figure out a way to make it work better for my test case.
Gotcha. I wasn't quite sure from your post, are you storing free slots in the map, or booked slots? And do you know, for the cases that aren't fast enough, how much they tend to search through? Like are there just a ton of bookings before the first free slot, or are the slow cases maybe trying to find very wide slots that are just hard to find?
I'm storing free slots right now, but either would be ok if a data structure would benefit from the other representation.
The slow cases usually tend to be trying to find wider slots and skipping through many smaller slots. There are often a lot of bookings up to the first wide enough slot too though, so it's a little bit of both.
Maybe an additional "lower resolution" index on top of the map could help as a heuristic to skip whole regions, then using the map to double check possible matches. I have a rough idea that I think could possibly work:
Maintain the sum of booked time within each coarser time bucket, e.g. 1 hour or something (ideally a power of two though so you can use shifts to find the index for a timestamp), and keep that in a flat array. If you're looking for a slot that's less than 2x the coarser interval here, e.g. a 90-minute one, look for look for two consecutive buckets with a combined sum <= 30 minutes. 2 hours or more is guaranteed to fully overlap a 1 hour bucket, so look for completely empty buckets, etc. These scans can be done with SIMD.
When candidate buckets are found, use the start timestamp of the bucket (i*1hr) to jump into map and start scanning the map there, up to the end of the possible range of buckets. The index can confidently skip too full regions but doesn't guarantee there's a contiguous slot of the right size. I don't have a great sense of how much this would filter out in practice, but maybe there's some tuning of the parameters here that works for your case.
Updates should be relatively cheap since the ranges don't overlap, just +/- the duration of each booking added/removed. But it would have to deal with bookings that overlap multiple buckets. But, the logic is just arithmetic on newly inserted/deleted values which are already in registers at least, rather than scanning parts of the map, e.g. if you wanted to maintain the min/max value in each bucket and support deletes.
> Maybe an additional "lower resolution" index on top of the map could help as a heuristic to skip whole regions
That sounds a bit like circling back around to a tree-based system (albeit with depth=2) where each parent is somehow summarizing the state of child nodes below it.
Yeah it's definitely similar to having parent nodes summarize their children. The motivation for the flat array structure was to have an entirely fixed-size array that's contiguous in memory to make it friendly for SIMD filtering. Having the data in the tree on the other hand could probably filter a little better since it can summarize at multiple levels, but my bet is that the flat array allows for a faster implementation for the case here with a few thousand intervals. On the one hand, the whole working set should probably fit in L1 cache for both the tree or a flat array, but on the other hand, a tree with pointers needs to do some pointer chasing. The tree could alternatively be represented in a flat array, but then elements need to be shuffled around. I don't know which is faster in practice for this problem, but my gut says the speed of a flat array + SIMD probably outweighs potential asymptotic improvements for the few thousand case
I don't think I've put the same amount of thought into it, but my gut feeling is:
1. If you're summarizing "a contiguous chunk exists from A to B", that would require a mutable rebalancing tree and the borders constantly shift and merge and subdivide.
2. If you're instead just summarizing "the range between constant indices X-Y is [1/0/both] right now", then that can be done with a fixed tree arrangement that can have an exact layout in memory.
A simple example of the latter would be 8 leaves of 01110000, 4 parents of [both, 1, 0, 0], 2 grandparents of [both, 0], etc. The 3-value aspects could also be encoded into two bitmasks, if "has a 1" and "has a 0".
Hey congrats on launching–
Wanted to drop a note from someone who has been burned: avoid .xyz domains like the plague. They can revoke your control of the domain at any time if you end up on a spamhaus (etc.) list. This happened to me even though I was only using my .xyz for a non-prod env shared with a customer. One day control of the domain was just taken by the .xyz org and I had no recourse other than generic ticket submission.
Hi! This is Ron from .xyz - I wanted to chime in and welcome you to please reach out to us so that we can figure out what happened with your domain.
Please be assured that the XYZ Anti-Abuse Team does not suspend domains unless they are in violation of the Anti-Abuse Policy, and domains that are flagged by blacklists are investigated and confirmed for abusive activity.
The .xyz domain extension is treated just like any other domain extension. Email deliverability issues are most often caused by the email content or the setup, i.e. using free email services on shared hosting, low mailbox usage, using a shared IP to send email, and violations of the CAN-SPAM Act. If you are still having deliverability issues, please contact us at [email protected] so that we can assist you in correcting the issue.
We have been featuring successful businesses and prominent individuals using .xyz domains every Wednesday since 2014. Check them all out here: https://gen.xyz/ww.
I launched a service on .io in 2012 and moved away to a less-good .com in 2016 - apart from the political issues, its registry services feel & look very ancient. I almost lost the original domain through a forgotten password!
This is also true of many TLDs that correspond to localities. For example it is a TOS violation to disparage Las Vegas on a .vegas domain. Not all TLDs are created equal.
Hi! This is Ron from .xyz - I wanted to shed some light on the subject.
XYZ takes abuse very seriously and we do not allow spam, phishing, the distribution of malware, or any other type of illegal activities in our namespaces. We actively intervene when any of the above activities have been confirmed by our Anti-Abuse Team. As a result, the .xyz TLD has a lower badness index score than most legacy TLDs as reported by Spamhaus.org.
If you are encountering spam from .xyz domains, please forward the emails to [email protected] or open a ticket at gen.xyz/abuse to have this investigated by the XYZ Anti-Abuse Team.
Ron from .xyz here and I’d love to clear the air - some outdated mail servers irresponsibly block all new TLDs - even though their junk mail filters are proven to work and should work on content that is junk - no matter the TLD. The .xyz TLD has a lower “badness index” score than most legacy TLDs as reported by Spamhaus.org.
Email deliverability issues are most often caused by the domain reputation, email content, or the setup, i.e. using free email services on shared hosting, low mailbox usage, using a shared IP to send email, and violations of the CAN-SPAM Act.
We have made it our mission to provide affordable domain name options for the many entrepreneurs, small businesses, and individuals who need or want more options without extreme limitations, and they deserve to be treated the same as those using legacy TLDs.
Our Anti-Abuse team successfully combats abuse with the assistance of our numerous cybersecurity partners and contributions from individuals in the community like yourself.
This plus providing education on false information helps protect the reputation of all our amazing end users like HotGlue in the .xyz community.
We feature many of these successful businesses and prominent individuals using .xyz domains every Wednesday since 2014. Check them all out here: https://gen.xyz/ww.
True, but its just a gif file on the page. I can't seem to figure out how it stops the gif animation though. I've looked into this topic before, normally you apply a canvas element on top of it capturing one frame of the image. Or you can hotswap the image / gif at end of animation. But there's none here though, neither SVG or canvas.
Professor Philip Wadler has a fantastic talk titled "Propositions as Types" that introduces the Curry-Howard correspondence, some history behind it and ideas following it.
I use Calibre on desktop, sync’d to dropbox, and use the Marvin app on my phone. I like it a lot better than the iBooks app. It has features like search and tagging/collections. The only thing it doesn’t do that I wish it did was handle PDFs.
yeah after going to press they seem to have taken down the links to the PDF across the internets. Mildly irritating. Does anyone have the pdf downloaded from before it went to press?
Passive comment readers should be aware that ^shykes here cofounded Docker (my gratitude), so it's really worth a look.
Can anyone comment on the ergonomics of Dagger after using it for a while?
I was just looking at the docs earlier this week to consider a migration but got confused by the AI sections...
reply