How do you keep replayed tests trustworthy over time as dependencies and schemas evolve? (i.e. without turning into brittle snapshot tests)
Also, how do you normalize non-determinism (like time/IDs etc.), expire/refresh recordings, and classify diffs as "intentional change" vs "regression"?
1. With our Cloud offering, Tusk Drift detects schema changes, then automatically re-records traces from new live traffic to replace the stale traces in the test suite. If using Drift purely locally though, you'd need to manually re-record traces for affected endpoints by hitting them in record mode to capture the updated behavior.
2. Our CLI tool includes built-in dynamic field rules that handle common non-deterministic values with standard UUID, timestamp, and date formats during response comparison. You can also configure custom matching rules in your `.tusk/config.yaml` to handle application-specific non-deterministic data.
3. Our classification workflow correlates deviations with your actual code changes in the PR/MR (including context from your PR/MR title and body). Classification is "fine-tuned" over time for each service based on past feedback on test results.
Have been building Saleswhale (https://www.saleswhale.com/) for a while now - we got to >$1M in annual revenues mostly selling to the mid-market, but decided to launch a Free Tier this year.
The problem we are solving is helping marketers turn more underserved leads into revenue through an email AI assistant.
I think he's talking in absolute terms versus relative terms that founders are used to thinking in.
Imagine that you have $3 million in runway, but you are burning $500K a month. That only gives you 6 months of runway.
It takes perspective to zoom out to realise - "Hey, I actually have $3 million in the bank. That's a lot of money. If I can somehow reduce costs to $150K a month, I actually have 20 months of runway to figure things out."
Unfortunately, this means "cutting more than needed".
Exactly. That survival will be better for everyone, even those you have to lay off because you can build from there and likely hire some of them as things stabilize.
The 2008 recession clobbered my IT company - most of our customers were in the construction field and many of them went bankrupt.
We did the right thing - we cut deep into the workforce to protect what we could and that was the right thing to do. The remaining employees were able to confidently work knowing that their job was secure.
But the act of cutting deeply, and laying off people I liked and admired cause me to want to act like a turtle and to not want to venture out.
In hindsight, I should have marketed and reached out to new potential customers because other IT support companies were going out of business left and right. I estimate that my timidity after the traumatic cuts cause my company four years of growth.
If I have to do it again, I will. But afterwards I'll act like it never happened - I'll keep growing the company.
In Germany, and other European countries, there is a tool called "Kurzarbeit". It allows a company to reduce the workforces hours and temporarily cut pay. The state supports it with tax breaks. The result is that a worker only works i.e. 60%, income drops to 80% (bc of the tax breaks) and the company saves a lot of money.
The key point: people keep their jobs AND companies keep their trained staff. So when the crisis is over, they are ready to ramp up again. Also, since people keep their jobs, consumption does not drop that low.
This helped the Germany economy massively to recover from the 2008 crisis.
Not sure if it would work the same way for all parts of the economy, but it definitively works for the manufacturing industry.
I'm German, so I might be biased but I think Kurzarbeit (for our non-german speakers, it is pronounced: koorts-arbite [0]) (short-time working) is as close as possible to an economic wonder weapon during a recession:
People are still paid, companies do not have the full burden of wages to pay and especially do not have to go through hiring and training new employees after the episode when they want to ramp up again. It's an awesome tool to deal with a temporary situation and particularly aimed at making recovery from the situation easier (by riding it out instead of first succumbing to it and then having to get back on the feet).
Same in the Netherlands. There was always something similar, but as part of a stimulus package for the Corona crisis the government will pay part of the wages for companies that have 20%+ revenue loss, and this scales up to 90% wage payment for 100% revenue loss.
I’ve been telling my sales staff at every meeting that now is the time to push when others are panicked. Demonstrate your reliability and professionalism. It’s working..
I am a software engineer who learnt how to love sales and marketing.
A few months ago, I decided to reach out personally to a few “stale leads” in our database to audit our processes, and manually check if our automation systems were functioning normally.
I got more than a handful of emails responding with replies like -
"Yes, I reached out in April as the team was interested in Saleswhale. But we didn't hear back from your sales team. That said, the team couldn't wait to implement a solution, and have went with another similar vendor."
Which was alarming, to say the least.
So, I wrote a short article chronicling our struggles with the "nurture gap", and how we eventually solved this problem.. with engineering ;)
Hey there. I took a look at your landing page and it's an interesting premise. We are also in alpha stage right now for Saleswhale (http://saleswhale.io) since 4 months ago, and I've learnt alot about the process.
One thing I've learnt is to look at the pre-public phase (alpha, beta) as a continuum rather than as a hard milestone. The whole purpose of this phase is validated learning, and to make something that people love. Growth is generally not important to us now, although we are still tracking lead velocity (number of inbound email signups weekly).
Not pursuing more users right now at the expense of product development and customer development is generally the smart move, provided your initial set of alpha users are well qualified, and are the right target market for your offering. Reason is that you really don't want too much noise to signal ratio, and too many unqualified users will cause your product to be dragged into different directions or trying to be everything to everyone.
Just to go into specifics, we currently have over 800 alpha signups, and we are rolling out by cohorts weekly to different groups of alpha users, which we are prioritising by a weighted heuristic on how long ago they signed up and potential fit based on their industry and role for our software.
We are tracking engagement aggressively - every single click, action and event gets pulled into our analytics database, and we have a single most important metric that we are tracking - number of interactions a day per user. You may have your own metric that you are tracking. Until we see this metric improve exponentially month on month, we will continue to stay in alpha/beta to work on improving the product and features rather than launch to the public. Because what's the point of getting so many users, if they won't engage and you can't keep them?
Hope that helps, and all the best for your startup!
Congrats on 800 sign ups, thats huge! I agree with you on aggressively tracking metrics, although I think I need a bit more than 20 users for my alpha. Its nice to have a concrete number to compare against, even if we aren't in related niches.
Hi there.. it's really interesting because that's a core feature that we are focusing on at Saleswhale (http://saleswhale.io/product-tour). When you click into a contact's dossier (sadly, I'm not able to upload screenshots here), it will show you all related contacts, interactions and other smart insights that we managed to surface.
We are currently still in beta though, but we would love to work closely with you and get your ideas and feedback how to make Saleswhale more useful for you.
I'm the technical co-founder at Saleswhale and you can drop me a line at [email protected] if you want to chat more.
Also, how do you normalize non-determinism (like time/IDs etc.), expire/refresh recordings, and classify diffs as "intentional change" vs "regression"?