I don't know what are you using but I've built apps using Spring (known as a resource hug) which take up 10MB of memory and start up super fast. You only need some fine tuning to get these numbers. Alternatively you can also use things like Quarkus: https://quarkus.io/ which are hilariously cheap to operate and can work with Lambda as well.
To get super-fast Spring start-up, do you have to turn off classpath scanning? That seems like it would always inherently be slow. What else do you tune?