Not at all really. It's not about the technology, it's about the process. A pentest will (or at least should) determine if you shipped a "secure" product. This company (if it's serious about pentesting all their projects) will assign some kind of risk factor to the website you've built. Information Security is all about identifying risk (at all levels) and mitigating or accepting those risks.

In the case of a an Amazon S3 bucket, I would think the following items should be enumerated in a pentest:

  1. Leaking information via DNS
  2. Secure hosting for DNS records
  3. Secure passwords on your AWS account
  4. Proper permissions set on your bucket
  5. Multiple AWS availability zones
  6. Javascript libraries used are functionally correct
  7. No inclusion of any backdoor features by the developers ;-)

This is more of an audit than a pentest. But sometimes a company will only have peace of mind if they base their measurements off of an established internal process. Even if the tests don't seem to make sense for the technology or implementation they will make sense when it comes to identifying risk metrics across all of their web facing products.

> Is this as insane as I think?

Yup. But that's something they'll need to ask Amazon for permission to do before they can legally proceed. Else, CFAA/relevant local draconian law can smack them down pretty hard.

I do a lot of application security. A friend of mine does front-end design (no Javascript). I don't check her work for security holes because it's pointless; she can't touch the backend code.

As the GNY crew might say, "Context, people. Context."

http://www.textfiles.com/webfiles/ezines/GONULLYOURSELF/gonu...

I've seen similar projects -- if you told them how pointless spending money on such assessments is, and they still want it, then it shouldn't be your problem. Also, there's still a chance that they have some backup or privately shared files up there that DirBuster or similar software could find.