First of all, your proyect is OK as is, I respect your criteria, of course.
My point in that regard was about the 3GHz requirement was because of the implementation, mainly, not because accuracy requirement. I bet 2-4x faster is possible, without reducing portability. If for "maintainability and readability" you mean keep it in C++ (L1 code cache impact, indirections, data structure penalty, etc.), I agree, that will be a penalty as fix cost in time and space.
My point in that regard was about the 3GHz requirement was because of the implementation, mainly, not because accuracy requirement. I bet 2-4x faster is possible, without reducing portability. If for "maintainability and readability" you mean keep it in C++ (L1 code cache impact, indirections, data structure penalty, etc.), I agree, that will be a penalty as fix cost in time and space.