Reference counting is a form of garbage collection. It does spend CPU cycles trying to figure out what isn't needed, so your claim that it doesn't "waste CPU cycles" is not true. Its advantages over tracing GC are that memory can be reclaimed promptly, that it does a pretty good job of spreading out the CPU load in an incremental sense, and that it's simple to implement and reason about. The big downside is cycles; lots of large reference-counted codebases constantly struggle with cycles (for example, Firefox). Adding cycle detection to a reference counted system is not easy to do in a performant way and almost nobody that I'm aware of but Firefox even attempts to do it.

Without unsafe memory management, you can still do pointer arithmetic. See Cyclone's fat pointers, Go's slices, Rust's slices, etc.