I believe DLib's tracker is based on tracking hand-made features - probably Cmashift or similar (similar to HOG features on their non-CNN face detector). I'd note that it hasn't been updated sine 2018 as well.

That means the accuracy will be a lot lower - although it will be fast.

But the Roboflow tracker is using OpenAI Clip, which seems a complete overkill and will be pretty slow!

Something using a lightweight object detector should be much faster. This post shows how to use YOLO, which is the first things I'd try would be: https://github.com/yehengchen/Object-Detection-and-Tracking