It’s not clear how much the “hand abacus” pictured, with its sliding counters in grooves, was used in practice. Most calculations in Roman times were probably done with a free-form counting board with some kind of tokens (for instance “calculi”, i.e. pebbles). We don’t have a lot of surviving evidence though, since a wood/cloth/drawing-on-the-ground counting board with pebbles for counters isn’t exactly the most permanent artifact.

The way such counting boards worked in Europe in the Middle Ages was to put pebbles on horizontal lines representing powers of ten, or in the spaces between the lines representing five times some power of ten. The Roman ones probably weren’t too different.