Two ends of hardware acceleration
One of the ways that you can categorize hardware acceleration is to say that there is a continuum between two sorts of hardware acceleration: doing something instead of having the CPU do it, and doing something that the CPU can't even come close to doing fast enough.
If you have a choice, you obviously want to be on the latter end of the scale. Ideally you'll have a fairly solid proof that the best software implementation on a general CPU can't possibly be fast enough because it requires very hard hardware performance (very low-latency access to a lot of memory, for example, as you might get with extensive lookup tables). This gives you a reasonable amount of confidence that Moore's Law as applied to general CPU performance is not about to eat your lunch in a few years.
Life at the other end of the scale is much more difficult, because you run into the hardware RAID problem, namely that you need to find people for whom the problem is important and who are also CPU constrained at the same time. (It is a tragic mistake to merely find people with your problem; to put it one way, there are a lot more people with slow disks than people who will pay much money to speed them up.)
On a side note, sometimes doing it instead of the CPU can be a sales pitch in its own right, but you have to be in a special circumstance. The best example of this is hardware cryptographic modules for signing things, where the attraction is that the CPU (and its buggy, vulnerable software) gets nowhere near your signing keys.