I just found out that we can’t even target each term via CSS, as they do not have a unique ID assigned automatically, and we cannot add one via attributes (Make all dynamic data available for Attributes and Tooltips). I have tried with a static attribute and {termquery=term_id} as well.
It seems to be impossible currently to add a different background color to each term…