ToME: the Tales of Maj'Eyal

I noticed in the character guides that the option to boost a talent tree (by spending a category point on it) doesn't seem to be used very often, in general. The only tree I've seen this recommended for is pugilism (brawler). I guess it's because of diminishing returns? Is there no other talent tree...