Blocks that are unused by the filesystem still have to be carried around by the SSD if they are not TRIM'ed, and that is extra work. But it is not actually a great deal of extra work. In particular, filesystems do a lot of re-use of deleted blocks - when the logical block is re-written, the SSD knows that the old data is no longer needed.
The main purpose of TRIM, and the reason it was introduced, was not so much to avoid moving around unneeded copies of old data, but to make sure there are free erased blocks available when they are needed. With modern SSDs with over-provisioning and better garbage collection, that is no longer necessary - there are always free blocks, since there are many more physical blocks than logical ones.
If TRIM were well implemented, then it could still have been useful. But it is a synchronous command, and cannot be queued - this means it is slow, and breaks the flow of commands. So the result is that operations such as "delete" can take many times longer to complete if TRIM is enabled.
Mér hefur einmitt alltaf þótt TRIM undarlegur hlutur, software lausn á hardware vandamáli.