Need Help W/ Text Compression And Tokens!

Hi! I've been working hard on several compression techniques and believe some of them are doing well. However, I'm having problems with one group of them: they are designed to compress lists of strings such that they can be printed out individually. Right now, the best I got was slightly better than 50% compressibility. On the test file, Deflate gave me about 56%. I believe I'm doing tokenization wrong. My current strategy is to check if the current input is already a token; if it is, increment the tokens count, and, if not, add it. My best technique tokenizes words, and even that is not good enough. Is there a better strategy? If there is, how do I do it?