Rag Rug Progress -Fabric Balls Wound - Rag Rug Growing

I finally finished prepping all the fabric for the rag rug. That was sure a long process. Actually the ironing took the longest since the fabric wants to twist so much. That is a step that can be omitted. I decided to iron the fabric in half with wrong sides together so you see the fabric on both sides. I wanted as much color to show as possible. But, if you don't care if the back side of the fabric shows then ironing is not necessary. Here are my fabric yarn balls. Now to get started crocheting the rug.

Crocheting the rag rug sure goes a lot faster than prepping the fabric. I will have this done in less time than it took to prep all the fabric. I spent about 14 hours ripping the fabric, sewing ends together and ironing the fabric. I began crocheting this afternoon - I figure I am about a third of the way into the rag rug and only spent about two hours crocheting. Here is what I have done so far. Molly came in to check it out - a new rug to lay on.....

I am not liking the way it looks where I changed colors. I should have joined with a slip stitch before adding the next color. I didn't even notice this until I saw this picture. Darn, do I rip everything out now and start over, or start joining at the end of each color now, or continue crocheting the rug as I have been???? Will I be happy if I don't rip this out????