Even if someone is recording some illegal activities without permission, will they still get arrested?

Like on a tape recorder. I learned in school that if you record something without permission, you would go to jail. But what if the stuff that you are recording is just as illegal, then what will happen. For example: I am walking around one day and I hear these criminals conspiring and I record it and send it to the police. What if the criminals try to use it against me because I did record it without permission, but still they were doing illegal activities so I do have the right. Correct?