If you want to transfer to a new college but get rejected and remain in the original college does it make you look bad or affect you negatively?

Not at all. The only people at your old college that will know about it will be the people you asked to write you reccomendations (and they likely know you well enough not to judge you by it). College admissions offices send out transcripts to other colleges all the time, and although they keep records, they don't use the records for anything or report them to anyone. Applying to schools isn't like applying for credit, and the only risk you are making is the time and money you invested in the application.