It depends on what you mean by "...should you check." That is an odd way to frame a question. The "parts" of NJ that are close to New York City are: Woodbridge Township, Carteret, Elizabeth, Newark, Bayonne, Jersey City, Hoboken, Weehawken, Union City, North Bergen, Cliffside Park, Fort Lee, and Englewood, to name a few. That list starts in the south and moves to the north; I'm sure I've left out some, but that's a good start!
new jersey, pennsylvania
New York City
The New York Rangers
Connecticut, New Jersey and Pennsylvania. Lake Erie, Lake Ontario and Montreal.
I would say Jersey City is the closest city in NJ to NY. The holland and Lincoln tunnel are both in Jersey City and Ellis Island is walking distance from liberty state park, jersey city.
The Hamptons are located on the eastern end of Long Island, New York. They are not close to any other states.
New York, New Jersey, Delaware and, Pennsylvania
middle colonies
California has the highest population density with New York and Texas very close behind it.
The Hudson river I near New York and New Jersey.
New Jersey: Trenton New York: Albany
New York New Jersey