You shouldn't have to pick the flower - It sounds like you have a Solanum dulcamara L., or bittersweet nightshade. These can be poisonous. Check http://wisplants.uwsp.edu/scripts/detail.asp?SpCode=SOLDUL to see if that's the plant you're talking about.
Sounds like your looking at polkweed to me, but I am sure that is not the only one.
laburnum
Border grass has purple berries. Another name for it is Liriope muscari. Keep the berries away from children and pets.
Its a male plant.
Orpine?
they have either pink or yellow flowers
the answer to your question is the Deadly nightshade because it is highly poisonous plant with purple bell flowers and small black berries. A symbol of deception, danger, and death.Deadly nightshade A highly poisonous plant (Atropa belladonna) with purple bell flowers and small black berries. A symbol of deception, danger, and deathThe death flower for France, however, is the crysanthemum.
Ivy does produce flowers, which become berries in Autumn.
Ivy does produce flowers, which become berries in Autumn.
The tall, branching perennial plant can grow to a height of at least 5 ft (1.5 m),and can be identified by its bell-shaped, purple flowers and cherry-sized green berries that mature to a dark purple or black color.
There are gardeners that plant berries in the spring. However, once you have a berry garden established, you don't have to plant more. Berries are perennial so you trim raspberry canes or trim runners in the spring.
they grow in a cluster
SalalA North American plant of the heath family, with clusters of pink or white flowers and edible purple-black berries.- Small evergreen shrub of Pacific coast of North America having edible dark purple grape-sized berries (wordnetweb.princeton.edu/perl/webwn)