"To begin with" can probably be used in a couple of ways. It is usually used at the end of a sentence like so:
You shouldn't have even been at that party to begin with!
In this way it can be used instead of "in the first place".
I will now begin the demonstration. Let's begin at the beginning.
To say "to begin" in French, you can use the verb "commencer."
The solar eclipse will begin at 2:30 pm.
I would like to begin with a startling revelation.He seems to be unable to begin with the truth, much less end with it.
The old sandwich in the refrigerator was beginning to smell.
No, You cannot use but to start a sentence.
1887
The ceremony is about to begin.
yesterday
Z
2001
1975