Hi, First I assume you are familiar with the fiber characteristic wave equation. If not, it appears on many textbooks, like Okoshi Takanori, Amnon Yariv, Katsunary Okamoto or …Dietrich Marcuze. One option is to determine the "beta" ()propagation constant) at each "v" number separately. The low v numbers require high resolution since the char. equation oscillates very fast and erratically. Above v=1 it becomes relaxed and the beta shall be found very stably .
Another thing is that if can make the relaxed approximated of using LP modes, rather than the full vectorial HE / EH / TM / TE, the equation if much simpler, If your fiber has small delta-n, below 0.02, this is the usually the case. .
Good Luck. .
Y. Sh. ( Full Answer )