You can use matlab or mathematica software tool which are used to solve such equations.
Scilab is also free tool in case you do not have matlab license.
If A and B are constant I am not getting the reason that why there is no solution in terms of omega?
We have two equations and two variables.
Please tell me why there is no solution as I am getting all curious now
The product \[X(\omega)Y(\omega)\] has a unique value for each \[\omega\]
Then, never be simultaneously \[A+\frac{1}{\omega^2}\] and \[B+\frac{4}{\omega^2}\], unless A or B are functions of \[\omega\] and \[A(\omega)+\frac{1}{\omega^2} = B(\omega)+\frac{4}{\omega^2}\]
However, you would find only the product, not the factors.
yes you re right ,w has influence on A and B , but important thing must be remarked is that A,B haven't a simple relation with w .but totally yes we could interpret A(w),B(w)