For a larger company that manufactures wooden windows, we developed an extensive iPad iOS application for optimizing their internal processes (measurements, installation, etc..). The development of the application went smoothly, the project was completed successfully. However, there are now some situations when it is necessary to upgrade the application to new phases with new features/functions. There are some specific problems for which we cannot estimate exactly how much development needs to be invested, and the company's management wants to have a fixed price for implementation and new development (not possible from our side). Since we do not have experience in the field of estimating the scope of unknown work, we are looking for an expert who could consult us on how to estimate and charge software business. We will also need this knowledge for new upcoming projects.
For projects that are fairly open-ended, you can provide a fixed quote that takes into account enough buffer to ensure you are safe. The downside is that could mean losing the opportunity due to having a price that's not competitive. However, that's better than risking your business / layoffs on what could become a wildly unprofitable project.
There are plenty more ways to go about it, but in the end it's going to come down to either (1) determining the price where you feel you have derisked enough to make the potential reward worthwhile, or (2) seeing if someone with experience in the project area can give you a more accurate estimate.
Wish you well with this!
Adding more to Rayan’s point and also elaborating a bit. As a feasibility study you could check over internet if the problems are solved by others. Are there any clue/logic/open-source implementation of them? They could be your exact problems or similar to them. The respective developer can understand the ways. This will help you to identify the blockers if any. Because blockers are the ones which shoots you away out of your estimation. Once the logics are understood break the problem statements into various small demonstrable outputs and estimate them. For any Output there is always an easy part, average part, hard part and harder part. Easy or certain part which you can estimate correctly, because it done earlier, but add few buffers time to hard + harder part.
So, if you don’t have any expert to suggest & estimate, above is way you could approach.
As Rayan mentioned you should always balance between your good/exact estimated time-line and bagging the project. Sometime you need to estimate tight (effective/billable hours) in terms of providing cost but could stretch the calendar deadline through project management negotiations.