A model is never going to be a perfect reflection of reality, and the data that feeds a model is always going to have some gaps or errors in it. Thus, when making a decision with data, it's important to understand the limitations of the data (e.g., how it was collected, what it specifically describes, what it is missing, etc). It's also important to know what questions to ask about the underlying model (e.g., what assumptions are being made in the model). I have seen many examples of established corporations with large budgets that staff entire departments who sit between the data analytics people and the business strategy people, and this third party will just pressure test the work of the data analytics department.
Two final points I'll make:
- If you are not a large corporation, then it's a good idea to speak with a third-party data scientist who will pressure test the data analytics work honestly and thoroughly, knowing which questions to ask. It's only by pressure testing the data analytics that you understand its strengths and limitations. From there, you can gauge how valuable it is in your decision making.
- When I make a decision off of data, I always check if the results of the analysis align with my expectations of reality. If they do not, then I dig deeper to understand why not. From there, you either find problems in the data or you find a new insight.