It is all a matter of negotiations with the buyer.
The straight forward way is if you add a clause that you can use and resell the software with a perpetual license. This way they will have the software, but you will be able to use it also. This is the most neat and clear way to do it with just 1 paragraph. This is what I advise you to do, unless you see a risk of the buyer also having the software.

If you want to have exclusivity on the software, its another story. You should clearly understand how this software is booked as an asset. If its one of many, among software production, and you have not made a clear valuation, the 1M you stated is just an assumable figure. If you want this software to be left for yourself exclusively, you should be careful of public statements of valuation, or you will risk losing this value on the deal, if you exclude it.
To exclude it from the acquisition, you will have to exclude it from your balance sheet, e.g. you can sell it before you sell the company. You can sell it to yourself or another company for a nominal value.

Please provide more details, if you want further advise:
- Are you sole(single) owner?
- Why your accountant says its too late, when you haven't signed the deal yet?
- Did the buyer agree to exclude this software from the deal, or you want to do it without his consent?
- Did the buyer buy the business, knowing about the software?