I'd like to avoid the accounting/filing headaches and costs that come along with legal partnerships, but would still like to retain this investor.
You're confusing partnerships and corporations.
If someone owns 10% of a corporation they are simply a shareholder.
Many people create corporations and refer to themselves and other founding shareholders as 'partners.' This may be the source of the confusion. A partnership (general or limited) is a different kind of business legal structure.
What you may also be thinking of is a directorship. The investor may or may not want to be a director. This would give them a seat at the table for Board of Directors meetings.
In certain states and provinces, directorships lead to some types of personal liabilities.
If they want to be a director, you should make sure they get advice on what they may be exposing themselves to.
If your business is currently structured as an LLC, you can admit a 10% equity holder as a limited member/partner, which means he holds an equity stake in the company, but doesn't participate in the day-to-day management activities of the business - that responsibility remains with you. You'll have to amend the capital schedule in the LLC operating agreement to admit the new partner. If the LLC was previously only owned by you, the admission of a second owner means the LLC is now a partnership for U.S. tax purposes, so you're required to file an annual Form 1065 partnership tax return. If you don't want to create a partnership, you could explore having him contribute the funds in the form of a debt arrangement, rather than an equity interest.