The way I've always built my Customer Advisory Board (I call them my A-Team, Advisory Team - just sounds cooler)

Before Launch
- Find users who currently have the problem and have already solved it themselves.

After Launch
- Find users who are active / using the app AND have not asked to be on the A-Team.

The key isn't to add people who are excited to give feedback as they're usually trying to sneak in requests to meet their own needs ... instead find ideal users who have shown you by their actions and profile that they would give relevant feedback and have the time.

The other filter is the people who will be paying for the product / service.