This looks like it has the potential to be a no-win situation between you and the plumber.
1) You go to the plumber and tell him that you appreciate the referral but you should be informed in advance of your arrival that he expects a fee. Perhaps you can negotiate "the first one's a freebie and others after that are on a fee-basis" - - IF you want to establish a referral-for-fee element in your business. (How badly do you need his referrals or the additional business?)
2) You can tell the plumber to not refer you anymore if he expects a fee and that you're not going to pay this fee either. You have to know he'll badmouth you as a result though.
3) You can (laughingly) tell the plumber that THIS TIME you'll render to him his so-called-referral-fee (which, by the way, if not agreed upon by you prior to him making the referral, is extortion basically) THIS TIME and mention something about a "highway robbery" and "never again". Then you must decide between option a or b below:
a) Pay it yourself and don't cause the customer any more $ grief AND mention to the customer that you're above board and integrity-based in your business dealings and as such, you'd appreciate if they would make mention of your name to their friends, co-workers, neighbors, and acquaintances if the opportunity arises.
b) Pass the cost along to your customer knowing that he'll be aware that your price included the referral fee. This will also put you in the same light (reputation-wise) as the plumber since it appears for all intents & purposes that the two of you are sort of connected or even a team. If there's ever a problem with the plumbing or any ill will over the cost of the two elements of the project, you and the plumber both will bear the burden of blame. You probably won't get referrals via this customer either.