I get distracted easily and whistle browsing is one of the things that gets me distracted. 
Lately I couldn’t help but realize how many plastic whistles there are, with many cool perks to them (The carey parks tone ring, dual heads, two or three piece for carrying). The only problem is that out of the few whistles I have played I haven’t played a plastic whistle.
Is there a difference? Is it enough to matter? Should I even be worrying about/questioning this? 
Maybe but probably not. No. No.
Most makers and many of us acolytes will say that material alone makes no difference. That’s different than asking if how a maker incorporates a material’s characteristics into their instruments might not make a difference. But that’s a function of design and not, strictly speaking, material. I own Busman whistles in delrin, dymonwood and two hardwoods. All of them are slightly different but the differences are so subtle that I would defy anyone to tell me which is which based on material used.
You shouldn’t worry about it. It doesn’t matter in the end. In the beginning, you might find that a nickel whistle feels different than a brass whistle and that both will feel different than wood or plastic. But it’s a personal thing and won’t have any effect on your playing down the road. You may grow to prefer one material over the other but I haven’t gotten there yet, and it won’t be about how the material makes them sound IMHO. There are instruments I prefer over others but not because of the material.
I have a fetish for brass Copeland’s and old brass Chieftains but that is strictly my personal preference. My favorite whistles are neither of these.
FWIW Carey Parks makes excellent instruments. Get one, or two.
ecohawk
Is there a difference? Is it enough to matter? Should I even be worrying about/questioning this?
Yes
Maybe
Maybe, but probably not - kind of depends why you are worrying.
PM me - I’ll send you a plastic whistle to try if you would like.
the above was cross posted with ecohawk - I will amplify my 1st yes: I believe material does make a difference but it is generally quite subtle and differences in design and even very slight differences in the execution of the same design can make more difference to the end result. I happen to like plastic and wood whistles, and prefer either to a metal whistle usually - which is not to say that I would prefer any plastic or wood whistle to a metal one, design and construction are much more critical. Part of my preference is that I prefer the feel of wood or plastic.
Ah, “acolyte”. Thanks, ecohawk, for giving a name to where I fit in this little world.
Not so much the material, as what the maker does with it. There are things you can do in delrin that you can’t do in brass. There are things you would do in wood that you wouldn’t bother to do in PVC.
And, as highwood says, how it feels in your hands when you play it.
I agree. Those slight differences almost certainly come entirely from TINY variations in the voicing of the whistles, which in my case is done completely by hand. I’ve often made two supposedly identical whistles, with wood cut from the same piece, and had them coming out noticeably different in playing characteristics. This serendipity is the joy and bane of whistle making.
Thanks! 
I think I am understand this now. It’s the whistle and how it’s made, not the material that it is made out of.
ARE YOU A MINDREADER?
I had been thinking saving up for a Parks whistle because of the tone ring. Right now my Clarke Original alerts the whole house that I am practicing (not a good thing when I should be doing homework or chores).
alerts the whole house that I am practicing
Take the whistle mouthpiece out of your mouth and press it gently against your lower lip, blow over/across the window and play your tune. You will be able to hear but probably no one else will, even if they are in the same room. Great for practicing ‘under the radar’.