I think this is an odd problem. I have to put a lot of pressure on my bag. And I mean a lot. To get an upper E. And I have to keep the pressure up or it will fall off. But upper octave F#, G,A, B etc. are easy to get to with a mild increase in pressure. Anybody experience that? Any fix that I can try?
This is a strange one. On the one hand, E in both octaves is, on most chanters, a weak note: lower in volume than practically every other note on the chanter, typically with a thinner tone to boot. But the tradeoff is that it’s usually the easiest note in the second octave to sound; it’s even the gateway to the second octave: e.g., if you have to play a high “B” from the lower octave, a quick grace-note on high E is usually the best way to “prime” it.
So, you need to fix this. 
By “drop off” do you mean “goes flat,” or (as I’d assume) does it lose the octave altogether? If it’s just tending to sound flat, then I’m pretty sure the fix is to insert a rush or wire that reaches up around or even a little past the bottom of the ghost-D hole.
Hey, is there anything already stuck in the bottom of your chanter? If so have you tried removing it? People often stick a loop of wire, a bent strip of yoghurt-container plastic, or other foreign object into the bottom inch or two of the chanter, to make “hard D” more pronounced. But anything stuck too far up could cause unintended consequences on E and ghost D.
If none of that does it, do you have a different chanter reed you can try? While I’m inclined to blame your chanter or stuff stuck into (or needing to be stuck into) it, heaven knows I’ve got more than one reed with “one-off” bad notes, so your reed could be the culprit. Every reed is a unique, ungodly hairball of interrelated variables (and that’s when they’re working).
Good luck,
Mick
It’s an odd thing alright but it happens a lot.
If you look at the physics of it there is 2 note holes that produce that note so a good bit or air is escaping from those holes which reduces bag pressure and thus pressure necessary to create that octave.
Plus that E note is the the first second octave note on the chanter so the reed has to be gently persuaded to play that note.
There are reed adjustments you could make during the reed making process but you may sacrifice other tonal qualities too.
Don’t be too hard on yourself anyway, it can be a tricky note to get right. Practice a tune or 2 with an awkward E’ here and there.
All the best.
Tommy