At the higher elevations of the High Peaks region, where Panther Gorge is located, there will be snow and ice until at least late April. Yes, even this year with the warm weather and low snow pack. During a typical year there is snow and ice at higher elevations in this region until at least May.
Panther Gorge as a rock climbing destination is typically a July, August, September venue.
If you must visit the High Peaks region during the late March/early April time frame be prepared weather/condition wise for everything; below freezing temps, mud, rain, snow (both on the ground and falling from the sky), ice and maybe a little sun.
There are places that you can rock climb in the High Peaks region during late March/early April. Search the forum. There are several posts about this.
The suggestions of using the High Peaks Information Center (HPIC) and the Mountaineer as resources for current conditions are spot on.
Enjoy your backpacking trip and please try and be as low impact as possible during the mud/shoulder season.