Charlotte, l love the stone and I think it is smart to keep it. I also really love the wall paper. I think if you painted the stone white and places some of those white animal bust above the fire place it would look really nice. I also think you could build a simple built in shelf on the left side of the fireplace. That would take away from the weird indent between the brick chimney and the fireplace.
Here are some images I found
[caption id="" align="alignnone" width="442"]
painted stone[/caption]
[caption id="" align="alignnone" width="445"]
wall paper fireplace 2[/caption]
[caption id="" align="alignnone" width="469"]
wallpaper fireplace[/caption]
[caption id="" align="alignnone" width="302"]
animal bust[/caption]