Charlotte, l love the stone and I think it is smart to keep it. I also really love the wall paper. I think if you painted the stone white and places some of those white animal bust above the fire place it would look really nice. I also think you could build a simple built in shelf on the left side of the fireplace. That would take away from the weird indent between the brick chimney and the fireplace.
Here are some images I found
[caption id="" align="alignnone" width="442"] painted stone[/caption]
[caption id="" align="alignnone" width="445"] wall paper fireplace 2[/caption]
[caption id="" align="alignnone" width="469"] wallpaper fireplace[/caption]
[caption id="" align="alignnone" width="302"] animal bust[/caption]