It's just a big granite house full of stuff. A lot of food, about 20 picks, a few shovels, some crossbows, about 60 spears, some cloaks, some armor, some shoes, a few wands of stone to mud, one or two Life scrolls, etc.
Basically, it has rudimentary exploration supplies. It's one square up from town and 16 squares to the west. To the west beyond it is a large desert.
If you want to go way out to the edges of the world (it does have edges, in this game) try and hug the borders between zones, so if something bad shows up you can just go to another zone. Try and bring a missile weapon with a big damage bonus.
Good luck out there, if you decide to keep exploring.