To be precise, the Temple dates back to 1218 when the Second Regent Yoshitoki Hojo {yoh-she-toh-key Ho-joe} (1163-1224) built a prayer hall sacred to Yakushi Nyorai {ya-koo-she nyo-rye} (Bhaisajya-guru in Sanskrit). The original structures and statues were lost by fires in 1243 and 1251. Later in 1296, Sadatoki Hojo (1271-1311), the Ninth Hojo Regent, renovated the hall and named it "Kakuonji", nominating Shin-e Chikai (?-1306) as the founding priest. The primary purpose was to perform incantations and supplications for victory over the Mongolian troops. Back then, Japan was under pressure to become tributary to Mongol. They threatened to invade Japan if Japan did not agree to their proposal. After Japan's refusal, they attacked northern part of Kyushu twice, first in 1274 and then in 1281. Though their attacks failed due to timely typhoons, Japan was still exposed to their possible invasion.

Priest Chikai was a disciple of Priest Ninsho (1217-1303), the founding priest of Gokurakuji. During the entire period under the Hojo regime, the Temple was patronized by the Hojos, and functioned as an important institution for Buddhism gathering a number of priests. Likewise, Emperor Godaigo {goh-dye-go} (1288-1339), who temporarily restored supremacy in the 1330s, favored the Temple greatly. In the Muromachi Period (1338-1573), the Shogun including Takauji Ashikaga {tah-kah-woo-gee ah-she-kah-gah} (1305-1358) and his offspring helped maintain the Temple, repairing broken Buddha statues. Takauji in particular appointed the Temple as his main prayer site. After the Muromachi Period, Japan entered into the era of civil strife, and battles were waged nationwide. The Temple went into decline with no particular patrons. When the imperial supremacy was restored at long last in 1868, Shinto was institutionalized as the official state religion. The government ordered to dispose of all structures and statues that were associated with the Buddhism elements. Kakuonji may have been making a hand-to-mouth existence. In addition, the Great Kanto Earthquake in 1923 brought further devastation. It was as recently as 1948 that the real restoration of the Temple started.

Today's Temple consists of several wooden-buildings, and sits quietly at the foot of the hills with no cars running nearby. The road leading there is too narrow for a sightseeing bus to run on.

The Temple has many valuable assets as it is dubbed "a treasure-house of Buddha statues", but if visitors want to see them, they have to be here at a specific hour as mentioned above. A guide priest will take them inside the grounds. I joined this tour a couple of times during winter and there were only a dozen or so visitors. According to the priest, however, more than a hundred people visit here in a busy season such as late April to early May and he needs a mike. At the start of the tour, he will warn that no camera, no cellular phone are allowed to use.

Popularly called "108 Yagura", there are 177 small caves to be exact dug into tuff. Each one has niches for cremated remains. In other words, Yagura here are graves for middle-to-high class people like samurai and priests. Many of Yagura have gorinto or five-tier tombs or cenotaphs inside. The number 108 is familiar to the Buddhists in that there are 108 passions we human beings are subject to. On New Year's Eve, all temples start to ring the bell at midnight and ring 108 times so as to dispel those passions. Inside the caves, there are quite a few stone statues of Jizo installed, but many of them without the head part. During the Edo Period, rumors swirled through the gamblers that they could win the gamble if they carried a head of Jizo in their pockets. Those who wanted to change the luck are said to have stolen them.