The original temple was not located here. It was built at the east side of Amanawa Shinmei Jinja in 1225 by Masako Hojo (1157-1225) to pray for the repose of her departed husband, Yoritomo Minamoto (1147-1199), the founder of Kamakura Shogunate, and was named Chorakuji {choh-rak-gee}, which belonged to the Ritsu {re-tsu} sect. As Masako died the same year, her nephew Yasutoki {yah-soo-toh-key} Hojo (1183-1242), who later assumed the seat of the Third Hojo Regent, remodeled the Temple and changed its name to An-yo-in Chorakuji, taking Masako's posthumous Buddhist name "An-yo-in". Later in 1308, however, the Temple was ravaged by fire, and again in 1333 when the town was devastated by the troops of Yoshisada Nitta {yoh-she-sah-dah nit-tah} (1302-1338). Shortly afterwards, the Temple was relocated to the present site, where another temple called Zendoji {zen-doh-gee} had existed. Since Zendoji had been built by a member of the same Hojo family but was also burnt down by the Nitta's troops, the two temples merged to become a new one bearing the name "An-yo-in", and changing the denomination from the Ritsu to the Jodo.

Another fire in 1680 totally destroyed the Temple again. When reconstruction was planned, an idea came up to bring here its sub-temple Tashiro {tah-she-roe} Kan'non Hall situated near Myohonji, and consolidate the temples. Back in the late 12th century, Nobutsuna Tashiro {noh-boo-tsu-nah tah-she-ro} (birth and death years unknown) was a loyal retainer of Yoritomo Minamoto and joined Yoritomo's warfare against the Taira Clan from the beginning. As he rendered distinguished services in the battles and greatly contributed to the victory, Yoritomo granted him with rewards in recognition of his service. The Tashiro Kan'non Hall had thus been built to dedicate to his guardian deity.

Upon the completion of the new Temple, a statue of Senju {sen-jew} (Thousand-Armed) Kan'non or Sahasrabhuja in Skt. called Tashiro Kan'non, was brought here and was enshrined together with the statue of Amida Nyorai. A series of fires produced this unique coupling: A statue of Amida Nyorai or Amitabha in Skt. and that of Thousand-Armed Kan'non.

The Temple structure consists of the gate, main hall, Jizodo hall and priests' living quarter. The main object of worship is the statue of Amida Nyorai . Right behind this statue is a 160-centimeter-tall statue of Thousand-Armed Tashiro Kan'non. Kept inside this statue is another statue of Kan'non, which is said to be the deity Masako offered prayer to for her marriage with Yoritomo and her wishes were answered. As a result, it is venerated like Cupid by young women who wish to have successful love and marriage . Both Amida and Senju Kan'non statues are believed to have been carved during the Kamakura Period (1185-1333). The statue of Kan'non is listed on the third of the Bando and Kamakura Thirty-Three Kan'non Pilgrimage.

On the left-hand altar is a statue of Bishamonten {be-shah-mon-ten} or Vaisravana in Skt. (the God of Treasure) and had been one of the Shichifukujin {she-chee-fook-gin} or Seven Lucky Deities in Kamakura until 1997.

On the right is the statue of Bato {bah-toh} Kan'non or Hayagriva in Skt. (Horse-Headed Kan'non). It is represented with a horse's head carved on its headdress like Minotaur, though not bull's head. Bato is one of the six major Kan'non, but it always has a pugnacious expression.

As the Temple was originally founded by Masako Hojo, it owns a couple of her statues. The older one is said to show realistic figure of Masako. Today's hall, copper roofed, was built in 1928.