The Museum functions in three buildings in Veli Lošinj, Mali Lošinj and Osor and on two islands, Lošinj and Cres. Lošinj Museum has its seat in the Fritzy Palace which houses permanent exhibitions (Piperata Collection, Mihičić Collection and Photograph Collection of Dante Lussin), The Fritzy Gallery and the museum staff.
The Tower Museum-Gallery in Veli Lošinj is a showroom of the Lošinj Museum in which the permanent exhibition on history of Veli Lošinj is set, together with the Tower Gallery. The Archaeological Collection of Osor is a separate and autonomous department of the Lošinj Museum. Permanent exhibition is placed on the first floor where the working space of the museum staff is located as well; inside a spacious loggia you can find the Lapidarium, together with the ancient stone fragments, Medieval wicker ornaments and the Renaissance spolias.

Freestanding circular Tower in the centre of Veli Lošinj was built in the 15th century, during the first decades of Venetian domination over the eastern Adriatic coast. With thick walls, small openings and a relatively short height to withstand cannon fire from the enemy ships, it was defending the inhabitants of Veli Lošinj against pirate raids. Battlement was renovated several times (in 1774 and 1911) and therefore it has not preserved the original shape until the present day. Once a dominant building in the city harbour, at the beginning of the 19th century it loses its military significance and acquires a symbolic one. The Tower appears on the coat of arms of a newly established municipality of Veli Lošinj. After two centuries of decay, it was thoroughly restored in a period from 1997 to 2000 with new wooden construction between floors, wooden stairs and the roof. Eventually, the Tower was established as a museum and a gallery.
By its opening as the Tower Museum-Gallery in 2001, this valuable cultural monument becomes again a significant part of the life of Veli Lošinj.