Oller was born in Bayamon, Puerto Rico, the third of four children of Cayetano Juan Oller y Fromesta and María del Carmen Cestero ávila. When he was eleven he began his studies in art under Juan Cleto Noa, a painter who ran an art academy in San Juan, Puerto Rico. Here, Oller demonstrated enormous talent and in 1848 was offered the chance to continue his studies in Rome by General Juan Prim, Governor of Puerto Rico; the offer was not accepted as his mother felt he was too young.
When he was eighteen Oller moved to Madrid, Spain to study painting at the Royal Academy of San Fernando, under Federico de Madrazzo y Kuntz, director of the Prado Museum. In 1858, he moved to Paris, France where he studied under Thomas Couture and where he enrolled to study art in the Louvre under the instruction of Gustave Courbet. During his free time, Oller found work as a baritone in local Italian operas and would frequently visit cafés where he met with fellow artists including Picasso.
In 1859, Oller exhibited some of his artistic works next to those of Bazille, Renoir, Monet, and Sisley. By 1865, Francisco Oller was known as the first Puerto Rican and Hispanic Impressionist artist and in 1868 he became the founder of The Free Academy of Art of Puerto Rico.
In 1884 he founded an art school for young women which was later to be known as "Universidad Nacional". In 1871, Spain honored Oller by naming him "Caballeros de la Orden de Carlos III" which means: "Knighthood of the Order of Carlos III" and a year later he became the official painter of the Royal Courts of Amadeo I. Oller developed an interest in bringing out the reality of Puerto Rico's landscape, its people and culture through his works of art. Oller's paintings can be found in museums worldwide, including the Louvre in France. Francisco Oller died on May 17, 1917 in San Juan, Puerto Rico.