During the late 1950s, a group from John Carroll University and Ursuline College began studying writings on the Montessori Method by Nancy Rambusch, an American educator who studied Montessori in London. Mary Ruffing, a member of the group, invited Nancy to come to Cleveland to give a talk about her school, Whitby Montessori (the first Montessori school in the United States), and about Maria Montessori. Mary and her husband Jim visited Whitby and were inspired to found the Cleveland Montessori Association (CMA) in 1959.

From this start, several Montessori classes were established in existing school buildings throughout Cleveland, first on the west side, then on the east side, making Ruffing the second Montessori school established in the United States. Betty Hissong and Joan Ducas, two members of the study group, taught classes at Ursuline to spread the principles of the Montessori method. As a result, Cynthia and David Ingalls became interested in helping to establish a central location for CMA, purchasing property on Fairmount Boulevard and leasing it and its cottage to CMA for $1 a year.

Through the philanthropy of the Ingalls family, a second preschool classroom was constructed on the Fairmount Blvd. property in 1970; in 1971, the Ingalls donated the land and existing buildings, as well as funds to construct an additional building to house elementary classes. In recognition of this generous support, the Cleveland Heights location was named the Ingalls Campus.

As the scale of the citywide school grew, it was becoming increasingly difficult to administer split locations. The principal, Joan Ducas, spent one day a week at the Ingalls campus of Ruffing, maintaining an office on the west side. In 1977, the east side and west side locations became legally separate, and a new entity was created: Fairmount Montessori Association.

The first principal of Fairmount Montessori Association was David Kahn, who worked as both an elementary teacher and the principal (1973-1985). During his tenure, Ruffing achieved a number of milestones: becoming state certified, adding a third preschool classroom and increasing enrollment. After developing a Middle School curriculum and securing backing from families and local foundations, The FMA board purchased an acre of land from Beaumont School and began construction of new Middle School and Upper Elementary classroom building which opened in 1982. This additional space made it possible to launch a day camp that year: Summer Ruffing It.

In 1984-1985, David Kahn transitioned from principal to head of the Ohio Montessori Training Institute in Cleveland, offering preparation for elementary and preschool teachers leading to certification by the Association Montessori Internationale.

As a parent-operated school, volunteers and trustees had overseen many of the administrative functions of the school. The Board devised a joint headship plan to professionalize these functions and in 1985 John Long, a Middle School teacher, became Principal and Jan Katz, already on staff as Administrator, was appointed co-head for administration. In 1989, the Lower elementary and Middle School buildings were connected by a new gymnasium/community center, which also included an art room, an after-school environment and expanded space for Middle School.

In 1991, John Long resigned to return to teaching in the Middle School. Marilyn Wright, an experienced teacher from the Middle School, served as Interim Academic Dean for the 1991-92 school year. Timothy Duax served as Principal from 1992 to 1994, but then returned to his native Wisconsin. Ruffing teacher Robert Fleischhacker became Principal, without a change in the joint headship structure.

In 1999, the All Day Children's House was established to further serve the needs voiced by parents. In 2001, Gordon Maas was named Head of School, combining both administrative and academic leadership in one position. He sought greater involvement in the Cleveland educational community for the school. Ruffing joined the Cleveland Council of Independent Schools (CCIS) and applied to become accredited by the Independent Schools Association of the Central States (ISACS), receiving full membership in 2006. The Parents' Association was formed to further build community among the parents; both the President of the Parents' Association and a faculty representative began attending Board of Trustee meetings in an ex officio capacity.

In 2004, the Board began discussions about the need to redevelop the aging campus and voted to do so by following Leadership in Energy and Environmental Design (LEED) construction principles. After careful planning, the groundbreaking was held in June 2005. Conducted in three phases, the redevelopment project enabled the school to unify a multi-structure campus, open a Toddler Community, offer a Siesta program, physically expand or replace all classrooms, dedicate improved space for art, music, drama and world language, and most importantly, create a healthy and safe learning and working environment.