How to measure the efficiency of Brazilian Federal Universities?

Responsible researcher: Viviane Pires Ribeiro

Article title: BRAZILIAN FEDERAL UNIVERSITIES: RELATIVE EFFICIENCY EVALUATION AND DATA ENVELOPMENT ANALYSIS

Authors of the article: Alexandre Marinho, Marcelo Resende and Luís Otávio Façanha

Location of intervention: Brazil

Sample size: 52 federal higher education institutions

Sector: Education

Type of Intervention: Measure the relative efficiency of Brazilian Federal Universities

Variable of main interest: Relative efficiency

Assessment method: Other - Data Envelopment Analysis

Assessment Context

The study of multiproduct nonprofit organizations is the subject of growing interest in the empirical literature. The main difficulty associated with evaluating these entities is related to a precise characterization of their technologies. Universities and professional hierarchies are a good example of this type of organizations, since their technologies are characterized by multiple production factors and products and profit maximization is not the organization's main conduct. Furthermore, its operations are guided by several general missions and objectives. Consequently, efficiency cannot be defined simply and the issue of measuring efficiency becomes a central research and management challenge.

Intervention Details

Marinho et al. (1997) consider the efficiency measurement approach provided by Data Envelopment Analysis (DEA) in the context of the public sector. The application covered the main Brazilian federal universities for the year 1994, totaling 52 Federal Higher Education Institutions (IFES) and each IFES was treated as an individual production unit (DMU).

Some of the production input variables used in the analysis were: building area; hospital area; laboratory area; total number of students; teachers with doctorates; teachers with master's degrees; administrative staff with a degree or higher education; budget for current expenses; students who started their undergraduate course. Some of the production variables were: number of undergraduate courses; number of postgraduate courses – master’s level and doctorate level; certificates issued – graduation degree; number of approved master's theses; number of approved doctoral theses; weighted average or MEC evaluation of master's and doctoral courses.

Most of the data were obtained by the Ministry of Education (MEC) and the National Association of Directors of Federal Higher Education Institutions (ANDIFES). Information on current expenses, obtained from a public report released by ANDIFES, was also considered. Thus, given the large number of production factors and products, the authors raised the possibility of using factor analysis to explore common dimensions in the data set.

Methodology Details

The methodology adopted by Marinho et al. (1997) is Data Envelopment Analysis, a flexible empirical technique for measuring comparative efficiency and a non-parametric method that uses mathematical programming to construct production frontiers of productive units, employing similar technological processes to transform multiple inputs into multiple products. The DEA evaluates the efficiency of non-profit organizations, with universities being good examples of complex management problems. Efficiency is measured in relation to observed best practice. Furthermore, the methodology also deals with difficulties arising from the unavailability of data on market prices, production factors and products.

DEA models admit two orientations: increase in output (output orientation) or conservation of production inputs (input orientation). In the first case, efficiency refers to obtaining the maximum level of production, given a fixed use of inputs. In the second case, efficiency refers to the minimum use of inputs, given a level of production. When there are constant returns to scale, the efficiency frontier hyperplane is linear and passes through the origin; in this case, both orientations produce the same efficiency results. When there are variable returns to scale, this is no longer the case. However, empirical practice seems to show that the choice of inputs and outputs to be used in the analysis is the best choice, rather than the choice of guidance.

Results

 Marinho et al. (1997) developed a data envelopment analysis application, using information about Brazilian federal universities. The authors undertook an experiment using the DEA of the Federal University of Rio de Janeiro, as part of their budgetary and institutional assessment activities, having both challenging and positive motivations. The ranking of production units was generated according to the efficiency score and suggested the importance of universities' management activities. Furthermore, a notable result is that the majority of IFES recognized in academia were evaluated as efficient production units.

The production units that presented 100% efficient results constituted the "efficient frontier". The authors emphasize that the exploration of common dimensions in the data set, through factor analysis, was fundamental to allow an adequate discrimination of DMUs.

According to the authors, the objective of motivating the systematic application of data envelopment analysis as a subsidiary policy instrument has not been fully met. Thus, new information and inventories will certainly improve the results of AED applications.

Public Policy Lessons

How to measure the relative efficiency of Brazilian Federal Universities? Given that data envelopment analysis evaluates the efficiency of non-profit organizations, this alternative technique can be used to study the efficiency of higher institutions. In Marinho et al. (1997), the authors emphasize that the DEA provides targets for each production factor and product, in this sense, the menu can serve as an information support for planning and monitoring the activities of production units. Thus, this type of analysis can be especially useful for measuring comparative efficiency and can therefore constitute an important management tool in the domain of complex organizational systems, characterized by multiple inputs and multiple products (even in cases where technology does not is well known) and where budgetary and financial support needs stronger coordination and monitoring instruments.

Reference

MARINHO, A., RESENDE, M., FAÇANHA, LO Brazilian Federal Universities: Relative Efficiency Evaluation and Data Envelopment Analysis. Brazilian Journal of Economics . v. 51, no. 4, p. 489-508, 1997.