Last week Foundation for Puerto Rico, a local NGO on the Island, published a report that recommends a National System of Tourism Statistics for Puerto Rico. The report states that Puerto Rico inaccurately measures tourism activity by not following international recognized standards set by the World Tourism Organization. Therefore, government officials are incapable of evaluating sector performance, or conducting ROI analysis on advertising campaigns that cost as much as $30 million a year. However, that hasn’t stop the government from announcing positive returns from their efforts.
Nonetheless, the gap in tourism statistics is only a symptom of a deeper problem on the Island. In addition to the economic crisis, Puerto Rico has been stuck in a statistical crisis for over 30 years.
The United Nations develops a series of standards for the preparation of economic statistics in each country, known as the System of National Accounts. This System is the recommended framework for estimating economic activity in a country, from production, consumption, capital investment, income, stock flow, and other financial instruments. Based on these estimates a country can calculate its gross domestic product (GDP). From time to time the United Nations publishes new standards to adjust national accounting to new dynamics in the world economy.
The last time the UN published such standards was 2008. Prior to the 2008, standards were published in 1968 and 1993. However at the present time, economic statistics in Puerto Rico only meet the standards of 1968. In other words, Puerto Rico is measuring the economy as if it were 1968, as if things had not changed in the past 5 decades. Some relevant changes in the world economy include changes in the treatment of intellectual property and research and development (R & D), new financial vehicles, intermediate financial services and changes in the insurance market, among others. These areas of the economy have changed significantly, but because Puerto Rico does not use the latest UN standards the government probably is underestimating or overestimating their value or, in extreme cases, not even providing an estimate at all. Let's think about that for a moment -- How far off could economic estimates be in Puerto Rico? The truth is that nobody knows.
Unfortunately, the problem with economic statistics goes beyond the UN standards, as there are deeper management issues that run throughout the whole government, particularly at Puerto Rico Planning Board, the government entity responsible for national accounts. During the 50s and 60s the Planning Board was an innovative government agency with advanced statistical capabilities. However, it has not managed to keep up with methodological and technological advances, losing its relevance, within government and with the private sector.
We’ll provide some examples. We begin with an area that should be relatively simple in national accounts: data management and governance. This happens to be one of the major challenges at the Planning Board, as they rely on fragmented information systems and datasets, making it difficult for staff to collaborate and produce estimates on a timely fashion. Consequently, sub-programs within the Planning Board produce different estimates for the same variable.
The Planning Board also lacks adequate internal controls to manage and store data, with little use of ERP’s for data processing and sharing. Most estimates and calculations are done in Excel sheets. Although the Planning Board has sophisticated information systems, they are often underutilized by the staff due to lack of training. It is inconceivable that in 2017 Puerto Rico still relies in Excel to produce its national accounts. All estimates and national account information should be stored and managed from a centralized database, with easy access to all departments and other government agencies.
There are also no formal audit processes at the Planning Board, estimates are reviewed only when there are outliers within the data. In most cases, simple imputation processes are used when there is insufficient data, rather than using scientific procedures accepted by the statistical community. It should also be noted that there is very little rigor with survey design at the Planning Board. Sometimes the samples are biased or incorrectly designed, producing invalid and imprecise estimates. This is because most analysts at the Planning Board lacked advanced knowledge in sampling techniques and all require training on survey methodology. Additionally, there is little documentation on statistical procedures used at each department, making knowledge transfer to new staff when employees retire very difficult. Therefore, one can only conclude that many estimates produced by the Planning Board of Puerto Rico are wrong.
However, we cannot blame everything on the Puerto Rico Planning Board, as we have other agencies that share the blame in this statistical crisis. The main conspirator for this crisis is the Department of Treasury. Treasury has the best sources of information on consumption and production in Puerto Rico, as it compiles data through the tax forms. However, the Department of the Treasury does not provide direct access to its databases, and when it does, it does not do so in an open format or in a format that facilitates the analysis of the information.
As a result, Planning Board staff spends considerable time tabulating by hand files that receives from Treasury. In order for the Planning Board or any other agency to produce reliable and recurring estimates and statistics, an open data culture of inter-agency data is needed, after all, there is only one government in Puerto Rico.
Another thing we want to highlight is the input/output table. The input/output table is a macroeconomic tool that measures the relationships that exist between the different sectors of an economy. This tool calculates inter-industry multiplier effects in Puerto Rico.
It is recommended to update the input/output table every five years, since the interrelationship between industries is not constant over time. However, the last input/output table published by the Planning Board is from 2002. Consequently, when the Governor of Puerto Rico announces the economic benefit of its policies, it does so using very old economic assumptions.
Lastly, econometric models at the Planning Board are not capable of making dynamic projections and simulations. In other words, they do not have the capacity to create different scenarios using Bayesian methodologies. Analysts at the Planning Board lack advanced degrees in econometrics. Therefore, the central government (and now the Fiscal Board) are forced to hire external consultants to create models using proprietary methodology and applications. Because the models are not internal, every governor hires his own economists to create models from scratch, using different assumptions every time. Thus, its practically impossible to compare economic projections from two different administrations.
Although much of these failures fall on the Planning Board, to be fair, the Board has lost much of its human capital. In 1994 the Economic Planning department at the Planning Board had 192 employees, it currently has fewer than 35. Likewise, the Social-Economic Department at the Planning Board, responsible for assessing the island's socio-economic development (health, education, poverty) had 28 employees in 1994; it now has only two.
The failures of the Planning Board are due to the lack of interest of the executive branch and the legislature for evidence-based decision-making. In Puerto Rico, budgets are made, programs are created and incentives are given with no impact evaluation. And since we don’t evaluate government programs in Puerto Rico, there is no sense of urgency in improving statistics.
We therefore call for the end of the current statistical system in Puerto Rico and urge the governor and the Fiscal Board to build a new one from scratch. They can piggyback on the work of the Puerto Rico Institute of Statistics. The Institute’s independence, rigor and autonomy have given them an untarnished reputation with the citizens of Puerto Rico. The Institute has also led a crusade to achieve universal access to information and transparency on the Island, efforts the Institute has led many times without government cooperation.
Perhaps the Puerto Rico Institute Statistics can lead the difficult task of modernizing Puerto Rico’s system of national accounts. Maybe the governor can also look at the work of Foundation for Puerto Rico, who is leading an effort to create a national system of tourism statistics. Maybe other NGO’s can do the same with other industries. But the private sector must demand change first, and contribute to the effort. Government cannot and will not do it on their own.
