Best Practices to Avoid Misreporting University Data to External Sources
Any school that has had to correct erroneous data that affected, or could have affected, its placement in the rankings knows that ensuring the accuracy of data submitted to outside sources is critical. Regardless of whether the misreporting was intentional or the product of an oversight, the act can have serious repercussions, not the least of which is damage to a school's public image. Fortunately, there are simple and cost-effective actions that an institution can take to avoid releasing inaccurate data. The following are best practices that should be considered to avoid misreporting university data to external sources and surveys.
Maintain a Central Depository of the Data Used to Calculate Externally Reported Metrics by Utilizing a Standard Student Record System
Externally reported data should be generated from an institution's standard student record system, which undoubtedly includes various data security checks and balances. Using the same record system across all schools, including graduate and professional schools, allows for centralized review and contemporaneous tracking of the information used in data reporting. All actions affecting data in the system, such as who loads data or makes changes to data entries, will be tracked so that any changes may be reviewed and audited in the course of responding to external survey requests. Exclusive access to the system by username and password ensures that any additions, deletions, or alterations to data can be attributed to a specific user.
Consider a Centralized Process for Reporting Data in Response to External Survey Requests
To the extent an institution has a decentralized process for reporting data to external survey requests, it should consider a centralized process or implement centralized procedures in order to verify the accuracy and consistency of data. The mandated involvement of a last-stop, quality-control office ensures that any outgoing data will be reviewed a final time before it is disseminated externally. In addition to its role as the final gatekeeper for the thoroughness and accuracy of data, such an office would serve as the first and last point of communication for any outside entity soliciting data from the institution.
Questions from data source offices (i.e., the office of admission, the office of financial aid, etc.) should be directed to the centralized reporting office, thereby promoting consistency in the feedback and instruction to all data source offices. Another task for the centralized reporting office should be to conduct frequent and random audits of the data it receives. If errors are detected, the data should be returned to the data source office for review and revisions, and the data review process should start anew. Finally, the centralized office should maintain a data warehouse where, among other data, copies of all externally reported data are preserved.
Various Persons at the Data Source Office Should Review the Data for Accuracy
Data should be critically reviewed by employees within a data source office. Any assumptions made in calculating the data should be put in writing and examined by a senior decision-maker within the data source office before the data is transferred to a central reporting office. The onus to provide accurate data is with the employees of the data source office since they are the ones who are most familiar with the data and its collection.
Despite the importance of accurate data analysis, the task is sometimes relegated to employees with low seniority due to the fact that data calculation can be time-consuming and tedious. Such employees rarely have the level of statistical training or background that would compensate for their lack of experience. In cases where a junior employee takes the first pass at data analysis, the data should be reviewed by additional persons in the data source office, including at least one person of mid to high seniority. But in all instances, data should be reviewed by more than one person, as a practical internal control at the ground level.
Segregate Duties to Avoid Creating Conflicts of Interest and to Create Independent Verification and Accountability in Reporting
Persons whose performance is measured, even in part, by the results of any operation (e.g., the ability to recruit a certain number of students while maintaining existing admission standards) should not also be responsible for accounting for the results of those operations and should not be the final and ultimate reporting source. Place the responsibility for accounting for and reporting the results of operations with persons who are not evaluated by, and have no direct interest in, the results of those operations.
The unencumbered reviewer should review the reports he or she receives at an appropriate level of detail prior to their transmission to a central reporting office or to third parties. Each person involved in the review process should have a clear understanding of his/her responsibilities. The institution should develop a review checklist, which evidences the completion of each required review procedure. This checklist should be completed prior to transmission of any report to third parties. A copy of the completed checklist should be maintained in the centralized data warehouse with a copy of the transmitted report.
Create Clear, Unambiguous Policies and Related Processes for All Significant Operations
A clear, unambiguous written policy will guide personnel as they prepare responses to external surveys. Clear policies also protect against lost institutional knowledge in the event of personnel changes. The policies not only need to be explicit, but all personnel involved in the preparation of responses to external surveys must be aware of the policies and how they can be accessed, and that the institution is dedicated to following the processes set forth in the policies. Ambiguities in policies to be applied in making significant reporting decisions can lead to the rationalization of inappropriate choices. Developing and communicating clear-cut policies about recordkeeping and survey answering will lead to better quality decision-making and accountability.
Document Judgment Calls Being Made Contemporaneously with Rationale for the Decision
In the event an external request for numerical data is ambiguous and could reasonably and ethically be answered by more than one method of calculation, document any judgment calls relating to how an answer was determined. Record precisely the means by which the answer was calculated, why such an approach was taken, and who was responsible for making such decisions.
To the extent the data provided to the central reporting office relies on any assumptions made by the data source office, the central reporting office should be made aware of such assumptions and given the opportunity to accept or reject them before verifying the accuracy and consistency of the data for external distribution.
Consider the Implementation of a Code of Business Ethics and Conduct Policy and Hotline to Foster and Promote a Culture of Integrity
Assuming such a policy is not already in place, consider the implementation of a Code of Business Ethics and Conduct policy that informs employees of their professional duty to do their job honestly and ethically. In conjunction with this policy, staff should be trained annually and sign their acknowledgement and adherence to the policy. Training should emphasize each employee's duty not only to act in accordance with the policy, but to report other employees' violative conduct, even if a superior is involved. To implement this part of the policy, an institution may choose to utilize a telephone hotline to which anonymous calls may be made. Employees should not only be obliged to report conduct that they know is violative of the policy, but they should also be encouraged to report conduct over which they have concerns. The policy and hotline information should be highly visible, posted on the intranet, and in well-trafficked employee areas.
If Incorrect Data Is Reported Externally, Correct the Erroneously Reported/Disseminated Data and Conduct an Assessment of Risks Related to Other Data
Identify and correct any erroneously reported or otherwise released data. Where possible and after appropriate levels of review, contact the party to whom the data was reported and provide it with the corrected data. Once it is determined that an institution has reported erroneous data, it is recommended that the institution conduct a risk assessment as to the accuracy of other reported data. A key component in the risk assessment is determining what aspect of the internal controls fell short.