Audit finds problems with performance measures

According to a recent audit, the city’s current performance measurement system could use some work.

The audit, which covered the period of time between 2012 and August 2014, was conducted to determine whether the city’s current performance measure system provides “timely, accurate, and relevant information.” The Office of the City Auditor found some problems.

The October 2014 report from the auditor’s office and discussion at City Council’s Audit and Finance Committee suggested that the system could use some refining to be more effective and that the city should adopt a citywide policy for performance measures.

Specifically, the audit found that of the seven departments that were selected for detailed testing, five failed to consistently provide relevant and reliable information.

For example, a measure that evaluated the number of school zone signals that had received preventive maintenance wasn’t expressed as a percentage and didn’t show how close the Transportation Department was to meeting that specific goal. In addition, the auditor’s report notes that metric isn’t verifiable, as it is based on estimates that could be over- or understated.

Currently, the city’s system for measuring performance evaluates about 1,270 performance measures across municipal departments. Those measurements are part of a larger program called “Managing for Results,” which essentially allows the city’s Budget Office to keep track of how departments are doing and plan for the future. The measurements are an important part of the budget process to help keep track of whether the city’s many departments are meeting their goals.

City of Austin Chief Financial Officer Elaine Hart said she appreciated the audit and the impartial look at performance measures. Hart concurred with the audit’s findings and said that although she thought the framework currently in place was good, she would continue to look to expand training and improve the performance measurements within that framework.

Council Member Bill Spelman said the audit brought up a larger issue for him, which is that the accountability for performance measures for Council is “rather spotty.”

“We get to it at budget, we all focus on the departments we see the biggest opportunities and biggest problems with, but it’s nobody’s responsibility to look at a specific section of the budget or a particular department,” said Spelman.

He continued: “Our committee structure is not aligned with the city’s management structure, so there are a lot of departments, for example, that don’t have committees associated with them, and as a result there are a lot of departments that don’t get looked at nearly as carefully as some others.”

Spelman suggested that one of the things that could be done is to align the city’s boards and commissions structure to allow “every Council member to be associated in some way with a department” through appointments. He said that having Council eyes on departments during the whole year, not just during the budget sessions, might help the process.

“I think it would help,” said Hart.

Council is currently in the process of determining how Austin’s boards and commissions will change when the city switches to the new 11-member Council in January.

Spelman also offered a rough verbal sketch of a hierarchical system of performance measures, which Hart again agreed might be useful. He then suggested the performance measures could use some culling.

“My guess is that our performance indicators have grown in about the same way as our land development code has, which is as soon as there is a problem, someone comes up with a performance measure to measure the problem, and we never lose performance measures, we just add them,” said Spelman. “So we have four volumes of performance measures, but not everybody pay attention to all of them, and some of them are just a legacy from a problem that we had 20 years ago.”

Hart confirmed this suspicion somewhat, by recalling that when the city implemented the Managing for Results framework, four measures were required for each goal.

“Citywide, I think we were at over 3,500 measures,” said Hart. “We’re down to 1,200 now, but that’s still far too many for us to really get our hands on and focus on across the many departments that we have.”

“We do report an awful lot of data,” Hart continued. “And sometimes it’s just overkill, frankly.”

Hart said she understood that the audit was suggesting more training for directors and within departments, which she was willing to do. She also said that based on what she heard in the audit, she thought her office could incorporate additional instructions in performance measure training.

“I think we need to refresh and refocus,” said Hart.

In a letter written in response to the audit, City Manager Marc Ott said that departments had been instructed to ensure that goals are “aligned and measurable” and that “defined measures are in place that track performance.”

Mayor Pro Tem Sheryl Cole pointed out that her office requested the audit in order to help with the budget process.

“I always want to know if the dollars did what they were supposed to do,” said Cole. “And now I see we do have some concerns with that.”