Audit Reports

Objective

The City Carrier Cost System (CCCS) is a statistical study of mail delivered on city carrier routes. For each selected route, a data collection technician selects a sample of mail to be delivered on the scheduled test date. The technician records the mail class, product type, and other characteristics of each manually sampled mailpiece directly into a portable microcomputer using the Computerized On-Site Data Entry System (CODES) data collection software. The Postal Service conducted 9,257 and 8,694 CCCS tests in fiscal years (FY) 2019 and 2020, respectively.

Our objective was to assess the reliability of CCCS data and evaluate CCCS sampling methodologies to identify opportunities for improved efficiencies.

Findings

While technicians followed most CCCS sampling policies and procedures, there are opportunities to improve the consistency of data collection activities and enhance the reliability of CCCS data. During our observations of seven CCCS tests, we found that technicians did not always follow policies that indicated numeric intervals for mail selection and/or did not confirm the delivery status of parcels and mail that needed a signature.

These issues occurred due to:

• Technicians did not notice the random start number generated by CODES to identify the first sampled mailpiece. In addition, a software issue caused CODES to improperly change the random start number during some tests.

• The absence of system controls to discourage the use of selection intervals greater than one for parcels.

• Technicians’ lack of awareness of the importance of receiving carrier confirmation of the delivery status for each sampled parcel and mailpiece that needed a signature.

Management developed sampling procedures to ensure data is collected in a way that does not introduce error or bias. Technicians must follow these procedures to ensure that data collection is consistent regardless of who performs the tests. Improper or inconsistent sampling poses a data integrity risk and may not result in reliable data for attributing costs to mail products and services.

We also found there are opportunities for the Postal Service to provide delivery personnel with more comprehensive guidance to ensure they fully understand how to properly gather and document information on mail coming from the customer to the delivery unit (collection mail) for CCCS test routes. Two of the seven evening supervisors who gathered collection mail information for the CCCS tests we observed did not have a full understanding of how to correctly gather and/or record information for the test route. This was because the evening supervisors had little to no prior experience gathering this information and/or felt that the technician did not provide clear, detailed instructions for completing this step of the CCCS sampling process.

Without clear and comprehensive instructions on how delivery personnel should complete the required form, supervisors may not always complete the form accurately. This could negatively impact the reliability of percentages used to attribute collection mail costs to mail products and services.

There are also opportunities for the Postal Service to enhance CCCS sampling efficiencies by leveraging real-time census data. In the FY 2020 Annual Compliance Report, the Postal Service reported that it used scan data from the Product Tracking and Reporting (PTR) system as replacement sample data for 39 CCCS-Special Purpose Route tests cancelled in FY 2020, Quarter 3, due to the COVID-19 pandemic. Although the approved CCCS-Special Purpose Route sampling methodology did not include the use of PTR scan data, management believed it was necessary to leverage this data on an emergency basis to ensure statistically reliable sample data and mitigate the pandemic’s impact on in-person CCCS testing.

Management stated they believe there is an opportunity to use PTR scan data more heavily in CCCS-Special Purpose Route sampling beyond FY 2020. However, to do this, they must assess some operational challenges and propose a methodology change to the PRC for approval.

If proposed and approved, expanding the use of PTR data in CCCS-Special Purpose Route sampling would cut costs associated with manual sampling and improve sampling capabilities. Since management is currently assessing the operational challenges prior to proposing a methodology change to the PRC, we are not making a recommendation on this matter at this time.

Recommendations

We recommended management:

• Modify the CODES software to alert technicians of the correct random start number to use throughout the test and correct the software issue that causes the random start number to change.

• Require technicians to enter a justification when selecting a selection interval greater than one for parcel sampling.

• Reiterate to technicians the importance of communicating with the city carrier to confirm the delivery status of each sampled parcel and whether the city carrier plans to deliver all parcels and accountable mail on the test date. In addition, evaluate whether delivery scan data can be used to determine if any sampled parcels should be excluded from the test data.

• Develop online training and/or instructional materials that provide comprehensive guidance on how to properly gather and document collection mail information. In addition, consider developing a web-based form with guided prompts and instructions to replace manual documentation.