Cost-effectiveness can't Win

I've been griping about Cost-Effectiveness Reviews (CERs) for some time now. I just saw an E-Rate Central submisssion supporting the appeal of a CER that nicely lays out the problems with CERs.
The FCC has created the mess by requiring cost-effectiveness, but refusing to define what that means. Now there are three possibilities:

1. The cost-effectiveness "bright-line" standards are part of the secret tome of PIA procedures approved by the FCC,
2. USAC has developed standards, but the FCC has never seen them, or
3. USAC has no standards.

If the first is true, then the FCC is developing rules in secret. Well, actually, the rules would have been developed by USAC in secret and approved by the FCC in secret.

If the second is true, then USAC is exceeding its authority by creating program rules.

The third option would be the worst. It would mean that there is a person (or group of people) sitting in a room somewhere, reviewing applications, and sometimes deciding, "That's out of hand!" And in order to pass a CER, the applicant has to convince him/her/them that the request is not out of hand. With no objective criteria, the process is arbitrary in addition to being secret. It becomes a matter of giving the reviewer(s) a warm fuzzy. After the applicant has sent info off to USAC, hoping to hit on something that will sway the reviewer, the applicant and USAC just end up going back and forth with "Is too cost-effective!" "Is not!" "Is too!" until USAC issues a final "Is not!" in the FCDL and takes its ball home.

There are certainly criteria for triggering a CER; these reviews are not assigned randomly. It is clear to me that USAC takes the amount of the request, divides it by the number of students, and if the result is over a certain number, the applicant gets a CER. So there is a "bright line." The only question is whether this is part of the FCC-approved secret tome of PIA rules, or a rule that USAC just came up with on its own.

As ESPA pointed out in their recent filing, the result is that CERs disproportionately hammer small applicants, whose cost per student is higher than average.
The E-Rate Central filing supports an appeal from a small school serving disabled students. I have two clients like that, and I expect a CER every year. When you have 40 students and 200 staff members, the cost per student for everything from phones to network drops is unusually high.

As I've said before, the CER is the worst of all the program's audits/reviews, because the process is completely secret. No one even knows how many CERs are conducted, and how many applicants pass a CER.

Coal in my stocking

I happened upon a Christmas wish list I made at the end of 2006, and thought it would be fun to see how many of my wishes came true during 2007.

Came true:

1. New online Form 486
2. "Two-signature/two date" finally put to rest
3. The same PIA reviewer 2 years in a row [some of them, anyway]

Didn't come true:

1. COMADs only for Waste, Fraud, Abuse, not errors
2. No Form 486
3. All FCC appeals decided within 90 days
4. Publish the secret 700-page PIA manual
5. How about publishing 200 pages of it?
6. How about just telling us what triggers a Cost Effectiveness Review?
7. Don't take the Data Request Tool offline at night
8. At least let us know when the DRT will be offline
9. At the very least, let us know that the DRT is offline. As it stands now, you just get a message that no records were found.
10. A "copy" button on the 470 and 471 which lets you import all the info from last year's forms [For most small, low-discount applicants, those forms are identical year in and year out, and for all applicants, many elements are identical.]
11. Let applicants edit BEARs after they've been sent to service providers, so we can correct errors the service providers find
12. An online tech plan tool that applicants can use, set up so that it will generate plans that the SLD won't later decide are inadequate
13. A list describing the most outrageous requests. You know that PIA reviewers must pass around really hilarious requests. Let us all in on the joke.
14. Block 4 information in the Data Request Tool
15. Registration of consultants
16. The discount matrix topping out at 80% for equipment
17. Dark fiber eligible
18. An "about us" page for the SLD. There aren't that many people there, and I'd just love to see a brief resume for each of them.
19. Mel Blackwell in a red suit and beard, flying all over the country, handing out Priority Two funding to applicants with a 40% discount. Ho ho ho.

Oh, well.

Tea time

Oh, please. I just finished reading an article in eSchoolNews about the award of a contract for information systems administrator to SAIC, which also gets millions of dollars in E-Rate money as a service provider. Apparently several other service providers are incensed over a potential conflict of interest, since SAIC would have access to information in USAC's systems. It's a tempest in a teapot.

We all have access to most of the information in USAC's systems. The Data Retrieval Tool (DRT) lets you download information from the 471, 486 and 472/474. There is a similar tool for downloading info from the 470. What info would SAIC have access to that are not publicly available?

1. Contact info from the 471, 486 and 472
2. Info on individual invoices (472 or 474), rather than the annual total currently available through the DRT
3. Item 21 Attachments

None of that info is that valuable. Contact info is available through a little research, or from any number of marketing firms. Individual invoice amounts is of little consequence.

The service providers seemed to focus on the possibility that seeing the Item 21 Attachments would be an unfair advantage. Wrong. How would seeing the Item 21s be an advantage? If SAIC sees an applicant with a large contract with a competitor that they'd like to steal, rather than peak at the Item 21, they can just wait until the next 470 is posted and go after the business like everyone else. The information in the Item 21 is not going to be as good as the info they'll get from the applicant during the 470 process.

Of course, USAC could short-circuit this discussion by making Item 21 Attachments publicly available. Which would be a good move.

I know that USAC put out an RFP for this contract, and I trust they have picked the most cost-effective vendor, with price the primary factor. I don't want more of the program's funds spent on administrative costs just so some service providers can have a flimsy veil over the terms of their contracts, which, since we're talking about public schools and libraries, are public information. (And if SAIC, sub-contracting from Solix, is responsible for the significant improvement in USAC's online tools, then I'm overjoyed to see them take on more.)

Anyway, I like this set-up better than the original one, where a not-for-profit subsidiary of the telco's lobbying group gave the contracts to a for-profit subsidiary of the same lobbying group.

A final comment on a different tangent: As usual, most of the service providers demanded anonymity. It is shameful that this program still inspires so much fear that no one wants to publicly say anything negative. But secrecy breeds fear, and the level of secrecy in this program is astonishing.

SRIR hits the gym

Cue the Jaws music: it's the first Selective Review of the year. Good news, though: the Selective Review has slimmed down since last year. The SRIR has gone from 9 pages down to 5. Here's a sample of the new form.

What's gone?

The Item 25 Worksheet:
I'm so happy to see this go. It was that tedious worksheet where you had to give information on expenditures for hardware, training, etc. The one that really got my goat was the "Retrofitting" section. Most applicants are upgrading existing infrastructure, so they didn't need any asbestos abatement, so this number was usually zero, which set off red flags, even if you mentioned the earlier retrofitting in the Resource Plan and E-Rate Implementation Description.

Resource Plan and E-Rate Implementation Description:
This was that narrative where you described your implementation strategy. Not difficult, but tedious, and inevitably scads of people would want to review it, but would not actually add value.

Selective Reviews are still tedious, but both the items above make the process significantly less tedious. Now if they would just stop requesting all the correspondence between me and the applicant.

ESPA A-OK re: PIA

I'm starting to like this ESPA (E-Rate Services Providers Assocation). When the FCC approved the processing rules for PIA, I just posted a snarky blog entry. ESPA, on the other hand, formally submitted comments to the FCC.

ESPA's first suggestion is a bold one: all PIA procedures should be published for public comment every year, the same way the Eligible Services List is handled now. Now that would be cool. Of course, there are probably only 6 people in the country who would read all 700 pages, but it would be enough. Imagine being able to comment on PIA's procedures....

Wherefore art thou firewall?

I can't resist the opportunity to say "I told you so." Back when the Eligible Products Database was introduced, I asked at the train-the-trainer workshop, "What if a vendor certifies that a product is eligible, and USAC certifies that it's eligible, and later it's found not to be eligible? Who will end up paying?" The answer: "The applicant."

Today I saw an appeal where that's exactly what happened. Back when firewalls became eligible, security appliance vendors came out of the woodwork to claim their product was a firewall. I'm a network engineer who understands the eligibility rules, so my clients are safe, but what is a school business administrator to do when a vendor says that their product is eligible, and USAC agrees? They don't cover the difference between "stateful packet inspection" and "behavior-based threat detection" in professional development for educators or business administrators. And the definition in the Eligible Services List was nowhere near as cut-and-dry as saying "Firewall means Network Address Translation and/or stateful packet inspection, but not application-level proxying" (which was apparently the rule when firewalls were first added).

That's why the Eligible Products Database died. Not because it was cumbersome (it was), not because it was largely unknown (it was), but because USAC didn't stand behind it. And at least one applicant suffered.

FCC wants you

I can't remember ever seeing anything like this: the FCC is requesting comment on an E-Rate appeal. On the contribution side of the USF, it seems like the FCC requests comments on all sorts of requests from service providers, but I don't think they've made a request for comments on the appeal of a funding denial in the E-Rate program.

It's a case that's been kicking around for a while. USAC denied the applicants using "pattern analysis" of 470s, which the FCC tossed back in the Academy of Careers Order. So USAC sent out PAIR letters requesting explanation of the similarities. In the Caldwell Parish Order, the FCC decided that the eight applicants which did not admit service provider could not be denied funding, even though USAC did not find their explanations of the similarities convincing. However, one applicant admitted service provider assistance in the creation and posting of the 470, and the FCC denied their appeal.

So the service provider filed a petition for reconsideration, and now the FCC is asking for comments. I may not get around to filing formal comments, but here's what I think:

The whole thing smells a little funny, but the rules concerning service provider involvement in the competitive bidding process stink worse. As I've said before, it's almost Kafkaesque for the FCC to refer to something on a USAC PowerPoint slide from 2001 as if it were a program rule.

What's the answer to this mess? The FCC should get out of the competitive procurement biz. State laws have ample protections against waste, fraud and abuse, and the addition of the FCC's rules just makes things more complicated, not more competitive.

The latest secret rules

The FCC approved the Funding Year 2008 Program Integrity Assurance (PIA) FCC Form 471 Review Procedures on April 10th. The glass half-full view would be that now applications can be approved. But for me, it's more than half empty. First of all, why did applicants have to submit applications by February 6th if the procedures to process those applications had not been finalized by then? PIA has been reviewing applications, I know, but without an approved set of rules, they have been operating on presumptions. There should be a rule that applicants don't have to submit 471s until the rules for processing those 471s are final.

But what really galls me is that the procedures are entirely secret. I don't know how big the procedure book is this year, but for 2006-2007, it was 700 pages long. Which, as I've said before, means most of the rules for the E-Rate program are secret. This for a program where most of the funding requests are for transfers of less than $3,000 from one public entity to another. Do we really need such a large set of secret rules to administer that program?

And those are just the rules for processing 471s. I don't know if the procedures for processing 486es or BEARs are approved by the FCC, but there's got to be a couple of hundred pages there.