Search This Blog

Friday, March 22, 2019

The Midnight Save & Share

Here's an EPC "feature" that has stuck around.  I have previously described the "Midnight Save & Share."  We never studied it closely enough to discover exactly when or why it happened, but if you leave a draft form in your Tasks list (waiting for info or whatever), at some point EPC may decide that you should share your work, and at midnight will Save & Share the form, meaning that it drops into the Tasks list of every user authorized to see the form.  It doesn't seem to happen as much as it did back in 2016 and 2017, but it still happens.

One mystifying change: the Midnight Save & Share does not generate the email to all users that EPC sends when you click the "Save & Share" button:
That's a problem and an improvement. 

A problem because the person who had the form in their Tasks list isn't notified that the form has appeared in other people's Tasks lists.  All authorized users now have that form in their Tasks list, and any of them can now "Accept" that form and move it only to their Tasks list, causing it to disappear from the original owner's Tasks list.

An improvement because that email creates confusion.  No one who gets that email needs to "Create a Form 471," since the form has already been created; someone needs to continue a Form 471.  But some of the people who get the email don't need to do anything.  In our case, every time we Save & Share, we can expect an email or call from at least one employee of the client, asking what they should do.  Or worse, they just Accept the form and Submit it.

At On-Tech, we run into that problem all the time.  See, we like to have at least 2 sets of eyeballs view every form before we submit.  However, only one person at a time can have access to a draft file, and the only way to transfer a draft from from my Tasks list to yours is Save & Share.  So we have to Save & Share every form at least once.

Instead of "Save & Share," how about a "Transfer" option?  Or maybe when you click "Save & Share," you get to pick who you share with?

One improvement that we have seen with "Save & Share."  It used to mean "Save & Give Away"; when you shared a form, it vanished from your Tasks list.  So you couldn't see the form unless another user Accepted it and then Save-&-Shared it, so it would reappear in your Tasks list.

I just have trouble believing that building a portal from the ground up would have been harder than it's been trying to shoehorn the E-Rate process into Appian's business process management platform.  Even now, I think we should start from scratch.

Thursday, March 07, 2019

Does the "O" in "O'Rielly" stand for "overbuilding"?

Oh, boy, Commissioner O'Rielly is back on the overbuilding warpath.  In a nutshell, Commissioner O'Rielly feels that the E-Rate should not be funding any special construction costs for wide-area networks where fiber is already available.  He can't see why the E-Rate should be paying to lay new fiber if there is fiber already laid in the area, especially if the existing fiber was also subsidized with federal funds.

Before I get to the letter, I'll give my response to that idea.  Mostly, I agree.  The E-Rate should not pay for new networks where a network exists UNLESS -- and this is where I differ with the Commissioner -- it's cheaper.  I don't think the E-Rate program should be paying extra to avoid special construction.  I think that's wasteful.  Current rules only allow special construction if the applicant can demonstrate that it's cheaper than the other alternatives.  So overbuilding only happens when it's the cheapest option available.

I don't think of having E-Rate pay for a fiber network where one exists as "overbuilding."  I think of it as "encouraging competition."  But there's no way the E-Rate should pay extra to encourage competition.  The E-Rate should only pay for a second network if it's cheaper than using the existing network.

On to the specifics in the letter.

First, the most important questions: did the Commissioner capitalize the "R" in "E-Rate"?  Yes, consistently.  Huzzah!  Go big "R"!

I don't have the specifics on the case that the Commissioner refers to in the letter, but it strikes me as exactly the kind of project the FCC envisioned when they allowed dark fiber network: a huge rural area with spotty fiber access.

Now I'll take a shot at responding to the Commissioner's questions:
1. Does USAC understand the E-Rate rules to permit funding for special construction projects, whether self-provisioned networks or networks owned by a commercial provider, that would duplicate, in whole or in part, fiber networks that have been built using federal funds? 
Only if it's cheaper.
2. Does USAC understand the E-Rate rules to permit a consortium to receive funding for the construction of a WAN to provide Internet access to the entire consortium, even where existing fiber-based providers are already capable of serving individual consortium members? 
Only if it's cheaper.  And oh yeah, the Commission has wanted to "encourage schools and libraries to aggregate their demand with others to create a consortium with sufficient demand to attract competitors and thereby negotiate lower rates or at least secure efficiencies, particularly in lower density regions" since the Universal Service Order that started it all (see paragraph 476).  I don't believe that consortia lower prices, and I agree that telecommunications purchases that cover a large area often make it impossible for many companies to bid, but I don't make the rules.
3. Since the 2014 E-Rate Orders, how many applicants have requested E-Rate funding for special construction of consortium-wide WANs, and how much funding was requested by these applicants? 
State-wide networks had better batten down the hatches, because I think they're about to get hit with Hurricane Michael.
4. How many requests for consortium-wide WAN projects have received funding commitments, and what is the total amount of funding committed for such requests? 
The number won't be big: only 362 consortia filed a Form 470 this year (out of 16,976 applicants filing).
a. How many of those WAN projects would result in overbuilding another provider’s network, in whole or in part? 
OK, what do you mean by "overbuild"?  If a service provider leases existing dark fiber to use as part of the network, does that count as overbuilding?  For example, if Service Provider A fiber running between 2 towns, and Service Provider B leases a few pair from Service Provider B and then lays fiber from A's network to the schools in those towns, and connects that WANlet to a bigger WAN, did B "overbuild" A?  What if Service Provider A has existing fiber running past 2 of the 5 schools in a district, would a new WAN from Service Provider B that connects all 5 buildings count as "overbuilding"?
b. How many of those WAN projects would result in overbuilding, in whole or in part, providers that receive funds from the High Cost or Rural Health Care programs? 
And how many of those High Cost funding recipients replied to all Forms 470 in their geographic area, as required by the Second Modernization Order (see paragraph 67)? Commissioner O'Rielly didn't ask; in his Dissenting Statement to the Second Modernization Order, he opined that "a new requirement that Connect America Fund recipients participate in the E-rate competitive bidding process ... could have unintended negative consequences for the CAF program."
5. How many consortium-wide WAN projects have been denied on cost-effectiveness grounds? 
Interesting question.  How would USAC determine cost-effectiveness?  It's not like there's a list price for fiber WANs.  They do scrutinize cost in special construction applications much more closely  than other applications, but it's rare that you can do a truly apples-to-apples comparison of the cost to lay fiber, because it depends on so many factors.
6. How many consortium-wide WAN projects are currently awaiting a funding decision by USAC? 
Another interesting question.  I'd be sweating if I had a consortium WAN application sitting in PIA right now.
7. To the extent that USAC has approved funding requests for the construction of consortium-wide WANs that partially or fully overbuild existing fiber providers, and considers such projects, whether self-provisioned or owned by a commercial provider, to be eligible for funding under current E-Rate rules, has USAC alerted the FCC of the overbuilding risk created by such projects? Has USAC alerted the FCC of an apparent gap in current program rules that permits USAC approval of such projects?
"Overbuilding risk"?  What's that?  The risk that service providers with existing fiber will try to gouge applicants by charging more to use their existing fiber than it costs to lay new fiber?  Me, I have no problem with "an apparent gap in current program rules" that allows applicants to save money and lowers costs for the E-Rate fund.  But Commissioner O'Rielly seems to think that the rules need to be changed so that applicants and the E-Rate can pay more for WANs just to prevent any competition to carriers with existing fiber.

And here's my question for Commissioner O'Rielly: Before writing this letter, did you talk to any stakeholders other than service providers with existing fiber?

Sunday, March 03, 2019

Buh bye

I have plenty of complaints about the E-Rate, but you know what the *biggest* problem is?

The 35% drop in the number of schools and libraries applying for E-Rate.

Where'd I get that number?  Well, Funds for Learning counted the number of applicants filing Forms 470 in FY 2014 at 26,059.  In FY 2019 they counted  16,976.

The count of 470s is not a perfect count of participation, since applicants under multi-year contracts or will only a cable modem don't have to post a 470, but it's a good indicator of participation.

I compared this year's numbers to an analysis I did back in 2011 using FY 2009 numbers.  I looked at how many applicants could have applied vs how many did.  Now I've added in 2019 numbers.  (Call me lazy, but I didn't update the number of districts, schools and libraries; I just don't think the numbers have changed that much.)
Applied 2009 Applied 2019 Change
Total Number Participation Number Participation Number Participation
School district 13809 12833 93% 9627 70% -3206 -25%
Single School 33740 7142 21% 5072 15% -2070 -29%
Library 9225 3672 40% 1915 21% -1757 -48%
Consortium n/a 439 n/a 362 n/a -77 -18%
24086 16976

Wow, almost half the libraries that were applying in 2009 have left the program.  I would have expected a bigger drop among individual schools.

The drop in school districts is bigger than I would have expected.  We've certainly seen clients leave the program because the meager E-Rate reimbursement isn't worth the hassle of applying, but I wouldn't have expected a drop of 25%.

Tuesday, February 12, 2019

And I C2 far, and I C2 much

So the FCC's Wireline Competition Bureau has release the long-awaited Report on Category 2 Funding.  We've all been on tenterhooks, waiting to see if the budgets would continue or we'd go back to 2-in-5 (OK, no one really thought we'd go back to 2-in-5), whether the budgets would be increased or decreased, etc.

This is a Report, not an Order, so nothing is written in stone (but it's not written in pencil, either).  Will we see an NPRM before the FCC makes an Order?

But I'm not going to wait to run my mouth.

On the main point, yes, budgets are better than the old 2-in-5 nonsense.  I actually wonder if 2-in-5 might spread funding to all applicants now that the funding cap is $3.9 billion instead of $2.3 billion.  But 2-in-5 created some twisted incentives for applicants, so good riddance.

OK, I'm done agreeing with this report.  (I know I've been offline for a while, so long-time readers probably read the above and wondered if I'd gone soft.  Fear not; I'm still a curmudgeon.)

I'll jump right to the Conclusion, which has a few conclusions, and make my ill-considered response:

  1. "the category two budget approach is generally sufficient."  No, it isn't.  More on that below.
  2. "the category two budget approach is a clear improvement over the two-in-five rules approach."  True, but not for the reasons given.  The fund boost from $2.3 billion to $3.9 billion (and tossing voice out of the program) would probably have fixed the 2-in-5 problems mentioned in the report.
  3. "The category two budget approach may not be sufficient, however, for schools and libraries at the funding floor, as well as libraries outside of highly-concentrated urban areas."  Also true, but again not for the reasons you give.  The C2 budget is too small for everyone, not just small locations.  The exceptionally low participation rate among those locations is more due to low need; at that size, you buy cable Internet access and it comes with a wireless access point/router.  What else are you going to buy?  If you did need a switch or cabling or another access point, you bought all that years ago, and you don't need to upgrade to 802.11ac or 10Gb uplinks or anything, so you haven't spent anything in the last 5 years.
And the Recommendations:
  1. "retain the category two budget approach."  Agreed.  It's a pretty good approach, and it's not worth disrupting the program to try something new.
  2. "consider some targeted changes to the budgets moving forward such as an increase in the funding floor."  Agreed.  Wait, that implies all other applicants don't need an increase. No on that.
  3. "..." What about that too-close-to-free 85% discount level?  Not even a look at whether that should be changed?
  4. "..." Hey, what happened to the request for district-wide (as opposed to location-specific) budgets?  Not even a mention?  C'mon, that was the most urgently needed change.  In a couple of footnotes, we can see that the FCC got requests to ditch location-specific budgets from:
    1. Council of the Great City Schools
    2. Florida E-Rate Team
    3. SECA
    4. CoSN
    5. CSM Consulting
                They left out comments from a couple of major players that I can think of:
    1. SHLB: "allow school districts and libraries some flexibility to distribute their allotted funding among their locations as best meets their needs."
    2. E-MPA® "recommends that the FCC discontinue the 'per entity' budget in favor of a 'per district' budget"
I don't think I need to talk about the reasons we need district-wide budgets.  Everyone agrees we need them.  I think E-MPA® describes the reasons well in their NPRM comments.

My big beef with the report: "We also find that the category two budget approach appears to be sufficient for most schools and libraries."  Oh, hell no.

I'll get to criticism of the data in the report and the reasoning that led to the conclusion, but first, let's take a step back.  Why were the budgets put in place?  To keep demand within the fund's means.  Well, the demand estimate for 2018 was $2.77 billion, compared to a funding cap of $4.06 billion.  Including the $1.2 billion in carryover unused funds, we could have covered $5.26 billion in demand.  So we came in $2.49 billion under the program's funding cap.  Total C2 demand was $745 million.  Adding in the $2.49 billion, we find that we could have covered a demand of $3.23 billion for C2, more than 4 times the demand.  So instead of a $150-in-5 budget, it looks like we could easily afford a $600-in-5 budget.

If the purpose of the C2 budget is to keep demand within the program funding cap, let's make the budget $600 every 5 years.

OK, now I'll criticize the data. I'll just look at the data in the Conclusion.
  1. "no requests have been denied due to insufficient funding."  True on the surface, but how many applications have had their funding reduced?  And even more important, how many applicants only put part of their need on an application?  In the old days, at least the FCC could see what applicants thought they needed.  Now, they only see what fits in the C2 budget; applicants pay full price for all the other equipment they need.  That amounts to a partial denial of funding.  Add another layer to the ridiculous secrecy in the E-Rate program: hidden denials.
  2. "86% of schools have [requested C2 funding] in at least one year..."  That seemed really high, so I checked further up in the report and found what they actually measured: 86% of schools which were part of a C1 request were also part of a C2 request.  (It seems they used locations, not billed entities.) Actual participation is last time I checked, the number of school districts participating in the E-Rate at all had dropped to under 85%, and the number of private schools was down around 17%.  No question that many more schools are getting C2 funding than in the past. As you may remember, I like pictures, so the graph on page 7 works for me.  Let's not pretend that the increase in the percentage of applicants getting C2 funding is all due to the increase in the numerator (the number of schools getting funding commitments); it's also due to a decrease in the denominator (the number of schools applying for E-Rate at all).
  3. "Almost half of schools (48.6%) and most libraries (84.6%) have used less than half of their allocated five-year budget, and a supermajority of schools (73.2%) and libraries (94.4%) will use less than 90% of their budgets."  Those numbers would be useful if we'd already reached the end of the budget cycle.  Since none of those applicants have had 5 funding years in which to spend their 5-year budget, we shouldn't draw any conclusions from those numbers.  Some schools are going into year 2 of their 5-year cycle, so it's not surprising they've used less than half of their funding.  Wait a minute! Wait, on page 7 it says: "Approximately 42,000 schools have used more than 75% of their category two budgets, and 30,000 have used more than 90%. An additional 16,000 schools have used between 50 and 75% of their budgets."  Uh, that means 17% (not 48.6%) of schools have used less than half their budgets, and 71.7% (not 73.2%) have used less than 90% of their budgets (and at most, those schools have completed 80% of the funding cycle).  Where did that 48.6% number come from?  I can't find any more about it in the report.
  4. "the data from the first five-year category two budget cycle..."  Much more accurate to say, "the data from the first four years of the first five-year budget cycle."  0% of applicants have completed the cycle, and some applicants are just starting their cycle.
On to criticizing the reasoning.

"Low total budgets may be deterring participation by these entities because the administrative costs of participating outweigh the benefits of doing so."  If that's true, maybe we should look at how we can increase the budget to try to bring back the 33% of applicants who have left the program since FY 2014.  Let's increase the budget for everybody until we get back to the participation we had in 2014: 26,059 applicants filed Form 470 in FY2014, but only 17,446 applicants filed Form 470 in FY2018.  Take a look at the library data in the report.  Increasing the per-square-foot budget for urban libraries from $3.20 to $5 increase participation from 58% to 84%.  Bigger budgets will increase participation, and we all want that, don't we?

"We also find that the category two budget approach appears to be sufficient for most schools and libraries."  Yes, the approach is sufficient, but it seems like they're implying that the amount is sufficient.  That's based on some unexplained percentages which don't agree with numbers elsewhere in the report.  And those percentages indicate what applicants use, not what they need.  There are lots of reasons applicants don't use all the funding they need.  Here are a few off the top of my head:
  1. PARCC and SBAC arrived in FY2014, and that meant a major network upgrade for a lot of schools.  So when C2 funding became available in FY2015, a lot of districts had a pretty new network.
  2. Since funding is per-location, a district is likely to have locations where they aren't using 100% of the funding (early childhood), and locations where they need much more than 100% of funding.
  3. Having to go through the E-Rate process creates extra work, requires understanding of hundreds of pages of words, forces applicants to sign illegal contracts, locks in pricing and configuration 4-18 months before services are delivered, and restricts applicant flexibility in searching for the lowest price.  So unless you have to, you don't want to do it.
  4. Not all purchases fit the E-Rate window.  Actually, none do.  Technology budgets aren't set by February, so districts don't know what they'll be buying in the coming year, so the Form 470 is an educated guess.  But the reality is worse: most C2 applications are not approved by June 30, and vendors that do work for schools are booked for the summer by then, so for major projects, you really need to apply 15 months before the project starts.  It is extremely bad practice to contract for a tech purchase more than a year before you'll install. 
  5. The individual who certifies a form is taking considerable personal risk.  School and library employees don't get paid enough to take that kind of risk.
  6. Since funding is per-location and per-piece-of-equipment, trying to figure out what to change in order to reduce a funding request is like a hellish word problem, so most applicants will apply for less than the maximum.  And since you don't know pricing when you file the Form 470, the amount could be much less than the maximum.
  7. Indexing the budget decreases the percentage of funding used every year .  If an applicant used 100% of their C2 funding in FY2015 and didn't apply for another 5 years, by FY2019, 6.4% of their C2 budget would be unused (assuming a 2.6% inflation adjustment for this year).
  8. The vanishing E-Rate subsidy for voice services and equipment has hurt technology budgets, so districts don't have the money to pay their share.
Here's my anecdote on the sufficiency of $150 per student.  I've been involved in projects to build new high schools.  So those schools need the following eligible items: cabling (including racks, patch panels, etc.), switches (including uplink and stacking modules), UPSes, access points, wireless controller, a router, and the installation and licensing for all that equipment.  What do they get for $150/student?  The access points.  And maybe part of the switches.  Dropped off in boxes.  There is $0 for cabling, most switches, a controller, a router and any installation.

But you know what shows up in the data?  Those schools have unused funding, because they couldn't hit the cap exactly, and indexing has provided more funding for those schools.

Of course, existing schools don't need so much cabling (although I'm seeing more clients bump up against the limitations of Cat5e and OM2).  But $150/student is not enough to replace the APs and switches, and 6 years is a mighty long replacement/upgrade cycle on that equipment.

If the FCC really wanted to know what the need was, they'd ask applicants how much they spend on eligible equipment each year.  Or CoSN could do a survey.  (Maybe they already do.)  I'm confident they'd find spending on eligible internal connections is more than $150/student.

But let's get back to the biggest failure of the report: no recommendation concerning per-location budgets.  Everyone hates the per-location nonsense.  Switching to per-district budgets should be the top priority in C2 budget improvements.

Monday, November 13, 2017

Just sayin'

Ooh, look at item 4 on the agenda for this Thursday's FCC meeting: " Accelerating Wireline Broadband Deployment by Removing Barriers to Infrastructure Investment." 

Hey, here's my suggestion: stop the kafkaesque and ever-shifting review process for self-provisioned fiber.  Millions of dollars of broadband deployment has been held up by these wasteful reviews.

Tuesday, October 10, 2017

None shall pass

I'm sitting in USAC training at the moment, and I just heard EPC called a "portal."    Took me back to those rosy days when I dreamed of a portal that made my life easier.  After two years of toiling in EPC, the use of "portal" made me snort (really quietly).  "Portal"?  More like "portcullis."

Friday, June 30, 2017

Hey, where you going with that money?

The punches just keep coming.  In April, Chairman Pai let USAC have it. Last month, Commissioner O'Rielly said, "I believe that all options should be on the table, including putting USAC’s functions out for contract...."  In response to yesterday's GAO report that found problems with the Lifeline (née Low Income) Program of the USF, Commissioner O'Rielly said: "I have little the ability of USAC to stem the tide of problems. More significant reform is needed, including completely rethinking USAC.”  He sure has it in for USAC.

Of course, I only wanted to look at the summary of the GAO report (basically scanning for anything E-Rate-related).  Let's see: blah blah blah blah
"In March 2017, FCC developed a preliminary plan to move the USF to the Treasury."
[record scratch]

Say what now?!  Where is this plan?  To the full GAO report!

Page 23: "According to correspondence received from the FCC Chairman’s Senior Legal Counsel, as of March 2017, FCC has decided to move the funds to the Treasury....  FCC’s Office of the Managing Director prepared a preliminary project plan for moving the USF to the Treasury with the goal of completing the transfer in approximately 1 year."

Oh, crap!  As I said back in 2005, this is step one to killing the E-Rate.  Get it into the Treasury so that it will be easy to "so underfund it that it goes away."

Do you think I'm being paranoid?  OK, maybe, but that doesn't mean I'm wrong.  Back on page 23 of that GAO report:
"...FCC explained that having the funds in the Treasury could allow USF payments to be used to offset other federal debts...."

It seems to me they're saying that the USF could be used to pay down the national debt.  At this moment, the federal debt is $19.96 trillion, which means the $10 billion that USF collects annually is 0.056% of the debt.  The government is paying something like 0.8% interest on t-bills, so we'd need 15 times the total amount of the USF just to pay the interest on the debt.

This is bad.  When was the FCC going to tell us about this?

Is this a silver lining?
Page 24: " May 2017, while reviewing a draft of this report, a senior FCC official informed us that FCC experienced some challenges associated with moving the funds to the Treasury, such as coordinating across the various entities involved, which raised some questions as to when and perhaps whether the funds would be moved."

Stay tuned.

Meanwhile, back to the sidelong glance the FCC seems to be giving USAC.  Page 24: "beginning in November 2016, FCC sought to amend the contract between USAC and the bank to enable the bank to act on FCC instructions independent of USAC in the event USAC ceases to be the administrator."