Chris's Wiki :: blog/sysadmin/WhyLocalComputeServers Commentshttps://utcc.utoronto.ca/~cks/space/blog/sysadmin/WhyLocalComputeServers?atomcommentsDWiki2013-04-01T20:17:18ZRecent comments in Chris's Wiki :: blog/sysadmin/WhyLocalComputeServers.By Chris Siebenmann on /blog/sysadmin/WhyLocalComputeServerstag:CSpace:blog/sysadmin/WhyLocalComputeServers:758c7be3e2ea06b193a4bba1ec936c2e8170a4b2Chris Siebenmann<div class="wikitext"><p>I want to clarify something to be clear: when I said that the incremental
cost for graduate students to use a compute server is zero, I meant that
literally. No money changes hands or has to. Of course the actual
operating cost is non-zero, but that's another issue that is tied up
in many complex and highly political layers.</p>
<p>It may surprise or horrify you but we don't actually have a capital
equipment budget as such. Almost all of our compute servers have in
effect been free because they were part of dedicated, specific grants
(as opposed to purchased with general purpose money we could have
spent on anything). The other costs of operation are either negligible
(we have a well honed automation framework that makes managing one more
standard Ubuntu machine basically free) or diffuse (the power and
cooling costs). If we had enough compute servers to form a cluster
or the like these costs might become visible, but we don't and we're
unlikely to get that many.</p>
<p>(Researchers who get grants for computing clusters have so far bought
and operated them themselves.)</p>
<p>But with all of that said, I'm not arguing (at least in this entry)
about the relative economics if you take a whole-costs picture; I'm
willing to posit that cloud computing could cost no more than the
whole operating costs. But I think it would be very difficult on a
political level to put together a cloud computing environment where
compute servers were still 'free' to grad students in that neither
grad students nor their professors had to even think about how much
the computation was costing them.</p>
<p>(Things might be different if we were providing all of this computation
through an actual ongoing budget, but that is not the model we operate
under and I don't know if that will ever change.)</p>
</div>2013-04-01T20:17:18ZFrom 128.101.135.18 on /blog/sysadmin/WhyLocalComputeServerstag:CSpace:blog/sysadmin/WhyLocalComputeServers:bef7f41865c0cccaeffd2bd4145d1c23642efbe5From 128.101.135.18<div class="wikitext"><p>Each grant that comes in has an unavoidable overhead figure. (the amount varies between institutions.) This pays for nebulous things that are “shared” resources like garbage collection, electricity, and sometimes network bandwidth.</p>
<p>While the cost of running the system is not Zero, it is often a way to get benefit from the overhead.</p>
<p>Also, some granting agencies will not pay for ongoing costs, but will pay for large on-time purchases.</p>
<p>It is very much a perversion based on Accounting.</p>
</div>2013-04-01T19:55:37ZFrom 140.247.173.45 on /blog/sysadmin/WhyLocalComputeServerstag:CSpace:blog/sysadmin/WhyLocalComputeServers:7f059656cdf7fc08141daa55fe1702f38fb5a04eFrom 140.247.173.45<div class="wikitext"><p>You said:</p>
<p><em>The great advantage that local compute servers have is that the incremental cost for a grad student to use them is zero.</em></p>
<p>I'm not sure that you're looking at this the right way. You are currently covering the cost of compute time for faculty and graduate students through the capital equipment budget (to purchase hardware), the facilities budget (to pay for power and cooling), and the personnel budget (to pay for staff to maintain the hardware). None of these are zero-cost items, and yet faculty and students are able to take advantage of these services without needing to find money in their <em>own</em> budgets for each use.</p>
<p>You're certainly not simply "buying them and running them for free". It's true that sometimes the cost of facilities comes from a different part of the school than the hardware purchase, but the cost to the school is still substantial. And the purchase cost of the hardware, distributed over its useful lifetime, is also significant. We once did the calculation and found that the cost of hardware and facilities for our compute resources over a three-year period was roughly equivalent to purchasing equivalent computing power as reserved AWS instances. This did not factor in the cost of storage or data transfer.</p>
<p>If you were to cover the cost of access to a cloud-hosted compute service, then from the perspective of faculty and students there would still be no incremental cost for their use. And in theory, your expenditures for hardware and facilities would be reduced, ideally by an amount that could cover the cost of the cloud resource utilization, <em>and</em> you would save the staff time that would otherwise be used for maintaining the local compute resources.</p>
</div>2013-04-01T14:18:14ZFrom 91.13.155.245 on /blog/sysadmin/WhyLocalComputeServerstag:CSpace:blog/sysadmin/WhyLocalComputeServers:f633d57b2b8b1a072d8f0e33486fcca98f646828From 91.13.155.245<div class="wikitext"><p>A result of this is that high-powered machines idle (or, as it has been the case here, spin a few cores on 100% due to a hung KDE session...) for weeks because noone knows they are there. ;)</p>
<p>Electricity is cheaper than cloud computing, tho.</p>
</div>2013-04-01T13:34:15Z