Can you build a Linux server for web, email, printing, and 4 terabytes of media serving purposes from commodity x86 parts, and come in under 20W? Absolutely!
If you run a server at home 24/7, and it uses, say, 150W, that’s 1.3 MWh (megawatt-hours) of electricity per year, and probably $130 or so of utility cost. You can do better!
I recently assembled a server with the following specs:
(note, some Amazon product links follow):
- Jetway JNC9C-550-LF Mobile Atom Mini-ITX motherboard
- Ituner Networks Pico-PSU 90W
[amzn]
- 60w (12v/5A) AC-DC Power Adapter, efficiency level V
- 2x Kingston 1GB 204-Pin DDR3 SO-DIMM DDR3 1066, p/n KVR1066D3S7
- Western Digital WD7500BPVT
750GB 5400 RPM 8MB Cache 2.5″ SATA drive [amzn]
- 2x Western Digital Green WD20EARS
2T 64MB Cache 3.5″ SATA drives [amzn]
- LIAN LI PC-Q11B case
- Red Hat Enterprise Linux 6
Out of the box, the Kill A Watt
#!/bin/bash
# Idle media disks
# (should set shortish spindown time too...)
hdparm -y /dev/sdb
hdparm -y /dev/sdc
# 100baseT saves power over GigE
ethtool -s eth0 autoneg off speed 100
ethtool -s eth1 autoneg off speed 100
# kill Wake On Lan
ethtool -s eth0 wol d
ethtool -s eth1 wol d
# set ALPM on SATA links
for FILE in /sys/class/scsi_host/host?/link_power_management_policy; do
echo min_power > $FILE
done
# Set PCI Express power management
echo powersave > /sys/module/pcie_aspm/parameters/policy
# Try to keep 1 core idle when possible
echo 2 > /sys/devices/system/cpu/sched_smt_power_savings
# Disable the nmi watchdog
echo 0 > /proc/sys/kernel/nmi_watchdog
Ok, idling the 4T of storage might be cheating a little, but in reality, they hold music and movies, and simply do not need to be spinning all the time. (Without these drives in the system, it comes in at about 16W.)
At 18W, that’s 158 kWh/year, or about $16. An 88% reduction! And, the utility savings would almost pay for the motherboard!
I don’t know if non-mobile Atoms would fare as well; my original test of an Atom motherboard was lackluster in part due to lack of power management, but then I didn’t have the efficient power supply either, so I’m uncertain. I’d imagine that the new AMD E-350 (Brazos) motherboards would do very well too, but I’ve not been able to test one out.
And, one more bit of full disclosure – sandeen.net isn’t running on this box yet, the migration isn’t complete. Still, quick tests of WordPress running on the platform don’t seem any slower than the current old AMD box (a 40W hog ;) ). And if LKML keeps spamassassin really busy, maybe it’ll inch up in power a bit. I’ll let you know when you’re reading this blog on the new machine, and what the average wattage draw is over a week or so.
So, for you geeks out there with home servers, how much power does it draw? Would you consider something like this for your next build?
I’ve had something similar in place for a NAS, sitting under the TV. No fans, small footprint, and low power consumption out of the box: http://www.tranquilpc.co.uk/T7.htm
I wonder if my UPS built-in load meter reads out low (I’ll have to put it on the kill a watt to check). But I have a reading around 40w with two servers running – one is a dual-core conroe 6600 (2.4 ghz) and the other is a new sandy bridge i5-2500k. Between them they have 7 spinning drives of 1.5 or 2T each and the sandy has two 40gb ssd’s. There’s 6gb of ram in the conroe and 8 gb in the sandy.
And my ups shows load at 75w with the 24″ panel turned on, and it bounces around 38-42w with the panel turned off. Of course neither box is doing anything right now and both systems are speedstepped down to 1.6 GHz. The power supplies are premium but nothing too green.
That does sound awfully low… But see what the kill-a-watt says!
I’ve been running a via epia based server for a few years, comes in at around 50 watts with the media drives spun down.
I’d recently been looking around for a mobile atom board with 4 sata ports, but had somehow missed this one. Thanks for the pointer!
What do you do for redundancy? I have to admit I’m running server class hardware (Dell Poweredge 2400) but only for the hardware RAID. I have recently replaced a larger number of smaller disks with lesser numbers of large disks just to cut power usage (and I cut by almost 50% according to kill-a-watt) but I’ve lost data before and have found redundancy to be more effective in that regard to backups. My luck with backups is horrible. If I loose my wife’s email and file archive again…
I will actually have two 2.5″ drives in a mirror for the OS and mail/web data. The 2x 2T drives are for media, and they used to be in a mirror, but I think I will split that and rsync nightly to keep them in sync. For music & movies, a short downtime from one disk going bad is better than a filesystem problem eating both at once, I think.
Rather than the Jetway which wants ATX power and needs the Pico-PSU, might I suggest Intel’s D945GSEJT, which takes 12V DC directly? I suspect it also requires less power.
It has only 2 SATA connectors, so you’d have to put the root filesystem on one of the 2TB drives, but still.
I’m surprised there aren’t more of these boards with direct 12VDC inputs, it seems quite logical at this point. The PicoPSUs are still odd beasts, but even “small” ATX powersupplies are grossly oversized at this point…
But yes, the limited SATA connectors would be an issue for my usecase, I really wanted 4. I suppose you could always add a PCIE sata card too, but I wonder how many drives the motherboard can supply power to?
Disclaimer: I work with several ARM vendors as part of linaro.org
I always feel a bit frustrated when I read articles about low power servers @ 20W and there is no mention of ARM-based devices. It reminds me of our failing as part of the ARM community.
Here are a few random links from googling. [1] talks about the more recent boards (still under 5W) and [2] about a seriously underpowered one by today’s standards that still seem to be enough for some people.
One thing I’ll readily admit is disk performance or lack there of with these boards since they only take USB hard drives. If that isn’t a constraint, then you should definitely look at these.
[1] http://jeffbastian.blogspot.com/2011/04/my-pandaboard-has-arrived-it-was-only.html
[2] http://nslu2watt.dyndns.org/
Hi Amit – I know ARM would be lower power, and I would love to run on ARM. Really! I have a sheevaplug, and it’s fantastic at ~6W, but it can only do so much without more connectivity.
You’re right – I can’t add storage to the ARM solutions I see, other than via external USB disks, and the accompanying performance issues, tangles of cables and less-efficient wall-warts.
Why are there no mini-ITX formfactor ARM boards with 4 SATA connectors? Get one of those out there for under $200, and you’d have a winner, I think.
> Why are there no mini-ITX formfactor ARM boards with 4 >SATA connectors?
4 SATA connectors! You do know how to inflict pain. :)
We finally have one board (imx53 quick start) that has one SATA connector. Find details here: http://www.linaro.org/low-cost-development-boards
> 4 SATA connectors! You do know how to inflict pain. :)
Dumb, but honest question: Why is that so painful? By the time you have one, isn’t at least the second “almost free?” :)
nb: that board is not available now, and when it is, it’ll be $394. (Edit: oops, bare board is $199; but, still sold out) Sorry, that just doesn’t work for my server needs … But I want to believe! I really do hope ARM becomes useful in this space.
We’ve been asking for SATA for years, hence the pain. And after so many years we’ve finally managed to get 1 SATA connector.
But ARM is starting to look at the server space now, so things should change rapidly. (Fingers crossed)
When you get a 2- or 4- SATA board, send me one and I’ll blog about it…*grin*
Well here is an ARM board with two SATA II slots and Giga-e. http://www.embeddedarm.com/products/board-detail.php?tab=options&product=TS-7800
The problem is that it is a little expensive, it has limited RAM, and has a lot of IO that I NAS just does not need.
It would work but no cigar yet.
Thanks! I actually have one of those boards from an old embedded appliance, but I haven’t really played with it…
-Eric
BTW that board does offer an 802.11n option but if you added a USB to ethernet dongle so you have a second serial port and their wifi adaptor or a USB wifi adaptor you could make this into a SAN, Firewall, Router, WiFi router as we.
With all the serial ports that it offers you could also add a Zig Bee radio for some remote data collection projects.
The problem I have with the ARM SoC based “small PCs” is that a lot of the time the SATA ports are connected internally to the USB… at least that’s the case with the Dreamplug. Other than that I’m very happy with my Dreamplug.
Turns out the UPS is massively off. The sandy bridge idles around 65-68 watts with 16gb ram.
You can even bring it to 16W with the N455 version (single core atom)
http://www.newegg.com/Product/Product.aspx?Item=N82E16813153189R
I hope for a file server, we don’t need multi cores
What did you use for a case? I’m having a hard time finding a mini-itx case that holds more than one drive.
It’s a LIAN LI PC-Q11B – I’ve updated the parts list, sorry about that!
Pingback: Eternal Vigilance is the Price of Efficiency, part 1 | Eric's Blog
18Watt? Wow, great job Eric ;)
Pingback: What’s up with the Intel Atom? | Eric's Blog
Pingback: Quora
Great article!
Are you using the case fan with this board? And does it ever completely turn off with the bios smart fan configuration? I’m thinking of buying this board, but I wouldn’t want to hear any fan noise if the system is completely idle, while at the same time it seems that a case fan is required when the system is under high load.
The Lian Li case I got did come with a large fan on the front. To be honest I have not paid attention to the fan; I’ll try to look at the bios settings soon and reply again with more info. Fiddling with the sensors infrastructure might also be useful there.
The board is a bit pricey still; I’d be interested to know how an AMD Brazos board would fare power-wise; they are available at decent prices now.
FWIW f71882fg.ko (F71882FG Hardware Monitoring Driver) handles the fan sensors on the board; as it sits idle right now the main fan is at 675RPM. I think the smart fan control is on in the BIOS.
And at what temperature does it idle? I wonder whether it would be possible to configure fan control so that the fan is off when idling.
By the way, I read some problems about the GMA3150 running hotter on linux than on windows. Do you have any problems in that regard?
I’ve seen a Fujitsu D3003-s1 over here which seems an interesting alternative, 9W TDP, but still pricier and only 2 sata ports…
Thanks for the quick replies!
Here’s how my sensors output looks after being up for a while; I don’t know which sensor is on which component though. I’m not doing anything at all with the video card (headless server) so I don’t expect there to be any heat issues with the GMA3150.
# sensorsf71869fg-isa-0290
Adapter: ISA adapter
in0: +1.64 V
in1: +0.94 V (max = +0.00 V)
in2: +1.05 V
in3: +1.37 V
in4: +1.03 V
in5: +1.36 V
in6: +1.09 V
in7: +1.64 V
in8: +1.59 V
fan1: 0 RPM ALARM
fan2: 683 RPM
fan3: 0 RPM ALARM
temp1: +30.0°C (high = +85.0°C, hyst = +81.0°C)
(crit = +100.0°C, hyst = +96.0°C) sensor = transistor
temp2: +34.0°C (high = +85.0°C, hyst = +81.0°C)
(crit = +100.0°C, hyst = +96.0°C) sensor = disabled
temp3: +36.0°C (high = +70.0°C, hyst = +68.0°C)
(crit = +85.0°C, hyst = +83.0°C) sensor = disabled
coretemp-isa-0000
Adapter: ISA adapter
Core 0: +14.0°C (high = +80.0°C, crit = +100.0°C)
Core 1: +14.0°C (high = +80.0°C, crit = +100.0°C)
Impressively low! Thanks for posting this.
I’d be also using it with mythtv, so I’ll go dig a bit deeper into this gma3150 issue.
Pingback: Jetway JNC9C-550-LF resume after power fail | Eric's Blog
What about a QNAP TS-412 ?
http://www.qnap.com
http://www.qnap.com/en/index.php?lang=en&sn=822&c=351&sc=514&t=522&n=3415
I am trying to find a good low consumption motherboard, but I always end up choosing the QNAP products for home/office. It has everything you need, it’s Linux based and I am impressed with all the features it has.
The cost of a mini-itx, plus a processor, plus a power adapter, plus a case remains high.
Pingback: Spinning down a WD20EARS | Eric's Blog
Pingback: Watts on? | Eric's Blog
I’ve been building a home nas/media server using spare parts I had laying around. My motherboard is a Gigabyte P965-DQ6, 4GB ram, 3x2TB WD Red Drives, and one 500gb drive as the boot/swap drive. The lowest power cpu I could find for the board was a single core Celeron-L 440 @ 35watts. Running the system headless, the system is pulling 120w at idle. I’m going to try a few things, but my guess is that the Intel chipset on the MB is not power friendly. The only saving grace is that the MB has a ton of sata ports. How much of a difference will a new power supply make. I have a big 450 watt supply in the machine, but would investing in a smaller wattage supply actually decrease my consumption? I always though that the PS would only supply the wattage needed. Thanks for the article! – Sean
A smaller, 80Plus power supply could make a difference, depending on what your current PS is rated. Yes, a power supply will only supply what’s needed, but with varying efficiency. I’ve used Antec Earthwatts 380(?) Watt power supplies with good results, it does drop the draw at the wall by a bit – and found it for around $30 on sale IIRC. The 80Plus website has a listing of all certified power supplies and their efficiency curves; you could look for one that does a good job down around 120W. A 200W 80Plus power supply might be the ticket, if 200W handles all the spinup load etc.
Do you have the Red drives idling when they’re not in use? That might save 10W or so. Disabling unused peripherals (audio, unused sata controllers, etc) might help, but not sure about that.