NAS, Raid, and Unraid

These days many of us have home networks with multiple computers with growing collections of media files. And with that comes the idea of adding storage. Of course, you can add larger hard drives to your computers, but there are advantages to having Network-Attached Storage (NAS). One advantage is that it lets you make use of a computer that might otherwise be too underpowered for desktop use. Storing and serving files is not computationally intensive, and you can often do quite well with a computer that has a Pentium3 processor and 512MB of RAM. Another advantage is that you can configure this to stream files without interference from your other computing tasks.

One option is to use FreeNAS, a specialized OS installation based on FreeBSD. This is not too difficult to install, and comes with a lot of features. Because it is expressly designed as a storage server, it comes with support for Samba, FTP, NFS, RSYNC (for backups), software Raid (0,1,5), and disk encryption.

Another option, which trades money for time and effort, is to buy an NAS box. There are many of these on the market, and recently Maximum PC Magazine did a review of some of the better ones. Most of these options, like the FreeNAS solution, make use of something called RAID, which stands for Redundant Array of Inexpensive Disks. The idea of RAID is to built in some redundancy by making one drive a parity drive. The idea is that you can have at least three drives (you could have more), and one of those drives is devoted to keeping parity information. Suppose you have a drive keeping even parity information. On Drive 1, you have a 1 in the first position, on Drive 2 you also have a 1 in the first position. If you add those up, it becomes a zero, and that is what you would store in the first position of the parity drive. It might look like this:

Position Position 1 Position 2 Position 3 Position 4
Drive 1 0 1 0 1
Drive 2 1 0 0 1
Even Parity Drive 1 1 0 0

Now, RAID with Parity has the advantage of letting you replace any one drive without losing your data. For example, if Drive 2 were to stop working all of a sudden, you could reconstruct the contents of that drive by looking at the remaining drives. You could look at Drive 1 and see a 0 in the first position, look at the parity drive and see a 1 in the first position, and the only way that could happen is if Drive 2 had a 1 in the first position. So you can put in new disk and rebuild Drive 2 from the information you have.

There are a couple of things that are not so good about this, though. The first is that while you have some protection with RAID, it is not supposed to be a backup solution, and you should never use it in place of backup. If two or more drives go out, you have lost all of your data, and there are lots of ways that can happen (fire, flood, lightning strike, theft, etc.). The other thing that limits RAID is that the capacity of the system is defined by the smallest drive you have. If you had a 500GB drive, a 400GB drive, and a 100GB drive in a RAID array, each drive can only contribute 100GB to the array. So one drive is used for parity, the other two for data, and you can store 200GB of data in an array that had 5 times that in available storage.

One company found a way around that. Data Robotics, Inc. created a device called the Drobo, which lets you use drives of various capacities without wasting all of that space. They must be moving parity data around on various drives in some way to do this, but I don’t know the details. I suspect they are proprietary in any case. (See what they say here.) I can say that everyone who has one loves it, as far as I can tell. But it is pricey. A four-bay chassis, with no drives installed, is at least $400.

But there is another solution. A company called Lime Tech has created a product called Unraid, that does many of the same things Drobo does, but as a software-only solution. A Basic package, which supports up to three drives, is free. Basic Plus supports up to 6 drives for $69, and Basic Pro supports up to 16 drives for $119. You provide the hardware, including drives. So this is not a free solution, but in many respects cheaper than Drobo, so it may be the right solution for you. You can see more discussion and a video of an installation from Category 5 TV.

Freedom and Responsibility

Linux is an example of what is called Free and Open Source Software (FOSS), but the word free may require just a little unpacking to get at what is meant here. Free can mean “does not require the payment of money”, and to Linux users this is often stated as “Free as in beer.” This can lead to the obvious question “Where do you get beer for free, anyway? I always have to pay for it at the store.” Still, the intent here is clear enough, we are defining “free” in economic terms. Well, that may be good in some ways, but it ought to lead you to wonder where all of this free software is coming from. In my e-mail sig I have the acronym “TANSTAAFL”, which stands for There Ain’t No Such Thing As A Free Lunch. (An aside: When I was in graduate school in the economics department of the University of Michigan, the grad students put out their own department newsletter and called it “The Free Lunch.” I think the idea was that we could plausibly deny that it existed.) I got this acronym for Robert A. Heinlein, who used it in his book The Moon Is A Harsh Mistress, but I suspect he got it from somewhere else.

The other meaning of free is “free from restrictions”, and in the Linux community this is condensed as “Free as in speech.” In the more advanced democracies, at least, there is some legal protection for people who wish to express their own ideas, even if those ideas are critical of the government or unpopular. This is not just free, but it is freedom. In terms of the software world, this is best summed up by the Free Software Foundation’s definition of Free Software:

* The freedom to run the program, for any purpose (freedom 0).
* The freedom to study how the program works, and change it to make it do what you wish (freedom 1). Access to the source code is a precondition for this.
* The freedom to redistribute copies so you can help your neighbor (freedom 2).
* The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

Now the thing that you may notice here is that this definition does not in any way suppose that the software needs to be free of charge. The FSF definition is completely compatible with charging money, though it removes some of the ways that companies might enforce the payment of money. The point is that FSF concern here is not economic, it is about the freedom of the individual software user to use the software in any way they wish. By this definition any software that comes with an End-User License Agreement (EULA) is not free. EULA’s exist expressly to place limitations on what you can do with your software.

I think this distinction is important in several ways. First of all, the people who write software do have to eat, they do have to pay their bills, and so on. We should never kid ourselves about this, and we do have a responsibility to support the people who write the software we depend on. Back when I first got into computers, I made it a point to pay for shareware if I ended up using it. In the FOSS world, software is offered free of charge, but many software authors do have a PayPal button for donations. Last year I hit that button for PortableApps, and got an e-mail in reply from the author which made me think that it is rare for people to make a donation. That is just wrong. If you use the software (and PortableApps is software you should be using if you want to run anything from a USB thumb drive), you should support it. Just recently I signed up to “Adopt a line of code” in Miro, the open-source video app. I use this app daily, so I have a responsibility to support it. Now there are always going to be some people who just cannot spare the money, and I don’t think anyone should stop feeding their children to pay for software, but many of us can hit the PayPal button for $5 or $10 without having our homes repossessed, and we should do so any time it is software that we will be using regularly.

With Linux distros, the situation is a little different. Most of them do not expect to get your money nor do they ask for it. I can download Ubuntu, OpenSUSE, Fedora, Debian, Mint, etc. and never see a PayPal button. That does not mean I have no responsibility to support the distro, it only means that the way I support the distro is a little different. When you are talking about a Linux distro, you should mentally replace the word “free” with the words “Community supported.” And when we use the word community, we mean you. If you are using the software, you are part of the community. There is a saying in the United States that “Freedom is never free,” and it applies in a slightly different way to Free Software. If we do not participate in the production of free software, that quantity and quality of that software will diminish over time, until we are at the mercy of EULAs for everything we do.

Most of us cannot participate by writing code (though there are never enough coders, so if you can code, by all means offer your services), so I am addressing this primarily to person is just using FOSS but has never gone beyond downloading some software and using it. Every project I know of, from major Linux distros to small utility applications, needs lots of different contributions that call upon a wide range of skills. These can include marketing, public relations and publicity, graphic design, writing documentation, managing forums, and so on. The list can get quite large really. Maybe you can help in one of these areas, and if so, they would be glad to have you. But you may think “Gosh, I don’t have any of those skills. I don’t think there is any way I can contribute.” And you would be wrong.

The one thing every FOSS user can do, and should do, is to submit bug reports. Anyone who uses software will eventually have it go pear-shaped on them, and when that happens you can file a bug report. Maybe you have a software configuration that is unusual, or combination of hardware that no one looked at before, or maybe you just did something that no one else ever thought to do. Whatever the cause, you just discovered a bug. You might even be the first to discover it, or maybe you will be added to other users and help the developers find the common factor that leads to patching the bug. The key point here is that bugs never get patched unless they are reported, and the more reports the developers get, the easier it is for them to find the solution and fix the bug.

I feel strongly about this, so my New Year’s resolution is to devote some of my time and space here to writing how-tos and promoting bug reporting. I may have more information about Ubuntu, since that is the ecosystem I live in, but I will try to get some material on other distros as well. And if anyone reading this has information about bug reporting in other distros (on in Ubuntu, I may have missed something), please send it along to me at zwilnik@zwilnik.com.

Project Timelord, or How Not To Build A Project

As the latest versions of the Ubuntu family came out at the end of October, you could hear certain grumblings, particularly as regards the KDE version, Kubuntu. This was not a new issue, really. For the last few years there has been controversy and accusations about how Canonical has supported Kubuntu. For instance, Aaron Seigo of the KDE project gave an interview to Computerworld (Feb. 1, 2008) where he seemed to slam Canonical:

Canonical did not communicate well about long-term support and therefore neglected 35 percent of their user base. A user base they routinely neglect, but at KDE we ignore a lot of this.

OK, this is not something you want to read if you are a Canonical representative, and Jono Bacon certainly did not like it. He put up a response in his blog (Feb. 1, 2008), which I will not quote here since there is no short piece that works. He basically lists all of the things that Canonical does to support Kubuntu and KDE. Now, since I have not personally met either gentleman, let me just say that they both seem like nice enough fellows, but there is definitely a difference of viewpoint here.

I know my own experience with Kubuntu upgrades has not been especially gratifying, and I have at times thought that just possibly less care went into Kubuntu releases than went into the main Gnome version. But I am only an observer. Still, I saw what looked like an increase in complaints around the time of the Karmic release. On Oct. 9, 2009, a blog post by Jonathan Thomas (Jontheechidna) titled Kubuntu, the Blue-headed Stepchild detailed what he called a pattern of neglect. His argument was that while Canonical certainly does devote resources to Kubuntu and KDE, they don’t show anything like the same care. His examples seemed to suggest that Canonical would make changes to the code base, test in the Gnome-based Ubuntu, then move it into production without caring that it might break KDE and Kubuntu. The fellow who claims to have been placed “forever in the grasps of hell” for coining the term “blue-headed stepchild”, Richard Johnson, has a blog post that mostly claims there has been no real difference in care or in result between the Gnome and KDE versions of Ubuntu. I have (barely) met Richard, but we are not exactly bosom buddies. Still, he seems like a nice enough fellow as well.

Well, I think we can now say with some justification that there was enough smoke to indicate at least some smouldering, if not a blazing fire. The Kubuntu developers have announced Project Timelord, an attempt to overhaul Kubuntu and take care of some of the problems that have occurred. Here is the introductory paragraph to the announcement:

Through intense self-reflection, it has come to the attention of several Kubuntu developers that Kubuntu is not currently reaching its full potential. Whether due to major architectural changes in the software stack, the usage of certain Ubuntu technologies or limited developer time, we have realized that deep changes must occur. In order to fix this situation will do all in our power to make sure Kubuntu stands the test of time.

OK, not exactly the most stirring call to arms, but it is a start. Unfortunately, while they welcome involvement from a wide range of people, their idea of how to do this is to publish a page that lists all of the general Kubuntu mailing lists. Now, I have not yet purchased Jono Bacon’s book on The Art of Community, but I would hope for his sake that the book has better methods of building a community than this. It looks to me like they are saying “If you are super-motivated and can figure out a way to get through to us, maybe we can find something for you to do.” I am disappointed here. I do actually know one fellow at Canonical (Jorge Castro, who not only seems like a nice enough fellow, but really is a very nice fellow and quite devoted to the Ubuntu community), and I asked him to find out more about Project Timelord, like maybe a contact person, and so far nothing. I have asked people in our Michigan Lo-Co if anyone knew about it, and no one did. There did seem to be some interest in getting involved should anyone ever figure out how. I even had one fellow from my LUG ask to be kept informed because he would like to get involved. Now, I am not a developer. for which you should all be extremely grateful. I am a Project Manager by profession, I have previously worked in politics (largely volunteer-driven) and I have successfully led several non-profit organizations that depended on volunteer activity for their survival, and I am fairly certain that this is not-how-you-do-it. Volunteers can accomplish a great deal, but they need to be motivated and led, and I don’t at this point see much evidence of either from Project Timelord. Maybe it is too early, and all of this will happen in due course. If it does, and you hear anything, would you let me know? I am still kinda interested in getting involved.

Linux Training Resources from IBM

A great resource for anyone who is interested in advancing a career in Linux is the IBM web site at http://www.ibm.com/developerworks/linux. This site has a lot of useful information, including some structured tutorials for LPI certification. To see the LPI tutorials, go to the Training link on the left (http://www.ibm.com/developerworks/views/linux/libraryview.jsp?type_by=Tutorials), and look for the “LPI Exam 301 Prep” items.

Welcome To Zwilnik.com

Welcome to my site, I hope you will find it useful. This is a site devoted to my explorations of Linux, the free, open-source operating system. Free is a word with various meanings in English. One meaning is free as in freedom. Software is “free” if you have the freedom to use it as you wish without restrictions. Linux is free in this sense. You can use it freely, there is no EULA (End-User License Agreement) that governs what you can do. The other common meaning of free is free of cost. Linux is free in this sense, though there can be some purely nominal costs involved in getting it. You can download the software and burn it to a CD or DVD, and then install the software from the disk you burned, all without any expense beyond having an Internet connection, a computer, and a blank disk to burn it onto.

My first exposure to Linux was on a server at the University where I worked. I had somewhat limited rights on this Red Hat machine, and everything was done on the command line. Then about 6 years ago (I guess) I heard about Mandrake, and how it was the distribution of choice for newbies. I downloaded it and installed it on a box, but never quite got comfortable with it. Then a couple of years ago I found Kubuntu, the KDE version of Ubuntu, and it is now my main environment. I know most people use Ubuntu with the Gnome desktop, but I never could get comfortable with Gnome. Maybe it was the Mandrake history, but KDE is just what clicks for me. I joined a LUG, the Washtenaw Linux Users Group (http://www.lugwash.org), and now I am the president of the group. This is not because I know more than others, I am maybe intermediate (at best), but I have good organizational skills and that is what they needed.

What I want to do with this site is to record my thoughts as I work on projects in Linux, maybe comment on issues in the world of Open Source, and also provide some instructional content I have developed (and will continue to develop). I hope you find some of this useful, and will join me in this exploration.

You can e-mail me at zwilnik@zwilnik.com