http://www.freesoftwaremagazine.com/node/1683/issue_pdf by Imad Humayun

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Issue 15 By Tony Mobily Issue 15 of Free Software Magazine brings you another fantastic spread of articles to feast on. Nathan Sanders will have you "Stitching seamless panoramas with Hugin", Alan Berg has some "Fun with free software astronomy" and our favorite tank simulator "Bzflags". Sreejith Nair teaches will teach you to "Control machines with your machine" and Ryan Cartwright will show you how to plot your family tree with GRAMPS in "Roots access". And we have two much-anticipated sequels: Yousef Ourabi concludes his "Secure email servers from scratch with FreeBSD 6" article, and Graham Oakes finishes off "Creating a managed website". And, don't forget... lots more! Source URL: http://www.freesoftwaremagazine.com/issues/issue_015

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Have I already lost my bet? A gold coin sometimes is worth a million By Tony Mobily I am angry. It’s not a good state to be in, and it’s definitely not healthy. However, today I just can’t help it. The main problem is that I have a bet going on, and I feel I am going to lose it. My bet is that by 2010, more than 50% of the world’s laptop sales will have GNU/Linux preinstalled, rather than Windows. Until a little while ago, I was feeling optimistic. However, my optimism fell after I decided that I needed a new laptop. You’ve probably guessed already: I want a laptop with Ubuntu Linux preinstalled, and I’m having a great deal of trouble finding one. Yes, I know, there are some honourable attempts out there. For example, there are small laptop shops that make sure that “everything works” in GNU/Linux, and give you a working machine—no fuss. But what is still missing, in nearly 2007, is a big brand (see: Dell, Toshiba, Sony, etc.) marketing, selling, pushing their GNU/Linux laptops. Yes, Lenovo laptops often work well with GNU/Linux, but to me that’s not good enough—in fact, far from it for a start they don’t sell them with GNU/Linux preinstalled. Other brands have started releasing binary-only Linux drivers. Again, this is not good enough; in fact, many would argue that it’s a step towards the wrong direction… After much complaining, I guess I should come up with a possible solution. Unfortunately, there isn’t “a solution” as such. In order for me to win this bet (I haven’t lost all hope just yet), a number of things must happen. First of all, as manufacturers will only listen to high-demand, users—end users—must request that machines come with GNU/Linux preinstalled. Some manufacturers will realise that they don’t use Linux-friendly hardware, and will deal with the problem (by creating a new laptop line, or by changing the type of hardware they use). Also, chip makers must be lobbied so that they release the specs for their cards. Keeping specs secret is a complete waste of time; unfortunately, a few crucial companies out there haven’t understood this just yet. There needs to be a set of hardware (webcams, cameras, MP3 players and so on) which come with a “GNU/Linux Compatible” logo, so that users don’t worry about not being able to buy hardware at common shops. We all need to remember that asking people to recompile their kernels, or compile a driver, is not feasible for 95% of the users out there. Laptop vendors need to offer their full support to GNU/Linux. This is the scary part for them: they’ve started with Windows, have grown up with it, and, to a lot of them, supporting Windows has required extensive and regular retraining. However, vendors need to be reassured that supporting GNU/Linux will require work, but is definitely achievable. Finally, third-party software installation needs to be easy. I am talking about all those pieces of software which come with dictionaries, encyclopaedias, local governments, etc. These things will never be found in a repository, because they simply don’t belong there. For this to happen, a mixture of lobbying and customer demand will do the trick, but it will take time. I only bet a single dollar—a single gold coin that is worth less now than it was seven years ago when I made my bet. But, when I do end up paying that dollar to Andrea (who is waiting and grinning), it won’t be about

Have I already lost my bet?

Issue 15 me being a dollar out of pocket: it will be about millions of computer users who, because of absurd and illegal and unethically monopolistic behaviours, will still be given a virus-prone, unstable, expensive, privacy-insensitive systemâ&#x20AC;&#x201D;and wonâ&#x20AC;&#x2122;t even know it. (Maybe I should have said 2015, instead of 2010?)

Biography Tony Mobily (/user/2" title="View user profile.): Tony is the founder and the Editor In Chief of Free Software Magazine

Copyright information This article is made available under the "Attribution-NonCommercial-NoDerivs" Creative Commons License 3.0 available from http://creativecommons.org/licenses/by-nc-nd/3.0/. Source URL: http://www.freesoftwaremagazine.com/articles/editorial_15

A gold coin sometimes is worth a million

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Having your cake and eating it A beginner’s guide to understanding free software By Ryan Cartwright What is free software? Should you care and if so, why and what does it have to do with cakes and my mother?

My mother and computers “Write a complete beginner’s guide to free software”, they said. “It has to be something that your mother would want to get excited about”, they said. The problem is my mother neither knows nor cares what software is. The closest she gets to using a computer is when she uses the timer on her oven. I still need a typical reader in mind when I write this though so, for the purposes of this article, my mother not only uses a computer, she gets annoyed when it doesn’t perform as expected and she knows that there is no such thing as an “Any” key. She is the target audience, she may use computers at home or at work or both but she has not given much thought to the software she uses or the licence under which she received it. For the purposes of this article, my mother uses a computer and gets annoyed when it doesn’t perform as expected In this article I’ll be explaining a little about what free software is and why you should care. I expect and hope you’ll have some questions but for the most part I hope you’ll not get confused or lost. Note: I am not a licencing expert and I’ll be using some simplifications. If you are knowledgeable about the subject and feel the need then please comment but remember this is a beginners guide—simplification is the name of the game here.

What is free software? Wherever you encounter free software, you’ll find the term free is being used in relation to freedom and not cost. You may also encounter the phrase “Free as in speech, not as in beer” which reinforces this idea. Whilst reading this article I suggest you forget about cost altogether as it might be a distraction. The fact is that some computer users don’t like to pay for their software (but don’t slip it under their jackets in the store) and some don’t mind paying. When I say “free software” I am talking about software with freedom attached. I’ll explain what that freedom is later but for now remember that cost is not important. Confusingly some free software will cost you money. Much of it doesn’t, but what binds all free software (and much of the surrounding “community”) together is not cash flow—it’s freedom. Whilst reading this article I suggest you forget about cost altogether as it might be a distraction

Cakes

What is free software?

Issue 15

What do you want to bake today? Let’s talk about cakes for a minute. Some cakes cost money and some don’t. Similarly some come with the recipe and some don’t. What I’m interested in here is not how much the cake costs to buy or make but what you are allowed to do with the recipe and the cake once you have it. Imagine this scenario: • You receive or buy some cakes from someone and they give you the recipe as well. • They allow you to use the recipe to make your own cakes at home. • They allow you to change the recipe to leave out the nuts you are allergic to. • They allow you to redistribute (even sell) both their original and your nut-less version of the cake. • They insist that if you do redistribute you give others the same option. • They insist you do not pass the original off as your own work. [1] It may seem obvious but what they’ve given you (aside from cakes) is freedom: freedom to use, change and redistribute the cakes and the recipe. You might be thinking that people rarely give you recipes with such conditions attached but those conditions are often inherent in the gift. If you gave a recipe to someone and you were happy for them to pass it on, wouldn’t you be angry if they sold it to others but did not allow their customers to pass it on? You gave (or sold) those cakes and/or recipes with certain freedoms attached and you would expect those freedoms to be respected when the cakes and/or recipe is passed on. In short, you want the cakes and recipe to remain as free as when you distributed them. If you gave a cake recipe to someone … wouldn’t you be angry if they sold it but did not allow their customers to pass it on? An additional point to note is that when you have access to the recipe, you know what is in the cake—handy if you have that nut allergy or just don’t like them. If you don’t trust the baker then access to the recipe also means you can make your own cake and be assured of exactly what went in it. Substitute software for cake and source code [2] for recipe and you can see what the free in “free software” means. I bet you thought the cake analogy wouldn’t work! The advantages of having source code are similar to those for having recipes—except we’re not talking about allergies here. If you can see the source code, you can check for things you don’t like. You can personally (or have someone else) check the code to ensure it doesn’t contain any untoward bugs, back doors [3] or any other malicious code. You can also check the software’s security and integrity. Furthermore, you can also learn from the source code by seeing what does what and how. Having a recipe means not only can you make that cake but also that you can use the methods and techniques from that recipe in your own ones. If you like the way someone has implemented something in their software you can use it in yours—in accordance with the free licence. Try that with proprietary software and you might be needing a lawyer and a lot of money.

Cakes

Issue 15

Proprietary cakes can restrict your freedom and you don’t often know until it’s too late

What’s in a licence? The freedoms we’re talking about are within the licence under which you receive the software. All software has a licence. Public domain software says “Do what you want with it—we don’t care”. Proprietary software often has a licence which says “Use it but don’t pass it on, don’t change it and don’t try to find out what it’s made of”. Free software says “Use it, change it, redistribute it but don’t stop others doing the same”.

Why should you care about the licence? At first glance you might not care about the licence. When you buy cakes you don’t stop to think if you will be allowed to give them away—you just do it and when you buy software you often don’t think about the licence restrictions. This is where the analogy breaks down a little. When you buy a cake you own the cake—even the baker cannot take it back, it is legally yours. However, when you buy software, you do not own the software and the software company can take it back or at least stop you using it. That’s because you buy a licence to use the software and not the software itself. It’s like buying permission to eat the cake but never owning the cake itself. Of course, you have to imagine this cake never disappears even if you eat it (work with me here, I told you the analogy breaks down at this point).

Is free software licencing risky? Some people think free licencing leaves you open to abuse. After all, what is to stop someone abusing the freedom you have given them? One reaction to this potential abuse of trust is to stop giving away the recipe and start policing what people do with the cakes as well, but that brings all sorts of additional costs. You have to stop people analysing the cake to determine the ingredients, you have to stop people reproducing the cake and passing it off as their own. You have to start with the assumption that every customer wants to abuse your trust. All this costs money and time and these costs have to be recovered—so guess where they are recovered from? That’s right, the customer, you and me. Now you can see how much simpler (and better) life is with free cakes Now you can see how simple (and better) life is with free cakes! The baker need not worry about whether her customers will give her recipe to their neighbours in fact she actively encourages them to do so! For the customer there’s no worrying that sharing the cakes with their colleagues will land anyone in trouble. They are freely allowed to pass the cake to others. The baker may well receive feedback and additions to the recipe, including why certain ingredients are a bad idea in certain markets. If an individual customer wants the cake baked specifically to their needs they can change the recipe or have another baker change it for them. They can even (but don’t have to) redistribute the new version as long as they adhere to the licence and pass on the freedom.

What’s in a licence?

Issue 15 The recipe can now reach markets that a single baker could not possibly reach and each person who uses it is referenced back to the original baker’s work. Even if you’re the type who just likes to buy cakes off the shelf and doesn’t care about the recipe you should be able to see the advantage of free licencing. Again substitute software for cakes and you can see why we should all care more about our software licences. With cake recipes we expect this kind of freedom—why not with software? An important point to note is that (like cake recipes) free software licences do not insist you redistribute any changes you have made. If you make changes to the source code for your own (or your company’s) purposes, then you do not have to distribute them. But, if you do distribute the changes, then you must do so within the terms of the licence.

Does producing free cakes mean you can’t make money? At first glance, it would seem that giving away your recipe is a fast way to go out of business. Why would people buy from you when they can make their own or buy from a competitor? Again this is where the analogy breaks down a little but there are still a number of ways to make money through free licencing. For a start, not everyone wants to bake their own cakes—some of us are happy to buy them. The days of buying software in boxes at the store may be numbered but people are quite happy paying over the net. Additionally, some customers will need help with understanding or modifying the recipe; and where better to get help than from the original baker? Perhaps you could include free software in your hardware product—there are a number of broadband modem/routers on the market which do this. Some companies use free software in their own products and have been successful in making money (Google anyone?)—but if you want to do this pay attention to the licence. Other companies have two models: one for non-profits and individuals and one for businesses. The former coming at zero cost and the latter costing money but coming with additional services and/or features that business customers require. Finally, don’t forget you are not having to pay for all that additional security we mentioned above. You may not be passing that cost onto the customer but that will mean your product is better value. A free licence—on a good quality product—will assist in getting your software known as people will start recommending it to each other. As said, some of those will pay for it. In the end—as with anything—people will pay if the product is worth it. Nobody will buy a cake that tastes awful and makes you sick regardless of the licence!

The importance of free software

How often do you click “I agree” without reading the licence? I can’t understate how important freedom is in a software licence. That’s because like every other freedom, once you give it up it’s very nigh-on impossible to get it back. You may not have paid attention to the licence before but ignoring it means software companies will own your computer. You’ll own the hardware but they will (and often do) decide what you can and can’t do with it, and when you need to replace it. I should have mentioned that my mother is a diabetic. She (and anyone baking for her) has to substitute the sugar in most cake recipes. My father doesn’t bake and yet enjoys the same cakes baked for my

The importance of free software

Issue 15 motherâ&#x20AC;&#x201D;cakes baked from recipes my family passes around freely. Free software does not dictate that you have to be a developer to enjoy its benefits and developers are not forced to operate outside of copyright law or good business sense to use it. Now, anyone got any cake?

Notes and acknowledgements [1] Actually, not passing work off as your own is part of copyright law but I included it to show that free software licences are not outside of copyright they actually permit greater freedom within it. [2] The source code is like a recipe from which you make software. It is used should you wish to build the software yourself (rather than use a pre-built version). It also means you (or someone on your behalf) can examine in detail what the software does. [3] A back-door is where the person who writes the software deliberately leaves a way to access the software remotely. In this way, they can potentially manipulate your computer, read your data or monitor your computer usage, and usually without your consent. The chocolate cake image used within figures 1 and 3 is adapted from an original work released under a Creative Commons Acknowledgement licence. The original can be found at http://flickr.com/photos/ktlindsay/.

Biography Ryan Cartwright (/user/8833" title="View user profile.): Ryan Cartwright is IT Manager for Contact a Family (http://www.cafamily.org.uk), a UK National charity for families with disabled children where they make significant use of free software (http://www.cafamily.org.uk/oss). He is also a free software advocate and you might find him on the GLLUG (http://gllug.org.uk) mailing list.

Copyright information This article is made available under the "Attribution-NonCommercial-Sharealike" Creative Commons License 3.0 available from http://creativecommons.org/licenses/by-nc-sa/3.0/. Source URL: http://www.freesoftwaremagazine.com/articles/beginners_guide_understanding_free_software

Notes and acknowledgements

Issue 15

Notes and acknowledgements

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Free software and politics in Yankeetown How free software tools helped to turn the tide against big development in a small Florida town By Andrew Seely The picturesque village of Yankeetown on Florida’s Nature Coast has been the recent target of a large time-sharing resort condominium development proposal. Several townsfolk looked into the development and discovered what appeared to be corrupt practices in the town government. A loosely organized group of citizens decided to coordinate and share information via a web presence. Beyond any expectations, the picture albums, message boards, and mailing lists they used have been the catalyst for gaining state-wide attention and have led to direct intervention from the Governor’s office. This article describes the technical implementation and evolution of this grassroots movement, showing how a variety of basic free software tools were assembled to create a tool for political change.

Welcome to Yankeetown Yankeetown is a small town, somewhat isolated, lovely. Some say it’s perched like a bird, others say it’s nestled like a kitten, along Florida’s Nature Coast. It’s one of the last, if not the last, small Florida coastal towns without any significant commercial development. A handful of small businesses, a history of commercial fishing, the lingering memory of a movie Elvis made here, and a slowed-down charm make the town special. A rapidly changing demographic, skyrocketing property values, and a location between river and ocean make it a very inviting target for intensive resort-style development. Yankeetown’s 600 residents live alongside the 1200 residents of Inglis; together with a scattering of families outside the town limits, they make up the greater Inglis/Yankeetown area. This is my home town.

Welcome to Yankeetown While I have spent the last ten years moving around the world, first serving in the U.S. Air Force and now as a defense contractor, I have always held a tight bond with my home town. While homesick and stationed in Germany in 1998 I created the first Yankeetown web site, and this site remains in operation today with public photo albums and message boards for folks to share memories and stories. I added a second site in 2002 with the intent of offering townsfolk free webpage hosting and email addresses. In early 2006, a major development was being proposed for Yankeetown and a friend contacted me about making a web site to act as a clearing house of information about the development and what people might be able to do to stop it.

Welcome to Yankeetown

Issue 15

Evolution of technical requirements The initial requirement was simple: make a web page with information so citizens could have a resource other than word-of-mouth. I asked that the principle people interested in having the page do all the writing; as a very interested Yankeetown ex-patriot, I felt that I was too far removed geographically from the issues to be an effective participant. So I established my role clearly as a technical service provider and started work on http://www.saveyankeetown.com/. The initial requirement was simple: make a web page with information so citizens could have a resource other than word-of-mouth Like any new technology project, the initial requirement immediately escalated as piles of raw documents filled my inbox. My copy-paste-smile role quickly morphed into graphic designer, proof-reader, editor. Then the participants who initially wanted the web site moved on—not away from the development issue, but away from the technology part. After the initial rush to set up, they could rarely be found to vet new information, provide written material, or approve new layouts. It became my project, not unwillingly but certainly without my intent. I felt that three things were needed in addition to the simple, static information page we had so far: a listserv to facilitate private group discussion, a public message board, and a photo album. Three months later, the listserv is largely unused, but the message board and photo album are examples of vigorous civic debate. Three things were needed in addition to the simple, static information page we had so far: a listserv to facilitate private group discussion, a public message board, and a photo album

Technology used I leased a Red Hat 7.3 server from John Companies. While somewhat expensive, a John Companies server is perfect for someone who likes to compile their own web server software and who feels naked without a command line and a root password. PuTTY from my Windows XP office-automation load or OpenSSH from my Red Hat development box connect me to the remote server. To support the web site, and in support of other sites I maintain, I built Apache 2.2.0 from source and compiled support for SSL. I built PHP 4.4.2 from source with MySQL and gd support, and added appropriate LoadModule directives to my httpd.conf. An unrelated project I was working on had a dependency on the 4.x release of PHP and I didn’t plan on this new project being so high-speed as to require the latest 5.1.4 release. The MySQL 4.0.14 install package from binary, again in support of an older project, rounded out the server requirements. No server running PHP and MySQL is complete without phpMyAdmin, in this case version 2.5.3. I now had a good baseline system. I’m running qmail, ezmlm, vpopmail, and Squirrel Mail to provide virtual hosting of mailboxes on different domains. This setup is challenging to get tuned correctly, but the cookbook on setup maintained by Michael Bowe is simply brilliant. Having ezmlm already loaded, it was easy to set up any number of mailing lists. It seemed that only one was required to save Yankeetown. The Phorum message board system is clean and simple. I chose Phorum version 3.4.6 because I had good experiences with it from other projects and because it integrated well into my simple site design. I chose Phorum version 3.4.6 because I had good experiences with it from other projects and because it integrated well into my simple site design For a user-friendly photo album, I needed something that would allow anyone to upload a photo and any number of comments to be associated with any particular photo. I had some old Perl code lying around from the very first Yankeetown web site, and it seemed to fit the need. This code was at least eight years old but worked fine. Going to the photo album presented every uploaded picture in the original size, with all comments underneath the picture, and the upload form was presented at the bottom of the CGI-generated page. After several dozen large pictures were uploaded, townsfolk started complaining that their dial-up connections weren’t able to handle the photo album page at all. What I needed was a photo album that would

Technology used

Issue 15 create thumbnails. Having PHP and MySQL already installed, I went looking for a PHP-based photo album. There are many to choose from, but none that met all requirements. I finally settled on PhotoFrame 6.9. PhotoFrame creates thumbnails in a left-side navigation frame and presents full-sized pictures with comments on the right. Visitors can add any number of comments to an uploaded picture. The PhotoFrame code is a single file with just a few configuration variables and installs easily. Perfect, except that it has no facility for uploading pictures without an administrative password. I broke apart my ancient Perl code, extracted the upload subroutines, and scabbed a CGI onto PhotoFrame to satisfy all the requirements.

The inevitable problems The whole site seemed like a quick job, a few hours’ fun with HTML and I’d move on to the next project. Instead, it quickly mushroomed into a very busy server. Don’t let the “population 600” fool you; there are people from around the state and the country who have lived in Yankeetown, who plan to retire in Yankeetown, who hold the place dear to their hearts. I had daily server crashes.

Save Yankeetown website A quick consultation with John Companies technicians showed that my virtual memory was insufficient; after a certain point whenever httpd would try to fork() to handle a new connection it would simply die. John Companies increased my allocations and, after a very choppy week, the server settled down to the business of serving. No message board is complete without spam. In what appeared to be an automated attack, we started getting advertisements posted in every thread. The first round of attacks all came from the same IP address. A Deny statement in my VirtualHost directive solved the problem. A week later, the same advertising spam, but from a dozen different addresses. The users of the message board and I had a heated debate about requiring Basic Authentication passwords, mandatory registrations, moderated threads. One citizen suggested that I call the FBI to report the interference with grassroots political action by subversive parties. In the end, I simply added Deny statements for several dozen IP addresses and the problem has all but gone away. A request came for a private discussion area. The public forum serves a great purpose, but a small group wanted a collaboration area for strategic planning. My version of Phorum supports multiple message boards from the same instance, but I didn’t have any easy way of making one specific message board protected. I made an SSL-protected portal with Basic Authentication and installed phpBB2 2.0.21, using the same MySQL database instance already installed. phpBB2 is a versatile tool with many more security features, but it’s also more complicated looking from a user’s perspective. Despite their request for privacy, security, and registration-based accountability, the members of the group were largely unable to navigate the passwords and new interface. Since they all live within two miles of each other I suspect that they’ve just taken to having lunch together to do strategic planning, but this underpins the lesson that more is not always better when it comes to technical solutions. I had initially thought about upgrading my old Phorum to the new, “better” phpBB2 for the general message board, but this experience taught me that an upgrade would kill the perceived usability of the site.

The inevitable problems

Issue 15 The single biggest problem I’ve had isn’t related to the technology at all, it’s the people The single biggest problem I’ve had isn’t related to the technology at all, it’s the people. Many times since I’ve started this project I’ve found two emails in my inbox: Bob says that Jim is a jerk and needs to be kicked off the message board. Jim says that Bob is a moron and should be banned for life. People make threats to each other, people insult each other, and some days it seems like no one can spell their own name correctly. But I built the system using free software tools and I’m a free software guy, so I’ve tried to apply this sensibility to the content as well as the technology. Everyone gets a voice, no one gets banned, ever, but no one is allowed to threaten or insult, ever. Correct spelling is highly encouraged, but not required.

Lessons learned I’ve decided that it is almost impossible to predict the behavior of a site like this. There are 20-80 posts per day in the message board and thousands of hits per day to the site itself. While those seem like small numbers, remember that this is a town of 600 and the only advertising the site has had is word-of-mouth. Recently, a citizen set up a message board on a different web site with the intent to run a more civilized, directed-topic discussion about town politics. The only posts in that message board were his own, so he shut it down after two weeks. How do you plan for a site like this when it could go super-hot or super-cold? Were I to do it again from the start, I would build from the most current software versions to prevent needing to upgrade while the site is busy. I would also try to find a reliable content editor so I could focus on systems support, and I would expect nothing but prepare for everything. Coordination of effort and sharing of information have led to action that has clearly been a significant roadblock to big development in a small town

Results The political debate is still on-going. The mayor is being officially recalled. The state’s attorney is involved. The Florida Department of Law Enforcement is investigating allegations of misdeeds in Town Hall. The Governor has declared a state of emergency and has directed special elections. Yankeetown has emerged on the state-wide stage with front-page coverage from major newspapers like the Tampa Tribune and the St. Petersburg Times. For now it appears that the proposed development is at a standstill, if only because the town government has been shut down pending elections and investigations. There have been accusations that the Save Yankeetown movement has acted poorly, improperly, and in bad taste, yet coordination of effort and sharing of information have led to action that has clearly been a significant roadblock to big development in a small town. While it is the dedication and tireless effort of the townsfolk that has brought the town to the point of political catharsis and has stymied what looked like an easy paycheck for a development company, it’s commonly pointed out that it’s the technology that provided the edge. The use of free software tools allowed the townsfolk to have a common forum to collaborate amongst themselves and to draw wide-ranging attention to a political firefight and an incredibly vigorous grassroots effort. The LAMP (Linux/Apache/MySQL/PHP) combination is common, but in this small town it has been put to use with a very uncommon result.

Biography Andrew Seely (/user/20728" title="View user profile.): Andy taught Linux and UNIX courses at the University of Maryland in Europe for five years, and his interst in free software reaches back almost as far as his interest in small-town government. He's currently a defense contractor in central Florida. His wife Heather is his init process.

Copyright information Verbatim copying and distribution of this entire article is permitted in any medium without royalty provided this notice is preserved.

Results

Issue 15 Source URL: http://www.freesoftwaremagazine.com/articles/free_software_political_action

Results

Issue 15

Results

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Digital image resizing with the GIMP Are you sure you know everything needed to resize images? And can you do it properly with the GIMP? By Gianluca Pignalberi Processing digital images is a very common task today. Image processing tools are so common that users often process images by trial and error, without really knowing what they are doing. One of the operations people fail most commonly is resizing an image. In this article I will explain how to resize images. This is a way for me to celebrate 10 years since I started studying digital image processing!

Fundamentals There are plenty of digital image formats out there. Each of them provide common or unique features. The most common formats, such as TIFF (Tagged Image File Format), JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), are defined raster formats, as they represent the image as a matrix. Each matrix entry is called a pixel (picture element) and represents a color (also called light frequency). These images are commonly known as intensity images. There are other types of raster images, such as: • range images, where colors represent the distance between the object and the sensor (usually white pixels represent surfaces that are nearer to the observer and black ones represent surfaces that are the furthest away); • thermal images, where colors represent temperatures; • acoustic images, where colors represent the acoustic reflectance. You also have vector images, where the represented scene is not a matrix of pixels, but is instead described by formulas instead. Examples of such formats are SVG (Scalable Vector Graphics) and EPS (Encapsulated PostScript). They are usually used to store technical drawings, plots and camera-ready documents. The resolution indicates how close each pixel is to the others The advantage of a vector image over the raster is that it can be scaled (in or out) without loss of quality. However, vector images can’t efficiently store complex images such as photographs. In this article I’ll focus on raster images, and I’ll explain how to resize them with the GIMP [1]. Keep in mind that you can usually open a vector image with the GIMP, which will rasterize the input image before showing it. You can also read more on image formats in [2].

Images, pixels, resolution Open the GIMP and create a new image typing Ctrl + n (that means “hold the ‘Ctrl’ key and type ‘n’”) or by choosing New from the File menu. You’ll be given the window in figure 1.

Images, pixels, resolution

Issue 15

Figure 1: The GIMP’s window in which you enter the data to create a new image You are asked to enter the image size; this information is expressed in pixels by default. A summary tells you that the image resolution is 72 dpi (or “dots per inch”—I’ll talk about this more in a moment). You can also edit the advanced features to change the resolution, and other information I won’t talk about here. Once you have created the image, you can see how big it is exactly in inches or centimeters, because you know the image’s resolution which is expressed in dots per inch (dpi) or in dots per centimeter (dpcm). This piece of information tells you how many pixels will make up each inch or centimeter. Consider, for example, a 512 x 512 pixel image. The so-called print size is calculated using the following formula: size = pixels / resolution. So, since 512 / 72 = 7.11, the considered image is 7.11 x 7.11 square inches (or in²) at 72 dpi (on an output device like a monitor), and 1.71 x 1.71in² at 300 dpi (on an output device like a common printer). Of course, GIMP and similar programs will take into account the image resolution, and will show the image accordingly, regardless of the output device’s resolution. GIMP and similar programs will take into account the image resolution, and will show the image accordingly, regardless of the output device’s resolution You can also enter image size in other unit measures, as humans are more experienced with centimeters or inches than with pixels. You now know how to create an image of the right size from scratch. However, you will often work on third party images: digital pictures, scanned documents, screen shots, and so on. Opening such an image is straightforward: either type Ctrl + o or select File→Open.

Lossless resize If you’ve just opened an image to work on, at the top of the image’s window, in the title bar, you can see the image’s size in pixels. To find out more, you have to select View→Info Window (or Shift + Ctrl + i). The info window provides you the dimensions in pixels and the resolution, along with the print size. If you have an image that is 2775 x 3525 pixels, stored at 72 dpi, the info window would tell you that it is about 38 x 48in². How can you make it smaller? Select Image→Scale Image to get the window shown in figure 2.

Lossless resize

Issue 15

Figure 2: The window that the GIMP provides to let you resize the current image Change the X resolution value to 300 pixels/in (pixels/in is another way to say dpi). As soon as you enter the value, the corresponding Y resolution value will change to 300. This happens because the chain next to the resolution values is “locked” (in which case, the same dpi will be kept for the image’s height and width). If you “unlock” the chain, you will be able to change each of the resolutions separately. To apply the resize just click on Scale. Look now at the info window: you can see that the print size is 9.25 x 11.75in². You resized the image by putting the same amount of pixels in a smaller area. This means that you’re not losing information, and the image file remains substantially untouched in its byte count (how much space it will take on disk). If you want to change the image’s file size you need to change the number of pixels If you decrease the resolution, the pixels will be rendered bigger and the resulting image will also be larger. Because of the rendering, the final image’s quality will be slightly worse. In figure 3, I show a real example: FSM’s cover. Though both the images are 2775 x 3525 pixel, the left one is 300 dpi and fits a U.S. letter page (i.e., 8.5 x 11.0in²), once trimmed 0.375in per side. The right one is larger (72 dpi), so a U.S. letter page shows only a small portion of the whole image.

Figure 3: FSM’s cover image, rendered at 300 dpi (left) and 72 dpi (right), as included in a U.S. letter page

Lossy resize If you want to change the image’s file size, you need to change the number of pixels. This kind of change determines an information change: a loss of detail if you decrease the number of pixels; or an arbitrary addition of pixels, or interpolation, if you increase them. You have to decide if such changes are a good trade-off between file size and image intelligibility.

Lossy resize

Issue 15 An operation I perform very frequently for FSM (see textbox 1) is resizing FSM’s covers and ad images. I make them low resolution following a simple procedure: the new image will have the same print size as the original, but a lower resolution. So the new pixel dimension is determined with formula 1.

Formula 1: the simple way to calculate how many pixels the resized images will have I fixed medium resolution to be 200 dpi, and low resolution to be 72 dpi. Of course, what “high”, “medium” and “low” mean depends on you. For instance, images for a web site could be considered high resolution at 72 dpi, medium resolution at 60 dpi, low resolution at 48 dpi, since a web site is mainly viewed on a computer screen.

Textbox 1: How FSM provides small downloadable PDFs Once in a while, one of our readers ask me how we manage to stuff the whole magazine (including the cover and the ads!) in a 5MB PDF file. You might have noticed that the PDF available to our subscribers isn’t good enough to be professionally printed, since the image quality is too low for a printing house. Well, it’s as simple as that! Our typesetting system is based on a real programming language. So I was able to write a wrapper command to embed the right image in the final PDF. To typeset the downloadable magazine, the wrapper is programmed to include low resolution cover, ads and images. The images are preprocessed the way I described in the section “Lossy resize”. It’s now time to lossy resize images. Considering FSM cover: as I said, I want it to have the same print size (i.e., 9.25 x 11.75in²) but at 72 dpi resolution. So, using formula 1, I get X = 2775x72/300 → X = 666; Y = 3525x72/300 → Y = 846. So, 666, 846 and 72 are the figures I have to input in figure 2’s window, to replace the pixel size and the resolution. The resulting image will be print-sized as the original. You’re not forced to apply formula 1 every time you resize an image. You are free to enter the figures you like, and experiment to find the results that suit you best. What you might have realized by looking at my example, is that the resulting image has less pixels than the original. This means that it has less information, and thus it’s not as defined. A similar consideration applies when you increase the number of pixels, and, hence, add information (the program has to add “extra” pixels). This leads to the creation of artifacts. In figure 4 I show an example of both the cases.

Lossy resize

Issue 15

Figure 4: the center image (the original) is reduced (left image) and enlarged (right image) multiplying and dividing by 4 its number of pixels. The result is clear: the callouts are hardly readable on the left, and look “awkward” on the right with their blue halos

Conclusions In this article I explained how to use the GIMP to resize images. Now you can see that there’s a lot more to resizing images than you probably thought! You are strongly advised to check the GIMP’s manual [3], particularly when experimenting. Happy resizing!

Bibliography [1] The GNU Image Manipulation Program. [2] Graphics file format entry in Wikipedia. [3] GIMP User Manual.

Biography Gianluca Pignalberi (/user/4" title="View user profile.): Gianluca (/contacts/g.pignalberi) is Free Software Magazine's Compositor.

Copyright information This article is made available under the "Attribution-Sharealike" Creative Commons License 3.0 available from http://creativecommons.org/licenses/by-sa/3.0/. Source URL: http://www.freesoftwaremagazine.com/articles/digital_image_resizing_gimp

Bibliography

Issue 15

Bibliography

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Fun with free software astronomy A photorealistic experience By Alan Berg Astronomy software comes in many forms—from the details of computer intensive Grid computing of the distribution of stars (okay that’s astrophysics) to rendering the night sky in artistically detailed and sumptuous graphics. Being a devoted backseat observer to the evolution of the Universe in general and GNU/Linux software in specific, I thought it was time to show off what I consider to be the elite of desktop elegance. I will describe the installation and use of two astronomy related software packages: Stellarium and Celestia. These packages are visually appealing and fun to use. You never know, you might learn something along the way. And, if not, at least your eyes will feast upon some delicious candy.

Figure 1: Stellarium with constellation art rendered Stellarium (figure 1) is a sky renderer that marries accuracy and correctness with pure, unadulterated art. The software is a precision tool with hairline accuracies, which places you anywhere in the world at any time. You may even choose to view the heavens from the moon. Don’t be fooled by the software’s 0.8 version label (at the time of writing). This is one of the highest quality products in the market place. Stellarium is one of the highest quality products in the market place Celestia is a real time 3D space simulation; using open GL graphics (figure 2) and a plug in infrastructure, this software delivers an educationally valuable double punch. You may tour the solar system or with the help of extra datasets discover the scale of the known Universe. Both Stellarium and Celestia have a supportive community of developers and their project homepages reside on Sourceforge.

Fun with free software astronomy

Issue 15

Figure 2: Celestiaâ&#x20AC;&#x2122;s view of Earth via the softwareâ&#x20AC;&#x2122;s default dataset

Memories Why am I writing this? This is an article based on a remembrance of my father and reinforced by the brilliance of modern well-written open source Astronomy software. When I was five or six my father used to walk with me on cold winter nights from my house at the top of a country hill to the bottom where my sports club existed. Cold nights were perfect for viewing the stars. My father loved to tell me stories of the Greeks and the Romans and relate them to the patterns above. We had an understanding that during these quiet moments only the stars counted and nothing else was said. Therefore, it was with great pleasure that I recently rediscovered the software version of this experience. Living in a big city (Amsterdam), where city lights, air pollution and airplane and car traffic limit the quality of stargazing, a high quality software simulation is the nearest I can meaningfully get to the past.

Installation There are many differences between a desktop environment and a minimalist production environment. At a guess, on my desktop there sit menu links to over a hundred useful or fun programsâ&#x20AC;&#x201D;from a rather overburdening array of addictively joyous games to utilities for burning CDs and reading RSS feeds. Regularly, there is a new set of updates. In fact, once in a blue moon, I sit at home with a glass of whisky and a dubious smile on my face installing the newest and the best. Package managers such as YUM (RPM) and APT (DEB) support this craving. Under this context, I would be foolish to compile and deploy new sources by hand. Updating all the software and seeking dependencies would not be achievable or pleasant. Therefore, for installing astronomy packages in a desktop environment, I would strongly recommend the use of a package manager. For a server environment, where you want a minimum of software tailored for specific tasks, the need is less stringent and perhaps even counter-productive. For installing astronomy packages in a desktop environment, I would strongly recommend the use of a package manager There are three main methods for installing software within a GNU/Linux environment. The primary method is the use of a package. Packages not only contain the software but are also structured to place the software correctly within the target GNU/Linux distribution. Complexities such as dependencies and file structures and placing menu options with the GNOME or KDE X windows environments are understood. Upgrading is trivial. Therefore, I will use this approach to install Stellarium and Celestia, even at the risk of stating the obvious. Zooming into the details of installation: there are two main competitors in the packaging domain RPM and Debian packages. For Debian, packages apt-get or tools sitting on top of apt-get do most of the heavy lifting. For RPM, YUM is currently my tool of choice. Both methods are best suited to a live internet connection, and if you are automatically updating your system every night through a cron job, then you will probably need a reasonably fast internet connection at that. The second approach is to download the tar files and compile by hand. If you are lucky, things work out via a couple of standardized commands such as configure, make, make test, make install. If you are

Installation

Issue 15 unlucky, patching and library dependencies can make for an evening of dependency hell. The third and final approach is the use of an executable custom executable (normally called “installer”) that copies the right files in the right spot, and sits outside of DEB or RPM package control. However, I won’t be using this method in this article. There are numerous GNU/Linux distributions, and the vast majority understand either RPM or DEB packaging; I will, therefore, describe both means of installation for Stellarium and Celestia. The target operating systems being both Fedora Core 5 for YUM/RPM and Ubuntu Breezy for APT.

Installing on Fedora Core 5 To install Stellarium and Celestia under Fedora Core 5 using YUM from the command line as either root (bad idea) or a user that can sudo, first check that you have the most current version of YUM: sudo yum update yum

Next, search for the Stellarium package: sudo yum search Stellarium

You should see a verbose output and discover a relevant description within it. The partial output should look similar to: stellarium.i386 0.7.1-7.fc5 extras Matched from: stellarium Stellarium is a real-time 3D photo-realistic nightsky renderer. It can generate images of the sky as seen through the Earth's atmosphere with more than one hundred thousand stars from the Hipparcos Catalogue, constellations, planets, major satellites and nebulas. http://stellarium.free.fr/

Notice that the package can be found under the “extras” repository. A repository is the place that all the packages, or a relevant group of packages, reside for a given distribution. If you don’t have the extras repository then search for the YUM extender under the System menu option and create a new entry. The main disadvantage of installing a package is that sometimes the package is older than the latest and the best. The Stellarium package was also true form. Assuming that you, like I, are running a boring old Intel based system then you may install the software via: sudo yum install stellarium.i386

If you run into problems at this point, I refer you back to the Stellarium homepage. You will find downloads for Mac and Windows users as well. When trying to understand a new package I always read the man pages: man Stellarium

At the bottom of the page is the “see also” section, which gives hints to other relevant packages that may be worth installing. Under the standard GNOME Windows manager, Stellarium is linked under Applications→Graphics→Stellarium nightsky render. To be honest this location confused me the first time as I was looking under the more intuitively obvious Edutainment section. Installing Celestia is just as simple sudo yum install celestia.i386

Installing on Fedora Core 5

Issue 15 The GNOME menu option to look for is again under the graphics section “Celestia space simulator”. And could life be that simple? Yes and no, both packages are now installed, but I still had an issue with a non-running Stellarium. Running from the command line, I received the following output:

------------------------------------------------------[ This is Stellarium 0.8.0 - http://www.stellarium.org ] [ Copyright (C) 2000-2005 Fabien Chereau et al ] ------------------------------------------------------Warning: Couldn’t set 1024x768 video mode (No video mode large enough for 1024x768), retrying with ste

Two causes of concern the copyright notice and the screen resolution of the windows manager. I alleviated the copyright concerns by reading the products FAQ Phew, the product was GPLed. The screen resolution issue was due to the default setting of 800x600 for my installation of Fedora. I had two choices. The first to change the settings in ~/.stellarium/config.ini: [video] fullscreen screen_w screen_h

= true = 800 = 600

Or modify /etc/X11/xorg.conf. Stating the obvious, if you damage this file then you may have to rescue your installation and copy back a backup of the original xorg.conf. So please be careful. I needed to update the configuration file all lines with modes needed to be updated as follows: Modes "800x600" "640x480" changed to Modes "1024x768" "800x600" "640x480" Now, if you had the same issues, restart your Xserver or reboot your system. For most modern systems, the new higher resolution should kick in.

Installing on Ubuntu apt-get is a valid tool for package installation under Ubuntu. To update the packages available you should run: sudo apt-get update

Next, you will need to search for the relevant packages: sudo apt-cache search Stellarium

And finally install: sudu apt-get install Stellarium

The same sequence is valid for Celestia. As a bonus try and install ssystem and kstars.

A super fast tour of the software Stellarium Stellarium is valuable for amateur sky watchers representing a polished approach to learning your way around the sky. The software has a highly intuitive point-click-and-zoom structure for accurate placements and the ability to set Earth coordinates and time. On starting, click on a named object (for example the moon) and then press the space bar. The space bar selects and centers the object. To zoom in you may use the scroll wheel on your mouse or the page up and down buttons. To ask for help press ‘H’, a list of keyboard combinations is then listed. At the bottom left-hand-side of the screen is a single line of icons that turn on and off particular features such as the drawing of the constellations, the blur effect of the atmosphere, or the adding of a background landscape. The spanner icon allows for configuration. On the bottom right-hand-side are icons that affect how fast time passes, placement of stars and planets relative to the horizon of the Earth or

A super fast tour of the software

Issue 15 the point in the heavens that the spacebar has actively centered. I would recommend turning off the atmosphere, then zooming in on Saturn and accelerating time. Then you can watch the planet’s satellites gingerly rotating around their mother. Stellarium is valuable for amateur sky watchers

Celestia To be impressed from the very beginning go to the help option at the top of the page and run the demo. Either you’ll be very impressed or you’re a time traveler from planet “Dezog” in the galaxy “No Emotions”. To gain insight, you can popup a list of mouse and keyboard controls via “Ctrl + H”. To move around the heavens you can drag the sky around with the left mouse button. Clicking on an object targets it, to centre press ‘C’. Right-clicking on an object gives you the “go to” option, and within a short time from selection, you will be near the chosen target. To zoom in on the target you may use the scroll wheel of your mouse or both the left and the right mouse buttons at the same time and then dragging. This last option sounds difficult, but, after a few seconds practice, it just seems so obvious. Celestia is highly expandable—with detailed image maps and models of spacecraft downloadable, and yes, there are tours. There’s also an excellent resource site and an expanded version of Celestia for the education market. If you have a good, fast connection and enough disk space, it’s well worth downloading the 1GB educational distributions and installing them. Installation is via expanding archived data into the correct file locations under the root directory of the package. I would, of course, recommend that you try the basic version out first.

Final Comments I am truly impressed by the total experience delivered by the software mentioned, in particular I was awed by the photo quality imagery. I therefore look forward to sharing quality time pointing to the virtual sky with my two sons Lawrence and Nelson and explaining the stories that my father had shared with me on those clear winter nights.

Acknowledgements I would like to acknowledge my father’s valiant efforts at trying to turn me into an educated man.

Resources • Celestia educational extras • Celestia homepage • Stellarium homepage

Resources

Issue 15

Copyright information This article is made available under the "Attribution" Creative Commons License 3.0 available from http://creativecommons.org/licenses/by/3.0/. Source URL: http://www.freesoftwaremagazine.com/articles/astronomy_software

Resources

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Stitching seamless panoramas with Hugin Adding new tools to your free software graphics toolkit By Nathan Sanders We can’t all afford four-figure priced twelve megapixel digital cameras with wide angle lenses. We can, however, all use free software to embellish the photographs taken by our modest equipment and belie their resolution and viewing angle. Set the GIMP aside for a moment and launch Hugin, a powerful cross-platform GTK frontend that will help you quickly and easily stitch individual photographs into one, large, seamless panorama.

Panoramas from start to finish Panoramas are created by stitching multiple images together. To do this, you must find spots where your images overlap and designate them as control points. Then you will optimize your collection by selecting the best way to rotate, distort, and align these images so that they come together seamlessly. Finally, you will stitch and blend them all together and do some postprocessing.

Taking good panoramic photographs When composing photographs with the intention of combining them into panoramas, it is best to adhere as closely as possible to the following rules: • Take all photographs from the same location, using a tripod if possible • Aim for a 33% overlap between each frame • Keep the camera’s metering equal between shots • Keep the white balance equal between shots You can take vertical panoramas as well as horizontal ones, or even multi-rowed ones Remember, you can take vertical panoramas as well as horizontal ones, or even multi-rowed ones. Use as many images as you like, but bear in mind that larger and more complex panoramas will take longer to process. Erring in any of these guidelines will not prevent you from composing a brilliant panorama, however it will make it more difficult and may aggravate seams between the individual photographs. Such issues may be fixed with the GIMP or another image manipulation tool after the panorama is stitched, though white balance and color deviation issues should be fixed beforehand.

Introducing Hugin and associated tools To combine your photographs you can use Hugin, a GPL-licensed frontend to Panorama Tools available for GNU/Linux, Mac OSX, and Windows in several languages. Hugin will let you design your panorama, but you’ll need additional tools to find control points and do the actual image processing. Try Autopano-SIFT for automatically finding control points, though you may also do this manually, and Enblend for blending the images together. Both these tools are also GPL and cross-platform.

Introducing Hugin and associated tools

Issue 15

Figure 1: Unfortunately, Hugin does not greet you with an intuitive or simple interface Larger and more complex panoramas will take longer to process You must first configure Hugin to work with the tools you have installed. Access the configuration dialog in File → Preferences. Ignore the first three tabs (“Panotools”, “Finetune”, and “Misc.”), they will most likely have the correct options predefined. In the fourth tab, “Autopano”, make sure Autopano-SIFT is selected, that it is pointing to the correct binary location, and that the Arguments are set to -o %0 %i. In the final tab, “Enblend”, make sure Hugin has found the proper executable and set its arguments to -v -m XYZ, where XYZ is equal to about half your physical RAM (this is how much image cache Enblend will use). You may want to enable the checkboxes in the “Enblend” tab to cut down on temporary file space usage. Make sure these settings are correct each time you launch Hugin.

Finding control points I have selected six four-megapixel JPEGs that I want to stitch into a horizontal panorama. The images were taken handheld and not well composed. However, Hugin and its tools should have no problems combining them seamlessly.

Figure 2: Use the “Images” tab to choose which photographs you wish to combine To begin composing your panorama, drag and drop your images into the “Images” tab or select them using the “Add Individual Images” button. Once your images are listed in the “Images” tab, select whichever photograph will represent the middle of your panorama and click “Anchor this image for position”. The “Camera and Lens” tab is used to correct images for camera error, ignore this since Hugin should have already filled in these variables based on your camera’s EXIF data. There is a choice to be made now: do you want to trust Autopano-SIFT to choose control points, or would you rather do it yourself? Autopano-SIFT will generally do a good job, but having these complicated decisions made by complex algorithms can be be a very lengthy process, often taking several hours, regardless of your

Finding control points

Issue 15 Desktop computer’s processing power. Manually selecting the control points may prove to be far quicker, but will take some experience and perhaps some frustration. Either way, you will probably need to do some manual adjustment to the selected control points based on the preview of your final panorama. Autopano-SIFT will generally do a good job, but having these complicated decisions made by complex algorithms can be be a very lengthy process If you choose to let Autopano-SIFT select your control points, click “Create Ctrl Points” on the “Images” tab and go out for some coffee. Read on to learn how to adjust Autopano-SIFT’s selections. To manually select your control points, turn to the “Control Points” tab. You will see three panes, two from which you can select any of your source images and a third below where you can view the details of your control points. Start by selecting “0” in the left pane and “1” in the right. If these two images do not overlap, you can rearrange them in the “Image” tab or simply move to “2” and beyond. Once you have found an overlap, look for a small, distinct point which is common to both pictures. For instance, a car’s tire that is present in both images. Objects like tree tops, brown spots in grass, mountain peaks, or rocks may be easier targets for landscape images—but be careful that you do not confuse one tree for another. Click on your first control point in each image. The pane will zoom in to let you select more precisely. Click the “Fine-tune” button to have Hugin adjust your selections to fit perfectly in each image, or check the “auto fine-tune” option to have this done when you first click. If your selections do not match closely enough for Hugin’s liking, it will spit back an error dialog displaying how far it deviated from an acceptable match. This may simply be caused by camera distortion, so you may wish to ignore the error. When you’ve found a control point you like, click “Add”. The new control point and its coordinates will appear in the lower pane’s list. Any control points in the list will be specific to the two images selected in the upper panes.

Figure 3: I have selected the peak of a rocky hill for my first control point Select four or five more control points before you move on to your next set of images. You can add, subtract, or change these later, so don’t put too much effort into this. Continue on selecting five or so control points between each pair of images that overlap. If you have a pair of images that do not quite overlap, don’t be afraid to simply set control points on the edge of the images. If they come together nearly enough, Enblend will be able to do an acceptable job of merging them to look whole.

Finding control points

Issue 15

Figure 4: These photographs were taken poorly and don’t quite overlap; control points were placed where they nearly match up If you have a pair of images that do not quite overlap, don’t be afraid to simply set control points on the edge of the images

Arranging your images based on their control points Now switch to the “Optimizer” tab to have Hugin arrange your photographs for you. If you haven’t been saving your Hugin workspace, now is a good time to do so (simply use File→Save). The optimizer tab consists of an “Optimize” listbox, an “Optimize Now!” button, and a multitude of figures. Select the “Everything” option from the list box and click “Optimize Now!”—this should only take a few seconds. If you have a very complex panorama with scores of control points and a somewhat slow computer, optimizing for “Everything” make take a few minutes. In this case, you may want to optimize for simply “Positions (y/p/r)”. Hugin will return some statistics about its optimization effort. Ed Halley’s “overview” tutorial recommends that: “the average error should be less than 1 pixel for an excellent fit, and the maximum error should only be 5 pixels or less.” If your results don’t satisfy these guidelines, continue reading to learn how to tweak your panorama. Now you can get a glimpse of what your panorama will look like for the first time! Click View→Preview window to take a look.

Figure 5: The initial fruits of my effort Depending upon how lucky you were in placing your control points, you may not be overly impressed. Fortunately, perfecting the panorama takes little knowledge beyond what you have already learned. If there are visible seams due to color disparity between your images, this will be smoothed out by Enblend during the stitching process and will only result in a noticeable seam in the case of strong differences in color between images. If this is the case, you should go back and preprocess your photos and adjust their color levels to an

Arranging your images based on their control points

Issue 15 approximately equal level. If there are visible seams due to color disparity between your images, this will be smoothed out by Enblend Start adjusting by deciding whether or not your panorama is well centered. Click the “Fit” button in the preview window to see how much black space is being left to either side. If your anchor point needs to be repositioned, make a mental note of the middle of the panorama and return to the “Images” tab. Select the proper midpoint image and click “Anchor this image for position” again. Then click “Adjust this anchor spot…” to be presented with crosshairs that will let you select the exact feature in the image you wish to focus on. If you want to see what your changes have done, optimize for “Everything” again and then open the preview window. Once there, make sure you click either the “Update” or “Auto” button to view the new panorama.

Refining control points Your image is well centered now, but there are probably other things wrong with it. If any two photographs are not lined up correctly, you may need to adjust the control points. Use the “difference” option in the preview window’s “Blend mode” listbox to help choose which images are not stitching properly. Ideally, all overlaps will appear as black boxes. If you find two poorly stitched images, isolate them by clicking off all the other image buttons in the “displayed images” frame of the preview window. Then click “Center” and “Fit” to zoom in on them.

Figure 6: These images didn’t stitch as well as I would like Take a look at figure 6 for an example of two images in need of control point revision. As you can see, the mountain peaks are somewhat askew and so is the stream’s bend. This image could probably be improved by setting control points at each side of the stream’s overlap, top and bottom, and at both the peak of the mountain and the treeline. Remember to take a look in “difference” mode to see how closely the images are being aligned. Optimize again with your new settings and have a look at the preview window. Repeat this process with different control points if you need to. Erase some if they look like they’re tilting the image. If any two photographs are not lined up correctly, you may need to adjust the control points

Refining control points

Issue 15

Figure 7: It looks much better now

Straightening out the panorama with guides Once you’ve corrected any stray stitching, turn on all the “displayed images” in the preview window again, “Center” and “Fit” the panorama, and look for other errors. You might find that your panorama is curved. Choose an image which is very out of balance and return to the “Control Points” tab. Select the chosen image in both upper panes. Now, look for any object in the picture which would appear straight either vertically or horizontally in reality. For instance, trees and buildings would be vertical and roads or horizons would be horizontal. Select either end of this object in each upper pane and use the zoomed preview to make sure you have positioned the control points exactly on the ends. Click “Add” to create a horizontal or vertical guide for this object. Hugin should automatically select which orientation is appropriate. If it has not, select the control point on the lower pane and adjust the “mode” list box to either “vertical line” or “horizontal line”.

Figure 8: This tree is a prime candidate for a vertical guide Optimize again and return to the preview pane to see if your guide straightened out the image enough. If it is still not perfect, add other guides. If you are not able to get a perfectly straight panorama, remember that you can crop off the curved parts when you’re done.

Stitching your panorama All the hard work is over, now, it’s time to let nona and Enblend combine your images for you. Switch to the “Stitcher” tab and look at the “projection (f)” listbox. Within it are three slightly different methods for stitching your images. “Rectilinear” stitching will give you a very distorted and elongated image suitable for creating a true panorama, the type you might view with Quicktime VR. “Equirectangular” stitching will give you something similar to a very large photograph—you’ll probably want to do this. “Cylindrical” stitching will be somewhere in between.

Stitching your panorama

Issue 15

Figure 9: You can see how stretched and distorted Rectilinear projection is Once you’ve selected a projection method, make sure the “Stitch the images” list box is set to “with custom settings below”, then click “Calculate Field of View” and “Calculate Optimal Size”. The field of view setting should be accurate, but feel free to change the size setting if you want a smaller and more manageable image. Keep in mind that the stitching process and final image may take up gigabytes of hard drive space, but the final, processed JPEG will probably be more like ten megabytes, depending upon its size and quality setting. Select “nona” as your stitching engine. Besides making scenic panorama-format images, Hugin can be used to artificially increase the resolution or viewing angle of your camera Next you must choose a method from the “interpolator” list box. There are explanations of the methods available, but you would do well to simply choose “Poly3 (Bicubic)”. This method is very quick and of acceptable quality. Set “Image format” to “TIFF” and enable “Soft Blending”, which will have Hugin use Enblend. When you’re satisfied, click “Stitch now!” and choose a filename. The process should take several minutes, depending again upon hardware speed and panorama complexity.

Figure 10: Enblend’s output Take a look at the TIFF file Enblend has created for you, hopefully you will be presently surprised. Any seams visible in the preview window due to color disparities or poor control points should be more or less smoothed out. You can now open the TIFF in an image processing tool and crop, convert, resize, and make any other corrections to the image you wish.

Enjoying your masterpiece You’ve just made your first panorama with Hugin, but don’t let it be your last! Using Hugin becomes second-nature after a few attempts. Remember that, besides making scenic panorama-format images, Hugin can be used to artificially increase the resolution or viewing angle of your camera. Try to make a few more

Enjoying your masterpiece

Issue 15 with a few more sets of source photographs or experiment with using Hugin to stitch scanned images. Let Hugin, Autopano-SIFT, nona, and Enblend be a part of your fully-featured, professional-level graphics production free-software workshop.

Figure 11: The completed panorama

Biography Nathan Sanders (/user/20396" title="View user profile.): Nathan Sanders is an experienced free-software user and frequent contributor to publications concerning open-source software.

Copyright information This article is made available under the "Attribution-NonCommercial" Creative Commons License 3.0 available from http://creativecommons.org/licenses/by-nc/3.0/. Source URL: http://www.freesoftwaremagazine.com/articles/stitching_seamless_panoramas_with_hugin

Enjoying your masterpiece

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Bzflags Kill or be killed By Alan Berg, Nelson Berg The aim of this article is to introduce the reader to Bzflags. Bzflags is a free software multiplayer 3D tank game that is frantic, full of immediate action, with a kill or be killed emphasis. The game is best served in multiplayer mode where you can hunt in packs, fight to the last ounce while chatting. Instant violent fun, gratification for those of you that need to let off steam and clear your minds living for the moment. However, because of my cowardly disposition and weak trigger finger, not to mention my poor reaction times, I have once again hired in my ten-year-old son Nelson to test out the various parts of the game and pretend to the world that he is me. I will interview him later in this article and take online credit for his kill ratios. The name Bzflags is an abbreviation of Battle Zone Capture Flags. Named after, one assumes, Battlezone, a classic, vector-based arcade game that came out just after space invaders. Showing my age, I was addicted to Battlezone and never had quite enough pocket money to kill all the enemies my psyche required of me. I even ended up doing a paper round to pay for my addiction. Capturing the flag involves exactly that, one of your team members needs to ride over the enemy flag and then take the flag back to your rest zone. This is of course made more complex by the horde of charging tanks that wish to communicate with the flag capturer their physical respect. Yes, frantic, fast, and furious. If you don’t die ten times in any given game then you haven’t lived. Four teams play at any one time. If you don’t die ten times in any given game then you haven’t lived

Installation Installation is straight forward for MS Windows users: you download the executable from Sourceforge and run it. For GNU/Linux users there are rpm packages available, and for Mac the equivalents. Pretending to be a stereotypical user, I downloaded Bzflag 2.04 onto my overloaded laptop running Windows XP home edition. Before entering the shark infested shoals of public servers, it is probably best to run a server locally on your own machine and play against computer controlled tanks. To achieve this under windows, add a shortcut with the following option: "C:\Program Files\BZFlag2.04\bzflag.exe" -solo 10 Where, of course, BZFlag2.04 is replaced with which ever version number of the game you have installed. The solo option states that you are playing against yourself and 10 is the number of tanks that the computer controls. Note: The Bzflag server requires very little in terms of network bandwidth and other system resources such as CPU. However, the man page does warn about running the server under Windows 95 due to constraints within the operating system itself. Installation is straight forward

Game basics To play the game, double click on the short cut you have just created and the game console will present itself. You are given four options:

Game basics

Issue 15 1. Join Game 2. Options 3. Help 4. Quit Before going further I would recommend reading the help page. The page contains a list of keystrokes and their related functions. After reading the help, it is time to join a local game against your computer. Yes, the evil computer that you need to fight with every fibre of your patriotic being, the self same evil computer that has stolen those hidden pixels and corrupted many undisclosed binary files. I will have my revenge! Hum, back to reality. Back at the main options, select “Join Game”, making sure the server mentioned is localhost (see figure 1). Select start server: you will find yourself under the server configuration screen, figure 2.

Figure 1: Screen grab of game joining options

Figure 2: Screen grab of start server options Feel free to modify the configuration options; the main ones of interest is the “Style”, which defines the type of game you will play. The three main styles are: “Killer Hunt”; “Free For All”; and, “Capture The Flag”. “Killer Hunt” is a simple game type, where one of the tanks is nominated as a rabbit. Every other tank has to try to kill the rabbit. Sounds a little like my average work day. In “Free For All” everyone is expected to shoot everyone. “Capture The Flag” involves team work. Four differently coloured teams try and capture the flag of one of the other three enemy teams and then tries desperately to bring the flag back to their base area. So much confusion occurs in this situation that it’s a wonder to behold and very enjoyable to be involved in. Once you have connected to the server of choice, you will find yourself in the middle of the action. If you’re a newbie, this can be a bit intimidating. You will see a console similar to that shown in figure 3. Notice the shell speeding towards you. If you dodge, you survive. If you blink, you are gone. If you are playing with a team it is good to know your team’s colour, which is indicated by the text (ready) at the top of the screen. To move

Game basics

Issue 15 you need to place your mouse in the outer box. If you place your mouse at the top of the outer box, you will move forwards. If you place your mouse in the centre you will stop, and if you place your mouse at the bottom of the box then you will move backwards. The further you put the cursor from the centre, the faster your tank will move. On some servers you may jump; this is achieved through pressing the tab key. It is considerably easier to dodge and move than it is to hit a target. Therefore, practicing your dodging ability should be your number one priority. Practicing your dodging ability should be your number one priority Within each world there are numerous flags. By moving over a flag you may change the properties of your tank or the weapons the tanks uses. You may also drop a given flag and choose another.

Figure 3: Screen grab of game playing console Chatting, or more accurately taunting, is achieved via the n (send to all) m (send to team mates only) key strokes. But, to be honest, I am too busy being a rabbit, even when I don’t have to be, to chat. Nelson somehow manages to multitask. Perhaps I should revert to playing bridge on the internet instead of damaging my tattered ego.

Interview So, did Bzflags pass the taste test of a ten-year-old psycho killer? Let us interview the young man and collect his always random thoughts. AB: Do you enjoy playing Bzflags NB: Yes until I die. Normally dying doesn’t matter, but then I noticed that you score points on killing and dying. Dying is not a winning option. AB: What is the best part of the game for you? NB: Lots of destruction. AB: What is the next best part of the game? NB: Lots of destruction. AB: What is the worst part of the game? NB: Duh, dying. You should know. I’ve killed you often enough. AB: Any suggestions for the makers of the game? NB: Can you change the point system into a description of how good you are?

Interview

Issue 15 AB: Please describe the best moment you have had playing the game? NB: When I beat the enemy team by capturing their flag. I had to run and dodge and I still managed to taunt them through the chat facility. It was like listening to 50 cent. (I don’t really know what Nelson meant by this last remark, but I know he was talking about his favourite band.)

Online cheating Cheating online is bad. When I play first person shooters alone in the dark, I cheat. I play against the computer as a god. I just love blowing things up and watching the game react with ever increasing waves of stronger opponents. Within the online world, it is possible to cheat as well. For example Bzflags has given a lot of power to the client program which allows for a multitude of cheats. Cheating online potentially takes the pleasure away from your opponents. What is the point of becoming as good as you can be only to have yourself destroyed by Mr Invincible newbie? De-skilling online games via cheats diminishes the overall motivation of your fellow gamers. Good for development purposes, but bad for the general populous. Think about it. Don’t drink and drive, don’t cheat online.

Building your own World Bzedit is a tool that allows you to design your own world. Installing is straight forward, you simply download and run the executable. Figure 4 is a screen grab of the main editor page. In the screen grab you see that there are a large number of objects needed to make a realistic world. To start with the editor, I found it easiest to download worlds from the internet. For example, I would recommend visiting the home of bmcclt, a scripted map generator, downloading a few worlds and modifying them. For really elegant examples of complex map designs, it’s well worth visiting the Purple Panzers homepage

Figure 4: BZEDIT’s main dialog To test a particular map, you may place the maps in the root directory of Bzflags and from there run the server with the world command option: bzfs -world mapname Figure 5 shows the clients rendering of the map that was shown in figure 4. Notice the rabbit (me) running like hell from a pursuing bot. I expect my son’s tank is hiding behind one of the trees; I felt his online breath down my neck.

Building your own World

Issue 15

Figure 5: Client rendering of map Prepare the coffee machine and teleport into worlds where nearly everything is deadly

Conclusions I had fun playing this game, my son more so. Of course, the graphics could be better and the range of options more diverse, but for shear frantic energy loss and immediate action Bzflags cannot be beaten. For those of you who own internet cafes or have regular LAN parties and wish to enhance your games section, setting your own protected server up takes relatively little effort. World making requires effort, but hey Rome wasnâ&#x20AC;&#x2122;t built in a day and you can always Google for the map efforts of others. Enjoy the game; prepare the coffee machine and teleport into worlds where nearly everything is deadly.

Biography Alan Berg (/user/8" title="View user profile.): Alan Berg Bsc. MSc. PGCE, has been a lead developer at the Central Computer Services at the University of Amsterdam for the last eight years. In his spare time, he writes computer articles. He has a degree, two masters and a teaching qualification. In previous incarnations, he was a technical writer, an Internet/Linux course writer, and a science teacher. He likes to get his hands dirty with the building and gluing of systems. He remains agile by playing computer games with his kids who (sadly) consistently beat him physically, mentally and morally. You may contact him at reply.to.berg At chello.nl Nelson Berg (/user/81" title="View user profile.): Nelson Berg is an intelligent and aggressive game player and seven days a week ten-year-old son of Alan Berg (/user/8) In his spare time he performs brain surgery and works for a secret organization that protects the world and sometimes the Universe.

Conclusions

Issue 15

Conclusions

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Roots access Genealogy with GRAMPS By Ryan Cartwright Genealogy is a burgeoning hobby and to help the home genealogist, a whole range of software is available. Much of it is commercial but here I’ll look at one of the most popular free software options—GRAMPS. Charting your family history needn’t mean compromising on licensing.

Finding your roots Genealogy, the study of one’s ancestry, is a growing industry. Fuelled by on-line resources like Ancestry.com [3] and television programs like the BBC’s “Who do you think you are”, thousands of people are to be found scouring web sites, visiting graveyards in remote locations and ordering obscure birth certificates all in the name of tracing their roots. Genealogy is highly addictive…you can easily find yourself staring bleary-eyed at the computer at 3am If you have ever considered tracing your ancestry, there are a few things to note: it takes time, it can create huge amounts of data and it’s highly addictive. Once you start you can easily find yourself staring bleary-eyed at the computer at three in the morning trying to figure out if the person you’ve just found is the same one you are related to. I should also say that it can be highly rewarding—especially when you amaze your relatives with the details of their family history.

From data to trees Managing all that information suggests a database and while it is possible to create your own, there exists an array of software which can ease the task of turning the data you have gathered into a family tree. Much of this is commercial although it is generally at a reasonable cost. Of course, for those who prefer their software to be free there are several applications available [4] and of these GRAMPS is arguably the most popular. GRAMPS stands for Genealogical Research and Analysis Management Programming System. A name which makes me wonder if the acronym or the full name came first. Before I plunge into the details of installing and using GRAMPS, take a look at some of the functionality that it provides, as available from the sidebar as shown in figure 1.

Figure 1: The standard GRAMPS interface showing a pedigree

Finding your roots

Issue 15 • People: The bedrock of all Genealogy software. In GRAMPS you can store people’s names (birth, marriage and others), multiple addresses (with dates) and events (e.g. marriage, emigration). • Families: People are automatically grouped by last name, you can also link them by relationship to each other. Thus allowing you to produce family tree style reports. • Pedigrees: A visual, interactive layout of a family tree. • Sources: During research the genealogist will gather information from many sources. GRAMPS allows you to record these sources so that you can find which have proved the most useful over time and waste less time by starting future searches there. • Places: It’s likely that you’ll be entering similar locations several times within the database. GRAMPS allows these locations to be stored as records in their own right, allowing you to track popular areas. • Media: You can store various typed of media file against records, including photographs, scans of documentation, videos, audio—pretty much any file. • Reports: There’s a plethora of report formats and styles available.

Taking your first steps As with most free software, there are many ways to install GRAMPS ranging from roll-your-own (download and build from sources)—to the more sedate distribution specific packages. As far as I know packages exist for most of the major GNU/Linux distributions. GRAMPS is available for both GNU/Linux and FreeBSD but I’ll review from the GNU/Linux perspective here and in particular Debian and Ubuntu. That’s because—quite honestly—it’s what I use myself. I also use KDE although this is of little importance here as GRAMPS uses both the GTK and GNOME libraries and presents a GNOME interface regardless of your desktop preference. For this reason if you use KDE you may need to install some GTK and GNOME library packages as well although any good package manager, such as Synaptic, will assist with these dependencies. GRAMPS presents a GNOME interface regardless of your desktop preference If you are a command-line person then apt-get install gramps will do the trick. For the GUI people among us, I’ll concentrate on Synaptic. Simply search for gramps, mark it by checking the box beside it’s name and click on Apply. See Marco Marongiu’s piece on Synaptic in Issue 11 for more detailed information on using it. If you are using Ubuntu, ensure you have the “universe” repository included in Repositories (under the Settings menu). Don’t forget to click Reload (or apt-get update) before you search though. In both Debian and Ubuntu GRAMPS is in the GNOME section of packages. If you are using Debian stable (at the time of writing this is 3.1r2/Sarge) then you will find your version of GRAMPS is 1.0.11-1. This article is referencing the more recent 2.0.11-1 found in the testing (Etch) and unstable (Sid) Debian distributions. Finally, if you are stuck with a proprietary operating system but still want to try GRAMPS, then why not try the Linux Genealogy Desktop CD [2]. This is a live CD based on Ubuntu and will run without installing anything on your hard disk, although you can do so from the CD if you wish.

Using GRAMPS When you first start GRAMPS you are greeted by a database dialog window where you can choose between opening a recent database, browsing for an existing one (which has dropped off the recent files list) or creating a new one. Once you open a database or create a new one, you are brought to the main GRAMPS interface with the ubiquitous “tip of the day” pop-up. Across the top is the toolbar with some fairly obvious buttons and above that a menu bar. These perform operations pretty much as expected. For example, clicking the “Add” button when in the family or people screens will add a new person, displaying the person editing window as shown in figure 2.

It’s about who you know 44

It’s about who you know

Issue 15 Let’s look at adding people to a database using the person editing screen in figure 2. Under the “General” tab you are given places for given (fore) names, family (last) names, a family prefix which may be something like “de” or “van”, a suffix, which could be something like “Jr.” or “III”, gender and dates and places of birth and death. Dates can be recorded in one of a number of formats but you are expected to be consistent and stick to the one you have set in the options dialog. Often the name you have for a person may not be their birth name—this is especially the case with married women gathered from a census. In these cases you can indicate what type of name this is, e.g. married, birth or also know as. Names can also be recorded with greater detail including the source used to find them. This is useful when looking up census information as the same person can often be using different names or nicknames in consecutive censuses. You can also indicate whether the information is to be considered complete and/or held as private—which allows you later to exclude it from exports.

Figure 2: People are the bedrock of genealogical databases, even if the images don’t quite match the names The other tabs are fairly self-explanatory and operate on similar lines to each other. You can record attributes like national origin, caste and such as well addresses and events, each of which includes references to its source. The gallery is a list of media associated with this person. In the case of figure 2 the photo of Einstein is linked against this person. When the gallery has content, GRAMPS will display the first item from it on the “General” tab. The order of the gallery can be simply changed by dragging and dropping and any media added here will be automatically logged in the main media section.

Families GRAMPS automatically groups people by family name, making them easier to find but, of course, not always accurate in terms of their relationship. A John Smith on your mother’s side may not be related at all to a Mary Smith on your father’s. The family editing window aids with linking people by more than name. Figure 3 shows a typical example of this and it’s a simple job to link partners, children and parents for the current person. These related people can be selected from the database or added directly from this window. Adding a new partner to the current person will trigger a marriage event entry as well. In cases of multiple partners, selecting a partner will display only the children from that relationship, both of which are nice touches.

Families

Issue 15

Figure 3: Managing relationships between people

Using pedigrees Of course at some point you will want to see, in a visual form, how all these people relate to each other. This is particularly the case when you find common family forenames in use across generations. For this purpose GRAMPS provides the pedigree view. A typical pedigree view is shown in figure 1. A nice feature of GRAMPS is that double clicking an entry in one view will open an editing window related to that record The pedigree is a horizontal family tree flowchart style layout. GRAMPS will display as many generations as it can fit on the screen—in my case this is five. If further generations are available at either end of the current view, a small button is displayed next to the relevant person. Clicking this will move the view in that direction. If there is more than one ancestor or descendant available, a list is provided from which you can select a person. Those which continue the tree—that is they have children or parents listed—are emboldened in the selection list. Thus if, in figure 1, Veronica Louise Smith has children, they would be listed when you click the dot to the left of her name. Any of those which themselves had children would be shown in bold. A nice feature of GRAMPS is that double clicking an entry in one view, say a person in a pedigree, will open an editing window related to that record. This enables you to move around without resorting to continuous back-tracking.

Getting information out of GRAMPS There will be times when, during your research, you come across another genealogist who has gathered quite a bit of information which you would find useful. Alternatively, you may find you have information they want. Although it is wise to never blindly accept another genealogists data as completely accurate, GRAMPS provides import and export facilities for several genealogical data formats including the ubiquitous GEDCOM file. Import and export “wizards” are provided to guide you through the process and make it very simple to perform. GRAMPS provides import and export facilities for several genealogical data formats

Reporting Any application with a database at its heart must have reporting features. GRAMPS includes a vast range of easy to use, pre-defined report types with significant customisation options. Reports can be produced on a single record in graphical and textual forms. The former being charts and graphs and the latter being textual information compiled from the database. For example a text ancestor report may contain this: “John William Smith. John William was born on 24-05-1924. John William is the child of John Henry Smith and Jane Brown. He married Susan Jane Lewis on 15-07-1949”

Getting information out of GRAMPS

Issue 15 Only the names and dates are entered by the user, the rest is composed by GRAMPS. Some examples of the graphical reports available are shown in figure 4. Each of these was produced in seconds with a few clicks of the mouse. Those reports were created in Open Document Text (ODT) format but reports can be produced in a variety of document formats including Scalable Vector Graphics (SVG) and HTML. GRAMPS includes a vast range of easy to use, pre-defined customisable reports

Figure 4: Just some of the wide range of graphical report styles available in GRAMPS

Database maintenance As your data grows and changes the database can become a bit disjointed and reports can therefore appear to give inconsistent results. Fortunately GRAMPS has a number of database maintenance tools that assist in straightening this out. This means it’s a simple job to check and repair the database, looking for things like children with a link to a parent record that no longer exists. You can also have GRAMPS verify the data itself, checking things like dates of birth and death for unrealistic lifespans. Of course, checking for duplicate people is also available. As said at the beginning of this article, genealogical research can result in a lot of data and these maintenance tools prove invaluable at highlighting parts which need your attention.

Conclusion Often, when reviewing free software, the temptation is to compare the application with a commercial alternative. It’s as if the free software application is valued by how well it could replace or replicate the commercial program. I’m going to resist that temptation here. GRAMPS is a good piece of software and deserves better than to be judged by some of its, in my opinion lesser, commercial rivals. The interface is clean, simple and easy to use and my only complaint, and I stress this is a personal preference, is that it forces me to use the GNOME interface which, at times, I find a little clumsy. The editing and reporting functions are excellent and it is very easy to quickly build up a good record of the research you have made without getting lost in a quagmire of names and dates. Clearly a lot of thought, time and effort has gone into GRAMPS and while genealogy is time consuming, addictive and data intensive, GRAMPS goes a long way toward smoothing the ride.

Bibliography GRAMPS resources [1] http://www.gramps-project.org—GRAMPS homepage. [2] Linux Genealogy Desktop CD (from the GRAMPS website)—an Ubuntu based Live CD which has GRAMPS, GeneWeb and Lifelines installed. Allows user of a non-free operating system to run GRAMPS without installing.

Bibliography

Issue 15 Websites [3] http://www.ancestry.com—a popular and well stocked genealogical website, includes searchable census and birth, death marriage records. http://www.gro.gov.uk—General Register Office for UK Government. From here you can purchase copies of birth, death and marriage certificates on-line. http://www.genesreunited.com—a genealogists networking site. Run by FriendsReunited and along similar lines. [4] Other free software genealogy programs GeneWeb—web based, available for a range of operating systems. LifeLines—NCurses (console) based genealogical database.

Bibliography

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Control machines with your machine Digital speed controller using RTAI/Linux By Sreejith Nair This article is intended for new Linux users who wish to use their Linux-box for some real work. Speed control of an industrial motor? Sounds complicated? Itâ&#x20AC;&#x2122;s not as complex an affair as it sounds. Whatâ&#x20AC;&#x2122;s interesting is that a PC powered with a Linux based Real-Time Operating System (RTOS) can be used to control anything from a small motor to a complex industrial drive with the utmost reliability. This article presents the implementation of a Digital PI (Proportional+Integral) Controller on a PC running RTAI (Real Time Application Interface). A system developed on Linux-2.4.24 kernel patched with RTAI-3.1 was used for the speed control of a 12V DC motor.

Control systems Control systems can be broadly divided into two: open loop systems and closed loop systems. Systems which do not automatically correct the variations in their output are open loop systems. This means that the output is not fed back to the input for correction. For instance, consider a traffic control system where the traffic signals are operated on a time basis. The system will not measure the density of the traffic before giving the signals thereby making it an open loop system. Consider now a temperature control system, say an air conditioner. The output of this system is the desired room temperature. Obviously, this depends on the time that the supply to heater/cooler remains ON. Depending on the actual temperature sensed by sensors and the desired temperature, corrective measures are taken by the controller. Thus the system automatically corrects any changes in output by monitoring the same; hence, making it a closed loop system. An open loop system can be modified into a closed loop system by providing a feedback. The provision for feedback automatically corrects the changes in the output due to disturbances. So, the closed loop system is also called an automatic control system. The general block diagram of an automatic control system is shown in figure 1.

Figure 1: Closed system block diagram

Automatic controller Automatic controller

Issue 15 A controller is a device introduced into the system to modify an error signal and to produce the required control signal. An automatic controller compares the actual value of the plant output with the desired value, determines the deviation, and produces a control signal. This will reduce the deviation to zero or to a smaller value. According to the manner in which the controller produces the control signal (called control action), controllers are classified as proportional (P), integral (I), derivative (D) or combinations of these (PI, PD and PID). The proportional controller is a device that produces a control signal u(t), which is proportional to the input error signal, e(t) (error signal is the difference between the actual value and the desired value) i.e.: u(t) = Kp * e(t)

where Kp = proportional gain or constant; proportional controller amplifies the error signal by an amount Kp. The drawback of the P-controller is that it leads to a constant steady state error. Integral control is used to reduce the steady state error to zero. This device produces a control signal u(t) which is proportional to the integral of the input error signal: u(t) = Ki * integral { e(t)*dt }

where Ki = integral gain or constant. Integral control means that we are considering the sum of all errors over an interval. So this always gives us a measure of variation over a constant interval. The other choice is derivative control where the control signal (u(t)) is proportional to the derivative of the input error signal (e(t)). We consider the derivative of e(t) at a given instant as the difference between present and previous errors. A large positive derivative value indicates a rapid change in output variable (here the speed of motor). In other words, the rate of change of speed is more. The drawback in integral controller is that it may lead to oscillatory response. For these reasons combination of P, I and D are used. Most (75-90%) of controllers in current use are PID. In this article, I will look at the design of a PI controller. The PI controller looks at: 1. the current value of the error; and 2. the integral of the error over a recent time interval. This not only determines how much correction to apply, but for how long. Each of the above two quantities are multiplied by a â&#x20AC;&#x153;tuning constantâ&#x20AC;? (Kp and Ki respectively) and added together. Depending on the application, one may want a faster convergence speed or a lower overshoot. By adjusting the weighting constants, Kp and Ki, the PI is set to give the most desirable performance. The implementation of software PI controller is discussed later in this article.

Figure 2: A digital PI controller Today, digital controllers are being used in many large and small-scale control systems, replacing analog controllers. Itâ&#x20AC;&#x2122;s now common practice to implement PI controllers in its digital avatar, which means the controller algorithm is implemented in software rather than in hardware. The trend toward digital control is

Automatic controller

Issue 15 mainly due to the availability of low-cost digital computers. Moreover, as the complexity of a control system increases, demands for flexibility, adaptability and optimality increases. Digital computers used as compensator are a better option in such cases. A general purpose computer, if used, lends itself to time-shared use of other control functions in the plant. The systems that ensure timing requirements are termed as real-time systems. An appropriate operating system should be used to satisfy time constraints

Why real-time? Using general purpose computers is a disadvantage, if the operating system employs various tasks to be executed on a time shared basis. An example of such an operating system is GNU/Linux, where the time constraints required by the control system are not met. The systems that ensure timing requirements are termed as real-time systems. An appropriate operating system should be used to satisfy time constraints.

Real-time systems A real-time system can be defined as “a system capable of guaranteeing the timing requirements of the processes under its control”. It must be fast and predictable. Fast means that it has a low latency (that is, it responds to external, asynchronous events in a short time). The lower the latency, the better the system will respond to events which require immediate attention. Predictable means that it is able to determine a task’s completion time with certainty. It is desirable that time-critical and non time-critical activities coexist in a real-time system. Both are called tasks, and a task with a hard timing requirement is called a real-time task.

What is RTAI? RTAI stands for Real-Time Application Interface. Strictly speaking, it is not a real-time operating system, such as VXworks or QNX. It is based on the Linux kernel, providing it with the ability to make itself fully pre-emptable. RTAI offers the services of the Linux kernel core, adding the features of an industrial real-time operating system. It lets you write applications with strict timing constraints for your favourite operating system. Like Linux itself, this software is a community effort.

PC parallel port as analog I/O interface The PI Controller described in this article uses the PC parallel port as an analog I/O interface. Just two bits are used as analog interfaces through a technique called Pulse Width Modulation (PWM). This technique allows an analog interface to be built without the use of A/D and D/A converters. Analog voltages and currents can be used to control processes directly. As intuitive and simple as analog control may seem, it isn’t always economically attractive or practical. Analog circuits tend to drift over time and can, therefore, be very difficult to tune. By controlling analog circuits digitally, system costs and power consumption can be drastically reduced. Pulse Width Modulation (PWM) is a powerful technique for controlling analog circuits with digital signals, and is a way of encoding digitally analog signal levels. The duty cycle of a square wave is modulated to encode a specific analog signal level, as shown in figure 3.

PC parallel port as analog I/O interface

Issue 15

Figure 3: Pulse Width Modulation (PWM) The PWM signal is still digital because, at any instant, it is either on or off. The relationship between the on-time and the off-time varies according to the analog level to be represented. Given a sufficient bandwidth, any analog value may be encoded with PWM.

Figure 4: PWM generation wave form In this work, the PWM technique is used to generate the gating signals for the Chopper circuit. The speed of a DC motor can be controlled by varying its input voltage. A DC Chopper circuit can do this job.

DC Chopper A chopper is a static device that converts fixed DC input voltage to a variable DC output voltage directly. A chopper may be thought of as a DC equivalent of an AC transformer, since they behave in an identical manner. Chopper systems offer smooth control, high efficiency and fast response. The power semi-conductor devices, used for a chopper circuit, can be power BJT and power MOSFET. Like a transformer, a chopper can be used to step up or step down the fixed DC input voltage. Here, for motor speed control, a step down chopper circuit with a power MOSFET is used. Itâ&#x20AC;&#x2122;s basically a high speed semi-conductor switch. It connects source to load and disconnects the load from source at a high speed, according to the gating signal. The average load voltage is given by the formulas in figure 5.

DC Chopper

Issue 15

Figure 5: Chopper equation

Figure 6: Chopper circuit Vs is the source voltage, Ton is the on-time, Toff is the off-time, T = Ton + Toff is the chopping period and Ton/T is the duty cycle.

Preparing the parallel port Although the parallel interface is simple, it can trigger interrupts. This capability is used by the printer to notify the lp driver that it is ready to accept the next character in the buffer. A simple outb() call to a control register makes the port ready to generate interrupts; the parallel interface generates an interrupt whenever the electrical signal at pin 10 (the so-called ACK bit) changes from low to high. The simplest way to force the interface to generate interrupts is to connect pins 9 and 10 of the parallel connector. A short length of wire inserted into the appropriate holes in the parallel port connector on the back of the system will create this connection. Pin 9 is the most significant bit of the parallel data byte.

Real-time speed controller Figure 7 illustrates the block diagram representing the closed loop speed control of a 6V DC motor controlled using a DC Chopper circuit. The switching duty ratio of the chopper is controlled using gating signals (PWM) from the PC parallel port. The PC is running the speed sensing (shaft encoder) module to read the current speed measured by the encoder. This device works by generating a square wave pulse produced when a perforated disc (keyed onto the shaft of the motor) cuts an IR beam. This square wave pulse train is fed back to the 10th pin of the parallel port. The controller program is running as a hard, real-time user program. The well implemented PI controllers are usually hard, real-time systems. I have implemented a Digital PI Controller in a standard PC machine running Linux/RTAI (Linux-2.4.24 kernel patched with the RTAI-3.1)

Real-time speed controller

Issue 15

Figure 7: Real-time speed controller The well implemented PI controllers are usually hard, real-time systems. I have implemented a Digital PI Controller in a standard PC machine running Linux/RTAI (Linux-2.4.24 kernel patched with the RTAI-3.1). The I/O was done by PWM through two pins of the parallel port: one for input and the other for output, dispensing with the need for A/D converter which is the most expensive component in a data-acquisition system. The PWM coding/ decoding has been dealt with by software routines implemented in Linux threads with LXRT RTAI hard, real-time constraints. The main routine, which is responsible for the actual PI calculation has also been implemented in a thread with hard, real-time constraints. The communication between threads was made through global variables. The controller program produces the necessary actuating signal (in terms of a PWM pulse), which, in turn, depends on an error signal produced by the difference of the set speed (a constant in the program) and the present speed. The chopper varies the average DC output voltage, used to control the speed of the DC motor. The motor used here is a small 6/12V motor commonly found in most tape recorders. The following sections present the speed sensing (shaft encoder) module and the controller program followed by a brief description of the same.

Speed sensing (shaft encoder) module Once interrupts are enabled, the parallel port generates an interrupt whenever there is low to high transition at pin 10 (ACK bit). This module includes the operations that should be performed when the interrupt occurs. The init_module function registers a handler for interrupt number 7 (IRQ number). In the first execution of the function lpt_irq_handler (void), the count is zero and the control enters the if statement. The 64 bit time-stamp-counter (tsc) value is obtained by the rdtsc macro function defined in msr.h. By getting two tsc values at consecutive interrupts, the number of CPU clock cycles in between these interrupts can be calculated, and hence so can the frequency. Frequency, once measured, is communicated with a User space program via a FIFO. Thus, the User space program can calculate the real-time speed. Here is the code which implements this idea: (Code) //#define MODULE #include <linux/kernel.h> #include <linux/module.h> #include <rtai.h> #include <rtai_sched.h> #include <rtai_fifos.h> #include <asm/io.h> #define rdtsc(low,high) \ /*define rdtsc(low,high)*/ __asm__ __volatile__("rdtsc" : "=a" (low), "=d" (high)) #define TICK_PERIOD 1000000 /*1ms*/ #define TASK_PRIORITY 5 #define STACK_SIZE 4096 #define FIFO 0 #define LPT_IRQ 7 #define LPT_BASE 0x378 static unsigned int cpu_clock_count; static RT_TASK rt_task;

Speed sensing (shaft encoder) module

Issue 15 static RTIME tick_period; static void fun(int t) { int count=0; while(1) { rtf_put(FIFO, &amp;cpu_clock_count, sizeof(cpu_clock_count)); if(count==450) { rtf_reset(FIFO); count=0; } rt_task_wait_period(); } } static void lpt_irq_handler(void) { static unsigned int low1,high1,low2,high2,count=0; if(count==0) rdtsc(low1,high1); else { rdtsc(low2,high2); if(count==100) count=1; cpu_clock_count=low2-low1; low1=low2; high1=high2; } count ++; rt_ack_irq(7); } int init_module(void) { RTIME now; int result; outb(0x0,LPT_BASE); result = rt_request_global_irq(LPT_IRQ, lpt_irq_handler); if(result) { rt_printk("rt_request_global_irq [7]......[failed]\n"); return result; } rt_enable_irq(LPT_IRQ); outb(inb(0x37a)|0x10, 0x37a); rt_printk("rt_request_global_irq [7]..... [ OK ]\n"); rt_set_periodic_mode(); rtf_create(FIFO, 4000); rt_task_init(&amp;rt_task, fun, 0, STACK_SIZE, TASK_PRIORITY, 0, 0); tick_period = start_rt_timer(nano2count(TICK_PERIOD)); now = rt_get_time(); rt_task_make_periodic(&amp;rt_task, now + tick_period, tick_period); return 0; } void cleanup_module(void) { outb(inb(0x37a)&amp;~0x10, 0x37a); rt_printk("rt_free_global_irq [7].... [ OK ]\n", rt_free_global_irq(LPT_IRQ)); stop_rt_timer(); rt_printk("stop_rt_timer ..... [ OK ]\n"); rtf_destroy(FIFO); rt_busy_sleep(10000000); rt_task_delete(&amp;rt_task); }

The rt_request_global_irq and rt_enable_irq functions together instruct the RTAI kernel to service IRQ 7 (the parallel port interrupt). After compiling this module, insert it into the kernel address space using insmod. Check /proc/modules to ensure that the above has been inserted.

Controller program The controller program is responsible for three things: 1. for reading the current cpu_clock_count from the speed sensing moduleâ&#x20AC;&#x201D;i.e., from the FIFOâ&#x20AC;&#x201D;which is stored in an array for future reference;

Controller program

Issue 15 2. for the PI task; and 3. for encoding the PI digital output—which is stored in a variable—into an external PWM wave signal, which is the gating signal for chopper circuit. The algorithm used for implementing PI controller is summarised as follows: 1. Create a periodic task with period Tc. 2. Read the state of the parallel input pins. 3. Compute control signal and update PWM signal. 4. Update output-write thread. 5. Wait for the next task period, Tc. The main function creates five hard, real-time threads, which are shown in listings 1-4. An LXRT task starts out as an ordinary POSIX thread. It gets transformed to a hard, real-time thread by calling the function available as part of the LXRT library. The effect of this transition is that the thread is no longer managed by the normal Linux Scheduler—it comes under the supervision of the RTAI Scheduler, which guarantees that its timings do not get disturbed by activities taking place within the Linux kernel or in other user-space threads. The main() thread runs as a low priority task. #include <linux/module.h> #include <rtai.h> #include <rtai_sched.h> #include <rtai_fifos.h> #include <asm/io.h> #innclude <rtai_lxrt.h> #include <pthread.h> #include <stdio.h> #include <math.h> #define FIFO 0 #define BASE 0x378 #define CPU_CLOCK 1675888202.0 /* Enter your Cpu clock frequency*/ #define Kp 3 #define Ki 0.01 #define set_point_speed 0 /* Enter set point speed*/ #define TIMERTICKS 5000000 /*5ms*/ volatile int stop = 0; static float cofk,Vm_sin=5,Vm_tri=10;/*Vm_=Vmax*//*n, no.of samples taken=400*/ static int eof[100],n=400,rt_speed; static RTIME tick_period,now; unsigned int cpu_clock_count; static float Vsin_k [100], Vtri_k [100]; static float Mag_level_Var_sin=64.0, Mag_level_Var_tri=256.0;/*Magnitude levels*/ int main () { RT_TASK *task; pthread_t th0,th1,th2,th3,th4; task = rt_task_init(nam2num("main"), 10, 4096, 0); if(task == 0) exit(1); rt_set_periodic_mode(); tick_period = start_rt_timer(nano2count(TIMERTICKS)); now = rt_get_time(); pthread_create(&amp;th0, pthread_create(&amp;th1, pthread_create(&amp;th2, pthread_create(&amp;th3, pthread_create(&amp;th4,

0, 0, 0, 0, 0,

rt_task_pid, 0); rt_task_in_put, 0); rt_task_out_put, 0); rt_task_sin, 0); rt_task_tri, 0);

getchar(); stop = 1;

Controller program

Issue 15 pthread_join(th0, 0); pthread_join(th1, 0); pthread_join(th2, 0); pthread_join(th3, 0); pthread_join(th4, 0); stop_rt_timer(); rt_busy_sleep(10000000); rt_task_delete(task); return 0; }

Follow this link to download the code.

The PI task The PI task is the core of the application, where the controller calculation is actually made. The PI controller receives, as input, the error given by e(t) = r(t) - y(t), and computes the control variable c(t) which is its output. The control signal c(t) is calculated from the discrete PI equation given in figure 8.

Figure 8: PI equation The PI thread looks at: 1. the current value of the error; 2. the integral of the error over a recent time interval; 3. the compute control signal and update PWM signal; and 4. the update output-write thread This thread has been assigned the fifth highest priority among the five hard, real-time tasks. The code is given in Listing 1. void* rt_task_pid(void *arg) { RT_TASK *task; static int n=0,k=0,sum=0; task = rt_task_init(nam2num("rtl0"), 4, 4096, 0); rt_task_make_periodic(task, now + tick_period, tick_period); rt_make_hard_real_time(); while(!stop) { eof[k]=set_point_speed - rt_speed; while(n<=k) { sum=sum + eof[n]; n++; } cofk=Kp*(eof[k]+(Ki/Kp)*sum); if(cofk&gt;0) Vm_sin=Vm_sin+0.5; else if(cofk<0) Vm_sin=Vm_sin-0.5; n=0; k++; sum=0;

The PI task

Issue 15 if (k==99) k=0; rt_task_wait_period(); /*5ms*/ } rt_make_soft_real_time(); rt_task_delete(task); }

Task rt_task_out_put Listing 2 shows a hard, real-time LXRT program generating a PWM of frequency 50Hz on a parallel port pin to feed the motor drive (DC Chopper). The PWM is generated by comparing the instantaneous values of a sine wave and a triangular wave of the required frequency. The instantaneous values of the two waves are generated in the threads given in Listing 3. These values are compared in the thread rt_task_out_put to generate the PWM. This thread has been assigned the third highest priority. void* rt_task_out_put(void *arg) { RT_TASK *task; static int i; task = rt_task_init(nam2num("rtl1"), 2, 4096, 0); iopl(3); rt_make_hard_real_time(); while(!stop) { if(Vtri_k [i]&gt;=Vsin_k [i]) { outb(0x1,0x378); rt_sleep(nano2count(50000)); /*50us*/ } else if(Vtri_k [i]<Vsin_k [i]) { outb(0x0,0x378); rt_sleep(nano2count(50000)); /*50us*/ } i++; if(i==99) i=0; } rt_make_soft_real_time(); rt_task_delete(task); }

Listing 3 below shows the code for the generation of sinusoidal and triangular function values used in the output thread for generating the PWM gating signals. /*To Generate Sine Wave*/ void *rt_task_sin (void *arg) { RT_TASK *task; static float f = 50, td, t, Vsin_t, s; /*frequency =50Hz*/ /*s = Step size*/ /* td = Time delay between two samples*/ static float Auto_correction; static int i,j; task = rt_task_init(nam2num("rtl1"), 1, 4096, 0); s = 2*Vm_sin/Mag_level_Var_sin; td = 1/(f*n); Auto_correction = (Mag_level_Var_tri - Mag_level_Var_sin)/2; rt_make_hard_real_time(); while (!stop) { t = i*td; Vsin_t = Vm_sin*sin(2*M_PI*f*t); Vsin_k [j] = (Vsin_t+Vm_sin)/s + Auto_correction; rt_sleep(nano2count(50000)); /*50us*/ i++; j++ if(i == (n-1)) i=0; if(j==99) j=0; } rt_make_soft_real_time(); rt_task_delete(task); } /*To Generate Triangular Wave*/ void *rt_task_tri (void *arg) { RT_TASK *task;

Task rt_task_out_put

Issue 15 static float f=100, td, Vtri_t, s, t;/*frequency =100Hz*/ /*s = Step size*/ /* td = Time delay between two samples*/ static float SLOPE; static int i,j; task = rt_task_init(nam2num("rtl2"), 0, 4096, 0); SLOPE = 4*Vm_tri*f; td=1/(f*n); s=2*Vm_tri/Mag_Control_Var_tri; rt_make_hard_real_time(); while (!stop) { t=i*td; if (Vtri_k [j] = Mag_Control_Var_tri) { Vtri_t = 1 * SLOPE * t; Vtri_k [j] = (Vtri_t+Vm_tri)/s - Mag_Control_Var_tri/2; rt_sleep(nano2count(25000)); /*25us*/ i++; } else { Vtri_k [j]= 0.0; i=1; } j++; if(j==99) j=0; } rt_make_soft_real_time(); rt_task_delete (task); }

Task rt_task_in_put This task continuously reads values from the FIFO. (Note that the shaft encoder writes real-time speed to the FIFO) and stores it to the variable cpu_clock_count. The value of this variable is converted to speed in RPM and is stored in rt_speed. This thread has been assigned the second highest priority. The code for this process is shown in Listing 4 /*The Feed back Task*/ void* rt_task_in_put (void *arg) { RT_TASK *task; task = rt_task_init(nam2num("rtl3"), 3, 4096, 0); iopl(3); rt_task_make_periodic(task, now + tick_period, tick_period); rt_make_hard_real_time(); while(!stop) { rt_speed=CPU_CLOCK*60/(float)cpu_clock_count; rtf_get(FIFO, &amp;cpu_clock_count, sizeof(cpu_clock_count)); rt_task_wait_period(); /*Tc =5ms*/ } rt_make_soft_real_time(); rt_task_delete(task); }

The real-time speed of the motor can be either output (by running the code below as a thread) or by running main() as a GUI application (by using FLTK/GTK). /*Print Real Time Speed*/ void* task_show_report(void *arg) { while(!stop) { printf("%d\n",rt_speed); } }

The complete controller code can be downloaded from here

Task rt_task_in_put

Issue 15

Conclusion The above experiment illustrates the use of a PI controller for the closed loop speed control of a small DC motor, which can usually found in tape recorders. The performance of this PI controller was found to be acceptable for such a motor. However on attempting to model a converter along similar lines for a real large scale industrial drive, I found that the large number of control parameters required a more in depth and detailed controller design, using the concepts of control system theory. “Value your freedom, or you will lose it, teaches history.‘Don’t bother us with politics’, respond those who don’t want to learn.”—Richard Stallman

Biography Sreejith Nair (/user/9527" title="View user profile.): Sreejith is a Linux enthusiast living in India. He has completed a B-Tech in Electrical and Electronics Engineering at Govt. Engineering College, Thrissur (Kerala, India). He is currently working in Trivandrum, India as an Embedded Systems Engineer for inDSP Audio Technologies Pvt. Ltd. His interests include Linux kernel programming, device drivers, embedded systems, robotics and process control. He spends his free time reading books on Linux and exploring the same.

Copyright information Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html. Source URL: http://www.freesoftwaremagazine.com/articles/digital_speed_controller_using_rtai_linux

Conclusion

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Hibernate feature in Linux Reduce TUX’s metabolic activity! By Bandan Das For a long time, hibernation has been associated with proprietary operating systems. Now, the feature has started taking the front row in GNU/Linux systems too. What’s more, it’s fast, flexible and is a real time saver!

Introduction Software hibernation or suspend-to-disk is a very productive operating system feature. It has many advantages over a cold system start or reboot. Not only does the hibernated system load up faster, the user also gets back to his previous session without much effort. This is important because of the time the user saves by avoiding the need to set up a work session again. So, how is hibernation different? It is different because it saves the contents of the RAM (Random Access Memory) into some non-volatile storage medium before shutting down the system. Consequently, when the system boots up again, the operating system loads the suspended image from disk to get back to the previously stored session. Software suspend, as hibernation is popularly known in Linux systems, has evolved a lot in present day kernels. It is constantly getting more and more stable and useful. Some implementations are available in the kernel while one is available as a patch that can be applied to the 2.4 or the 2.6 series. This article is meant to introduce you to the implementations available and the features that they provide. Not only does the hibernated system load up faster, the user also gets back to his previous session without much effort

History and available methods in Linux The history of software suspend in Linux can be traced to Gabor Kuti and Pavel Machek who developed a patch for the 2.2 series known as swsusp. It was probably an abbreviation for swap suspend as the snapshot image was written to the active swap partition of the system. Gabor used to maintain these patches on his personal page. The code base for this patch became the basis for three different implementations that we have today: swsusp, suspend2 and uswsusp. swsusp was first incorporated in the 2.5.18 kernel. It subsequently became a part of the 2.6 stable release. Later, a fork of swsusp, known as pmdisk evolved. The difference between swsusp and pmdisk was that the latter’s code was much cleaner and well structured. Moreover, while swsusp used the /proc interface, pmdisk used the /sys interface. pmdisk cleanups were eventually merged into the swsusp code base. Among all the available methods, the one that generates the maximum excitement is suspend2 by Nigel Cunningham. It introduces some important changes in the way the snapshot image is handled and is also noticeably faster. suspend2 is currently available as a patch for 2.4 and 2.6 kernels and will hopefully be merged into the mainline kernels in future. Also worth mentioning is uswsusp, abbreviation for userland-swsusp. It is an implementation of an interface for software suspend in the user space. All kernels starting from 2.6.17 have uswsusp merged in them. The module gives access to software suspend functionalities by exposing common file operation

History and available methods in Linux

Issue 15 methods that can be used to operate on a character device. This is only useful for developers who would like to write their software suspend applications in user space and hence I won’t delve into this topic in detail here.

swsusp How swsusp works swsusp works by taking a snapshot of the contents of the RAM. This snapshot or suspend image is copied and then saved in an atomic operation fashion to the active swap partition. Upon rebooting, if a valid image signature is found on the active swap, the image is loaded and the snapshot transferred back to memory. This gets back the saved state to the user. The following illustration shows the sequence in simple terms: • User requests for software suspend • All the running processes are given the suspend signal • The devices are frozen so that they don’t change the system state when the snapshot is taken • The memory image is atomically copied with interrupts disabled • Frozen devices are awakened so that the image can be written • The image is written to swap • Devices are suspended and the system powered off The resume process involves just the inverse of the above steps.

Configuration and running Configuring and running swsusp is a breeze since it is a part of the 2.6 vanilla kernel. If swsusp has not been compiled in your running kernel, you will have to enable it and recompile your kernel. It is always a good idea to compile a fresh stable version instead of tinkering with the running version because that way you always have a backup with you. To enable swsusp, bring up the kernel configuration window: $cd /your/linuxsourcedir $make xconfig

Enable Software Suspend under Power Management Options as shown in figure 1.

Figure 1: Enabling swsusp You can optionally add your swap partition path in Default resume partition. This step appends a resume parameter to the kernel section in grub.conf. You may also enable Encrypt suspend image that would encrypt the image stored in the swap partition. The key that is used for encryption is stored temporarily and deleted after resuming. swsusp will not work with Symmetric Multi Processing (SMP) enabled in your kernel. So, you will have to disable Symmetric Multi Processing_support under _Processor types and features to enable

swsusp

Issue 15 the swsusp module as shown in figure 2.

Figure 2: Disabling SMP As swsusp uses the AES module to encrypt the suspend image, you will have to select Cryptographic API and AES cipher algorithms as built-ins into the kernel (not as modules as they wonâ&#x20AC;&#x2122;t be loaded). Refer to figure 3.

Figure 3: Enabling AES `swsusp` will not work with Symmetric Multi Processing (SMP) enabled in your kernel Now compile your kernel and if the compilation succeeds, install your kernel. $make $make modules_install $make install

You should also check whether a resume parameter has been appended to the newly installed kernelâ&#x20AC;&#x2122;s entry in grub.conf. If not, you can add it manually as resume=/your/swappartition. (Refer to figure 4)

Configuration and running

Issue 15

Figure 4: Editing grub.conf The default location of grub.conf is /etc. It may be different for your system. Finally, to hibernate, type the following command : $echo platform > /sys/power/disk; echo disk > /sys/power/state

If everything is fine, your system should enter the hibernate process and after saving the image should power off. You should again restart your system to find out if it really worked. The boot sequence should load the suspended image and get back to your previous session. There are a few other important points worth mentioning: • To limit the suspend image size to N bytes, type: $echo N > /sys/power/image_size

where N is the number in bytes. • Instead of using echo platform, you may use echo shutdown to hibernate your system. (Refer to the command above) • If the system becomes too slow after a resume, you may try turning off the swap and then turning it on again: $swapoff -a $swapon -a

• You may try a SysVInit patch that would enable you to suspend using: $shutdown -z now

The patch is available here • Append the noresume parameter to your kernel if you would like to boot normally but be warned that something might go wrong!

suspend2 How suspend2 works The fundamentals behind the working of swsusp and suspend2 are almost the same. However, there are some major differences in the way suspend2 does things and the flexibility it provides to the user. suspend2 first saves the lru (least recently used) pages in memory to the disk backstore, then does an atomic copy of the rest of the memory and then saves that. Since it is not doing an atomic copy of the whole of the memory, the process becomes much faster than swsusp and the image size is not limited to just half of

suspend2

Issue 15 the memory. The module sees to it that once we start saving the page lists, no pages are added or deleted, hence maintaining the integrity of the saved data. To summarize the steps: • User requests for system hibernation • suspend2 waits for the number of processes in their critical paths to reach zero • New processes are prevented from entering states that can cause deadlocks or inconsistent states • The page cache is saved to disk • Drivers are suspended and the processor state is saved • The rest of the memory is atomically copied and then saved to disk • The system is shut down The resume process is just the opposite of the above steps.

Configuration and running To enable suspend2, you will have to patch and compile your kernel. Patches for many versions are available on the suspend2 downloads page. However the stable version is suspend-2.2.5 for 2.6.16.9 kernel. Using the stable release reduces the chances of breaking the kernel compilation. First of all, you need to apply the suspend2 patches using the apply script supplied. $tar -jxvf linux-2.6.16.9.tar.bz2 $tar -jxvf suspend2-2.2.5-for-kernel-2.6.16.9.tar.bz2 $cd linux-2.6.16.9 $../suspend2-2.2.5-for-kernel-2.6.16.9/apply

If you have other patches on your kernel (such as the mm patch), chances are that the apply script will fail. In that case, patching the files individually would be a good idea. The patch file that refuses has to be manually applied by looking through the diff file and replacing code at the right places. After patching, you will have to configure your kernel. As shown in figure 5, enable Suspend2 under Power Management Options. From the options that show up in the Option window, enable File Writer and Swap Writer. You may ignore the rest of the options but feel free to go through their descriptions.

Figure 5: Configuring suspend2 By default, suspend2 uses the LFZ modules for compression and encryption which may be built into the kernel by selecting Cryptographic API and LFZ compression algorithm under Cryptographic Options.

How suspend2 works

Issue 15

Figure 6: Enabling LFZ Once you are done selecting all other modules relevant to your system, compile your new kernel and install it. suspend2 gives you the flexibility to choose the location to save the suspend image. You can either choose the file writer or the swap writer for this purpose. While the filewriter can save the image to a normal file, the swap writer can save the image either to a swap partition or to a swap file. Unlike swsusp, suspend2 can write to any swap partition! It need not be the system’s active swap partition. I’ll cover the above three steps one by one. Unlike swsusp, suspend2 can write to any swap partition! It need not be the system’s active swap partition • Writing to a normal file: First of all create a file that is at least equal in size to your system RAM. Here, I create a 512 MB file called suspend-image. $dd if=/dev/zero bs=1M count=512 >> suspend-image

Next supply this file to the suspend2 filewriter target. $echo suspend-image > /proc/suspend2/filewriter_target

Finally, you will have to get the actual physical offset of this file and append it to the kernel section in grub’s configuration file (grub.conf). $cat /proc/suspend2/resume2

The above command gave me the following output: file:/dev/hda5:0x3c000 So I need to append resume2=file:/dev/hda5:0x3c000 in my grub.conf. (Refer figure 7). Don’t just copy-paste the above output as it will definitely be different for your system.

Configuration and running

Issue 15

Figure 7: Editing grub.conf â&#x20AC;˘ Writing to a swap partition: This is the easiest of them all and is recommended for first timers. You just need to append your swap partition path to the resume2 parameter for your kernel. Find out your swap partition: $/sbin/fdisk -l | grep "swap"

and edit grub.conf accordingly. (Refer to figure 8)

Figure 8: Editing grub.conf â&#x20AC;˘ Writing to a swap file: The steps to this are similar to writing to a normal file except that here, you create a swap file instead of a normal one. To summarize: $ $ $ $

dd if=/dev/zero bs=1M count=512 >> swap-suspend-image /sbin/mkswap swap-suspend-image 512 sync swapon swap-suspend-image

The output of the following command tells you what to append to the resume2 parameter. $cat /proc/suspend2/headerlocations

The following output was obtained: For swap partitions, simply use the format: resume2=swap:/dev/hda1. For swapfile `/home/bandan/PTPRO/FSM/swap-su`, use resume2=swap:/dev/hda2:0x1d92678

The second output is what you would append in grub.conf.

Configuration and running

Issue 15 After choosing from the filewriter and the swapwriter (a combination of both is also possible), you should take a look at the hibernate script. This is a shell script that greatly simplifies the hibernation sequence. Get the latest version from the suspend2 Downloads page.The script is available as RPM, SRPM, deb or tar.gz. Choose one of them and install it. By default, it should install in /usr/local/sbin. If you use initrd (common in redhat distributions), you need to tell it about suspend2. Open the mkinitrd script in a file editor and go below the line that says: echo "echo Mounted /proc filesystem" >> $RCFILE

Add the following: echo "echo > /proc/suspend2/do_resume" >> $RCFILE

The default location of mkinitrd is /sbin. The advantage of suspend2 lies in the flexibility it offers. A set of configuration files are provided by default in /etc/hibernate. Hopefully, suspend2 will work “out of the box” but it is always a good idea to go through these configuration files if you want to tweak suspend2 to get the best out of it. Here is a brief summary of all the configuration files suspend2 offers. The verbose commented sections give a detailed explanation of all the available options. • blacklisted-modules: Lists all the modules that have been reported to cause problems while suspending or resuming. • hibernate.conf: The file actually read when the hibernate script is executed. • disk.conf: Used when using the hibernate script to suspend using swsusp. • common.conf: Includes list of common configurable options. • ram.conf: Used when suspending to RAM instead of disk. (useful for laptops) • suspend2.conf: Configurable options for suspending to disk with suspend2. Now all you have left to do is reboot into your new kernel. Keep a close eye on the system output while booting up. You should see something like Suspend2 enabled. Finally to hibernate, simply type the following as root: $/usr/local/sbin/hibernate

The system should enter the hibernate sequence and shutdown. Reboot again to check everything is fine. If you are having problems with the filewriter, you may switch to the swap writer instead. It is usually convenient to try suspending to the swap partition first because that’s the easiest. Then, you may try suspending to a normal file or a swap file. If the hibernate script doesn’t work for you for any reason, you should try the following: $/usr/local/sbin/hibernate --bug-report > suspend-debug

Go through the contents of suspend-debug and see what went wrong. The most important information will be towards the end of the file. If you find its contents meaningless, you would be better off sending it to the suspend2 mailing list. Most probably, someone will help you out.

Eye candy Not only does suspend2 work differently, you can also make it look different. You have the option of a text based user interface or a graphical interface to show during the hibernate sequence. Download and install the Suspend2-User-UI package from suspend2 website. Additionally, obtain a theme from here and extract it to /etc/splash. Create a symbolic link to it and name it as suspend2. $ mkdir /etc/splash (in case it does not exist) $ cd /etc/splash && tar -zxf theme.tar.gz $ ln -s theme/ suspend2

Eye candy

Issue 15 Add the appropriate entry in suspend2.conf depending on what user interface you would like to have. For text UI ProcSetting userui_program /usr/local/sbin/suspend2ui_text

For Graphical UI ProcSetting userui_program /usr/local/sbin/suspend2ui_fbsplash

Now hibernate and see if everything goes as expected. If not, you can use the --bug-report option with hibernate to diagnose the problem. hibernate’s log file also provides a lot of information about what went wrong. Its default location is /var/log/hibernate.log.

Some tips and tricks • Using the power button to suspend: First of all, make sure you have compiled your kernel with acpi support. Then install acpid and add it to the default run level. $ rc-update add acpid default

Edit /etc/acpi/default.sh as follows: #!/bin/sh # Default acpi script that takes an entry for all actions set $* group=${1/\/*/} action=${1/*\//} case "$group" in button) case "$action" in power) /usr/sbin/hibernate & ;; *) logger "ACPI action $action is not defined" ;; esac ;; *) logger "ACPI group $group / action $action is not defined" ;; esac

• If you would like to reboot instead of shutting down your system, simply press “R” during the suspend process.

Conclusion suspend2 clearly seems to be the successor to swsusp. It provides a lot more features than any other available method and is also very configurable. It allows you to store the suspended image almost anywhere. A more interesting feature which is in the TODO list is suspending to a NFS share! However, kerneltrap reports Nigel as saying that he will devote less time to suspend2 which may prevent its merging into the vanilla kernel. The three different methods available target three different types of users. If you just want your system to hibernate and have no desire to tinker with your kernel, swsusp is definitely for you. On the contrary, suspend2 gives you greater speed and flexibility. If you are one of those who fancies writing his own userspace program, go for uswsusp. Get set and take your pick!

Conclusion

Issue 15

Biography Bandan Das (/user/12942" title="View user profile.): Bandan is an active user and supporter of Open Source Softwares. He is currently pursuing Master of Science in Computer Engineering from Syracuse University, NY. He loves programming in C and LISP and loves playing his guitar when he wants a break from gadgets and geek stuff.

Conclusion

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Creating a managed website—Part 2 Focus on the message not the tools. Selecting a CMS, installing it and promoting your site. By Graham Oakes Free software Content Management Systems (CMS) are capable of running most websites these days. Indeed, low initial costs and strong community-based support mean that many sites which can’t afford a proprietary CMS can now benefit from the facilities a CMS provides. In the first part of this article I looked at how a CMS might help and what you need to do to define your site’s target audience and structure. Now I’ll get down to the nitty gritty of selecting a CMS, installing it and setting up and promoting your site.

How do I choose the CMS? What CMSs are available? There are a wide range of CMSs available. Wikipedia lists about 130 free software CMSs. CMS Review lists more. CM Pros, a community of content management professionals, is developing a database of CMSs and their features to help narrow down the search, but the best source of up-to-date information about any particular CMS is currently the website for that CMS.

Does the CMS do what I want it to do? Start by thinking about content creation. If you have a large team who collaborate extensively to create content, and then subject it to rigorous review before publication, then you probably need a fairly sophisticated CMS such as Plone, Apache Lenya, or Alfresco. These are more complex to configure and learn, but they’re more likely to support the workflows and collaboration tools you need. On the other hand, if one or two of you are running a small site, then a simpler CMS may let you publish content without so many overheads. In this case, something like Joomla, Mambo, openCMS, or Drupal might work for you. Start by thinking about content creation. A large team, may need advanced workflows. For a smaller team, a simpler CMS may work well While you’re thinking about content creation, consider the interface for editors. Will they enter everything via a browser, or will some content be created in OpenOffice.org and then imported? If the latter, then ensure tools are available to do it. If the editors also control the formatting and presentation of the content on the site, then the CMS will need to support their formatting tools. If they simply enter plain text and let templates control the rest, make sure the CMS works well in this mode, and that your designers have the right skills to define the templates and style sheets it needs. Think about other aspects of content delivery too. Do you want to publish to multiple websites, or in a format for mobile browsers? Do you want to support RSS? Do you want to generate features like breadcrumbs? Do you need to put embargoes on press releases or other time-sensitive material? Do you want extra security for some pages? Do you need site search? If so, make sure your CMS supports it.

What technical infrastructure does the CMS need? The CMS will run on a server, and that server needs to sit somewhere. Your options are:

How do I choose the CMS?

Issue 15 1. Buy and host your own server. This gives you complete flexibility, but it means you need a network link with sufficient bandwidth. You also have to manage all the hassle of running the server, dealing with network outages, etc. If you want to focus on content rather than technology, then this may not be the way to go. 2. Co-location. You buy and install the server; your ISP provides space and power for it. This is marginally less complex than (1), but again only really worth considering if you like playing with hardware. 3. Dedicated, hosted server. Here the service provider supplies and sets up the server as well as the space. This is less flexible, as you’ll be restricted to servers that they’re happy to support, but it can be a good way to deliver a high-traffic site without too much technical pain. When choosing a service provider, be clear about what software they support (some will support the database and certain CMSs as well as the operating system). Check what service levels they offer (e.g. what bandwidth they’ll provide; what out-of-hours support; what levels of resilience and security they have in their datacentre and networks; what reporting tools they offer). Finally, all of this comes at a price. If you want higher service levels, you’ll pay for them. 4. Shared, hosted server. In this case you share space on server with other people (although you shouldn’t be able to see each others’ space if the server is set up correctly). Otherwise, this is similar to a dedicated server—slightly less flexible and with lower performance, but a good way to run a low-traffic site. It’s generally a lot cheaper than a dedicated server. A dedicated server is a good option for a high-traffic site. A shared server is cheaper and good enough for a low-traffic site. Be clear about the software the service provider supports and the service levels they offer If you want to stick with your current ISP, check what operating systems and applications they support. You may still have a good choice of CMS.

Consider the administrative interface. The Joomla CMS gives a simple console One final technical consideration: if you want to integrate the CMS with other applications, or if you want to customise it, check it’s written in a language that you can work with. The most common languages are PHP, Java and Perl. As Rick Brenner observes: “Most CMSs have a limited dynamic range. They make some modifications easy, but you still need technical interventions from time to time”. If you’re not happy to work within your CMS’s capabilities, make sure you have the skills to extend them. (Of course, one of the benefits of free software is that such extension is at least possible.)

What technical infrastructure does the CMS need?

Issue 15

To manage Plone, you need to find your way through the mysteries of the ZMI (Zope Management Interface). This is more complex than Joomla, because Plone gives more advanced functionality. Fortunately most users don’t need to see this complexity

What support is available? That brings us onto support. If it’s really important to have someone to beat up when there’s a problem, then there are many very good proprietary CMSs. Free software is supported by people who care about helping people use the software, rather than about how quickly they can get you off the phone. If that’s the type of support you want, check out the community behind the CMS. How rapidly do questions get answered on the discussion boards? Does there seem to be a regular pattern of releases, with multiple developers contributing to each new version? Look for signs of a sizeable and active community. Free software is supported by people who care about helping people use the software, rather than about how quickly they can get you off the phone

How do I set up the CMS? Once you’ve identified the CMS, you can start to set it up. The specific steps to do this differ according to the CMS, but the general stages will be: 1. Set up an account with your hosting provider. Depending on how you’ve chosen to host the CMS, you need to set up your account and ensure you have all the necessary passwords to access your server. 2. Set up the server. You need to install the server that will run the CMS. (For a site that’s handling very large traffic volumes, you may use separate servers for database and CMS. For such a site, you probably have a dedicated technical team…) You also need to install and configure the operating system, and set up network connections, firewalls, etc. 3. Install any software needed to run the CMS. Most CMS need supporting software. For example, Joomla needs the Apache webserver, mySQL database and PHP script processor. Plone needs the Python language processor and Zope application server. Read the installation instructions on your CMS’s website to find out exactly what you need. (If you have trouble understanding the instructions, that’s a sign that either this isn’t the right CMS for you, or that you should use a hosting service with the CMS pre-installed.) 4. Install and configure the CMS. Download the CMS and install it, as per the instructions on the website. You may need to configure a number of parameters such as port assignments, directory settings and site names as you do this. This may mean contacting your hosting provider to get information or to set up the necessary structures. If this seems like a lot of work, that’s because it is. On the positive side, it’s a one-off effort. Once the CMS is installed, you can make site changes (or at least ones the CMS supports) quickly and easily. For a site that doesn’t change very often, the initial effort may not be worth it. For a dynamic site with lots of fast-breaking content, the long term benefits of investing some effort upfront can be substantial.

How do I set up the CMS?

Issue 15 For a dynamic site with lots of fast-breaking content, the long term benefits of investing some effort upfront can be substantial Buying a full hosted service can remove a lot of this initial hassle. You still need to set up and configure your site, but the more technical stuff of setting up servers and installing software is done before you begin. Again, you’ll need to pay for this service.

How do I set up my site? Now the CMS is installed, you can set up your site. The stages will look something like: 1. Create templates to deliver the site design. The pages of your site will probably have common design elements (header and footer, navigation bars, etc) plus space to display the text, images and other content elements. Templates tell the CMS how to assemble all these elements into pages. You’ll also need to define the hierarchy that pulls these pages together into a coherent website. 2. Create forms and workflows for content entry. You may need to define the forms that editors will use to enter content, and the workflows they’ll use to publish this content. (These may be predefined within the CMS. Most CMSs assume you will want a lot of flexibility in the way you present the content to the world, but that your internal editors will be prepared to invest effort in learning the CMS’s ways. If this isn’t true for you, then allocate time for this stage.) 3. Set up user accounts and other administration. The CMS will need accounts for editors and other such users. It’s also a good idea to set up processes like backups at this point, so that you can recover everything if there are glitches while you set up the rest of the site. 4. Migrate existing content. You can now migrate content from your existing site into the CMS. There may be automated tools for this but in many cases, especially for smaller sites, it’s easier to do it manually. Be pragmatic here: if no-one is viewing the content on the current site, is it suddenly going to become more interesting on the new site? If not, don’t invest effort in migrating it. 5. Create new content. While you’re migrating content, you can also start to create new content. It’s important that the site looks full of interesting goodies—nothing turns away people more than an empty-looking site. So make sure you’ve got enough to say to justify the size of the site. And remember, this is just the beginning of an ongoing process of content creation. 6. Launch. When the site is ready, you can launch it. In the simplest case, this just involves switching your URL to point at the new site. You’ll probably need to work with your hosting provider to do this. If you have more complex editorial workflows, then you may have set up a staging area (an area where editors can view content before it goes live, so they can see what it looks like to external browsers). In this case, your launch may also entail moving content from the staging area to the live site. Be pragmatic about content migration: if no-one is viewing the content on the current site, is it suddenly going to become more interesting on the new site?

Promoting the site Your site is up. You’ve had the launch party. Don’t leave it there—you need to promote the site and keep it fresh so that people will keep coming to it. “You’ve made a significant investment in your website. That investment will only bring results if people visit it. Ensure the site is prominent on search engines such as Google, Yahoo and MSN. Consider using other internet marketing channels such as pay per click, email, pod casts and online communities to extend your reach.” Jo Green, Pazang The key is: promotion only works if there’s something to promote. A campaign might drive a one-off surge in traffic, but if your site doesn’t say anything interesting then people will leave quickly, and they won’t come back. Before you invest in promotions, invest in creating valuable and regularly updated content. Once you’ve got content, think about things like: • Search engine optimisation (SEO). There are plenty of people who sell ways to improve your search engine rankings. Be wary. Search engines try very hard to work around any tricks you might use to

Promoting the site

Issue 15 bump up your ranking. You might gain a temporary advantage but it’s unlikely to last. What should you do? Focus on substance: if you have something distinctive to say, other people will link to it and search engines will rank it accordingly. At the same time, don’t make it hard for the search engines. Create meaningful URLs for key pages and include sensible keywords. Register your site with search engines using their (free) facilities, e.g. at Google. “In the past, search engines didn’t track dynamic pages, for a number of technical reasons. Now they do some, but friendly URLs can help. (They’re also easier for people to use!) Create topical pages that aren’t dynamic, use essential keywords on those pages, and then link to local dynamic pages. Then at least the topical pages are likely to be seen.” Nynke Fokma, Moebius • Linking to (and from) other sites. Link trading (networks of sites that exchange links with each other to increase search engine rankings) is like SEO. Be wary. Think of it this way: if someone comes to your site from a totally irrelevant site, how likely are they to be interested in what you’re saying? Invest your energy in building relationships with people who are interested in something that’s relevant to you. When you respect someone, link to their site. Ask them to reciprocate. People (and search engines) will follow these links to see what you’ve got to say. • Other media. Don’t ignore other media. When you send emails, include your URL so people can easily come to your site. Include your URL in any advertising you may be doing. Include it in your letterhead. (Some people still spend time offline.) These things build off each other. Rick Brenner notes “there’s a lot of synergy across channels: if people find your site in a search engine and also see you mentioned in a magazine, then that makes you seem a lot more credible.” • Measure and refine. Monitor who is coming to your site and where they’re coming from. Note which content gets the most viewers. (Your site reports should help here. Make sure your CMS supports the reports you need.) Use this information to refine your content and promotion strategy. “A client of mine compared a website to a shop: you have to re-arrange the window every once in a while.” Willem van den Ende, Living Software BV

Conclusion Setting up a CMS isn’t easy. For a complex site, it can take months of work to clarify requirements, choose a CMS, and set up the site. Even for a small organisation with a simple site, this can take several days of work. But the core message is clear: sites with interesting and regularly updated content are the ones which attract readers. If this matters to you, then a CMS could make the whole content creation process a lot easier. In that case, your initial investment will be repaid well. Over the last few years, free software, with its reduced licence costs and active support community, has significantly reduced the size of that initial investment. CMSs have become a viable option for even very small sites.

Biography Graham Oakes (/user/20" title="View user profile.): Dr Graham Oakes is the principal of Graham Oakes Ltd, a consultancy formed in 2003 to help organisations untangle the complexity within their systems, processes and governance. He helps define business and technology strategies that people will adopt, and then to assure implementation of those strategies. He can be contacted via his web site (http://www.grahamoakes.co.uk).

Conclusion

Issue 15

Conclusion

Issue 15

Published on Free Software Magazine (http://www.freesoftwaremagazine.com)

Secure email servers from scratch with FreeBSD 6 (Part 2) Configuring the core components By Yousef Ourabi In the last article we parted ways after configuring a base FreeBSD system, enabling it with upgrades via cvsup and portsupgrade, and securing it with a simple ipfw2 firewall. The previous article created a solid foundation which this article will build on, covering the configuration of Postfix, amavisd-new, ClamAV, SpamAssassin, MySQL and finally SquirrelMail for web mail. The final setup will have all the bells and whistles of a high end-mail setup: web-mail, anti-virus filtering, spam filtering, and hosting unlimited domains with virtual domains and users stored in MySQL. Postfix is released under the IBM Public License, and not the GNU Public License

Postfix The first and most important component is Postfix, a well known mail transfer agent developed by Wietse Venema at IBM and initially known as the “IBM Secure Mailer”. Venema, is a respected software engineer who also developed the popular security tool “S.A.T.A.N” (Security Administrator Tool for Analyzing Networks). Postfix is released under the IBM Public License, and not the GNU Public License; the “IPL” has been approved as an open source license by both the Free Software Foundation (“FSF”) and the Open Source Initiative(“OSI”); however, it has been declared incompatible with the GPL. Initially Postfix was created in reaction to a long list of security vulnerabilities in Sendmail, the then dominant “MTA”. The direct result of the “security first” mind, Postfix has a well earned reputation for being easy to setup, fast and secure. Postfix has two central configuration files: • main.cf: which configures the “properties” of the mail server such as where user and domain information is stored, or which domains to accept mail for. • master.cf: which configures the “behavior” of the Postfix daemon, such as configuring interfaces to non Postfix programs, and other configuration settings for the Postfix daemon itself. FreeBSD places the configuration files of packages installed from ports under /usr/local/etc; so normally you’ll find the Postfix configuration files under /usr/local/etc/postfix. First of all, you should install Postfix from the ports tree, in the same way that MySQL was installed in the first article.

Installing Postfix using ports cd /usr/ports/mail/postfix make install && make clean

You will then be presented with a dialog box: select TLS and MYSQL.

Installing Postfix using ports

Issue 15

Postfix configuration dialog Note that when MySQL functionality is selected, the default action is to install the MySQL 4.1 client library. If you plan on running a newer version of MySQL, such as 5.0, simply cancel the Postfix installation, install the MySQL client library of your choice, and then re-run the installation. Example: cd /usr/ports/databases/mysql50-client make install && make clean

During the installation of the client libraries you may be prompted for options to the gettext package. It’s not necessary to select any of the options, but feel free to do so if you wish. After Postfix is built, you will be prompted asking if you want to activate it in the mailer.conf file: say “yes”: [Prompt] Would you like to activate Postfix in /etc/mail/mailer.conf [n]? y

Configuring Postfix—main.cf Now that Postfix is installed, it’s time to dive into the most important of the two configuration files, main.cf. In this one file there are essentially two sets of directives: one for the domains the server will be hosting, which being with the virtual prefix; and one for the mail server itself, with lines that begin with my as in myhostname. I’ll be giving in line commentary, so read the configuration file closely.

# These virtual_* directives configure the domains, users, # and aliases this Postfix instance will handle. # Use proxy: for performance virtual_alias_maps = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_alias_maps.cf virtual_mailbox_domains = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_domains_maps.cf virtual_mailbox_maps = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_mailboxes_maps.cf proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps # Where to store the mail virtual_mailbox_base = /usr/local/virtual # Ownership of the mail directory virtual_uid_maps = static:125 virtual_gid_maps = static:125

Configuring Postfix—main.cf

Issue 15

smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_security_options = noanonymous # Secure SMTP-AUTH smtpd_use_tls = yes smtpd_tls_enforce_tls = yes # Uncomment the following line if you only want auth to happen over tsl # smtpd_tls_auth_only = yes # This setups the ssl certificates which I'll configure a little later smtpd_tls_cert_file = /usr/local/etc/postfix/smtpd.crt smtpd_tls_key_file = /usr/local/etc/postfix/smtpd.key # Mostly for MS outlook clients broken_sasl_auth_clients = yes # Built in restrictions smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unauth_destination, reject_unauth_pipelining, reject_invalid_hostname, reject_rbl_client opm.blitzed.org, reject_rbl_client list.dsbl.org, reject_rbl_client bl.spamcop.net, reject_rbl_client sbl-xbl.spamhaus.org # Enables virtual hosting virtual_transport = virtual # Filter with amavis-new which uses clam-av for content_filter=smtp-amavis:[127.0.0.1]:10024

Now, look at the configuration for the actual host itself: # Configure your hostname, and domain names here! myhostname = myhost.mydomain.mytld mydomain = mydomain.mytld myorigin = $mydomain # Which network interfaces to listen on inet_interfaces = all # This allows delivery for mail to root@[host] for system messages mydestination = myhost, myhost.mydomain.mytld unknown_local_recipient_reject_code = 550 # For other systems on network? mynetworks_style = host # # # #

This is what people will see if they telnet to port 25 on your mail server it might be worth placing a warning message, for legal reasons. Systems with banners outlining acceptable use, and the legal actions that will be taken have a slightly stronger case in court.

smtpd_banner = mysqlever.mydomain.mytld ESMTP (Insert witty message here)

The framework for the rest of the article is in place. Even though the system is un-usable in the sense the no users can login to send or receive mail, Postfix is itself configured enough to send and receive mail. The next step I’m going go take will be putting the finishing touches on the master.cf file. The default file may look something like that listed below, but you’ll only need to edit a tiny bit of it, and as such I’ll only show

Configuring Postfix—main.cf

Issue 15 the relevent lines.

Configuring master.cf The file master.cf requires very little tweaking out of the box: # ========================================================================== # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # ========================================================================== smtp inet n y smtpd [...] virtual unix n n virtual [...]

smtp-amavis unix n -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20

smtp

127.0.0.1:10025 inet n n - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

Postfix Virtual Maps Postfix has the notion of “maps”, which is the mechanism used to map data from one form to another. In our case, it’s mapping the data in a MySQL database, using a simple select statement, to certain configuration parameters. The map files contain all the information needed to connect to and query the MySQL database, which contains the user information. Each file contains only one query to the database. Create the files, copy and paste the content, and double check the user name and password used to access the database, which I’ll cover setting up in the MySQL section. Please note that this article is geared towards Postfix 2.3 or newer; however, if you plan on using the configuration presented with an earlier version, the format for mysql_maps may be different, and broken up into several lines, so make sure to double check which version you are working with. The file mysql_virtual_alias_maps.cf looks like this: user = mail_admin password = mail_admin_passwd hosts = localhost dbname = mail query = SELECT goto FROM alias WHERE address='%s' AND active = 1

mysql_virtual_domains_maps.cf:

Configuring master.cf

Issue 15 user = mail_admin password = mail_admin_passwd dbname = mail hosts = localhost query = SELECT domain FROM domain WHERE domain='%s'

mysql_virtual_mailboxes_maps.cf: user = mail_admin password = mail_admin_passwd dbname = mail hosts = localhost query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1

Postfix can store user and domain information in LDAP directories, databases such as MySQL and PostgreSQL or flat files

OpenSSL & Certificates The Postfix configuration listed above makes use of SSL certificates for secure authentication over SSL/TLS. To create an SSL Certificate: openssl genrsa -des3 -out smtpd.key 1024 openssl req -new -key smtpd.key -out smtpd.csr # Remove Passphrase from private key -- make optional? cp smtpd.key smtpd.key.bak openssl rsa -in smtpd.key.bak -out smtpd.key # Generate Self-Signed Certificate openssl x509 -req -days 365 -in smtpd.csr -signkey smtpd.key -out smtpd.crt

MySQL Postfix is extremely flexible, and affords a wide array of options in storing user and domain information. The right choice depends mostly on whatâ&#x20AC;&#x2122;s appropriate for the environment you plan on working in. If you only plan on hosting a single domain, with a handful of users it may make sense to avoid the overhead of maintaining a database; however, if you plan on maintaining multiple domains with hundreds or thousands of users, or simply want the flexibility of manipulating the data in an automated way, LDAP or MySQL is the way to go. An in depth discussion of either MySQL or the way Postfix interacts with it is beyond the scope of this article. The only key concept is that of maps in Postfix, which is discussed towards the end of the Postfix section.

Installing MySQL cd /usr/ports/databases/mysql50-server make install && make clean

You should now have the MySQL database installed. You may need to run the startup script, and make sure it has been enabled in the /etc/rc.conf file.

Creating the mail databases The SQL listing below creates the three tables used by Postfix. Simply follow the steps, create the mail database, and then copy and paste the table creation statements. After that, use the GRANT command to grant access to the mail_admin user configured in the Postfix maps described earlier. Logon to the MySQL database, initially the MySQL root password isnâ&#x20AC;&#x2122;t set:

MySQL

Issue 15 mysql -u root create database mail; use mail; CREATE TABLE àlias` ( àddress` varchar(255) NOT NULL default '', `goto` text NOT NULL, `domain` varchar(255) NOT NULL default '', àctive` tinyint(1) NOT NULL default '1', PRIMARY KEY (àddress`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='Postfix Admin - Virtual Aliases'; CREATE TABLE `domain` ( `domain` varchar(255) NOT NULL default '', `description` varchar(255) NOT NULL default '', àliases` int(10) NOT NULL default '0', `mailboxes` int(10) NOT NULL default '0', `maxquota` int(10) NOT NULL default '0', `transport` varchar(255) default NULL, `backupmx` tinyint(1) NOT NULL default '0', àctive` tinyint(1) NOT NULL default '1', PRIMARY KEY (`domain`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='Postfix Admin - Virtual Domains'; CREATE TABLE `mailbox` ( ùsername` varchar(255) NOT NULL default '', `password` varchar(255) NOT NULL default '', `name` varchar(255) NOT NULL default '', `maildir` varchar(255) NOT NULL default '', `quota` int(10) NOT NULL default '0', `domain` varchar(255) NOT NULL default '', àctive` tinyint(1) NOT NULL default '1', PRIMARY KEY (ùsername`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='Postfix Admin - Virtual Mailboxes';

Grant privileges Run the following command: GRANT SELECT on mail.* to mail_admin identified by 'mail_admin_password';

Inserting information An in depth introduction to SQL is beyond the scope of this article too, but here are some simple insert statements to get you started adding users, aliases, and domains Inserting a user:

insert into mailbox (username, password, name, maildir) values ('my_user_name', 'my_password', 'my_nam

Adding a new alias, for an existing user: insert into alias (address, goto) values ('myaddress@mydomain.com', 'myalias@mydomain.com');

Adding a new virtual hosted domain: insert into domain (domain, description) values ('mydomain', 'my_domain_is_so_cool');

Amavisd-new acts as a proxy, accepting the mail from Postfix, and filtering it through ClamAV and SpamAssassin

Creating the mail databases

Issue 15

Amavisd-new & SpamAssassin Amavisd-new is a high performance interface between Postfix and other mail components. Postfix, as well as most other MTAs can only reliably connect to one other content checker such as an anti-virus or spam package, so in order to use both spam and virus filtering we configure Postix to filter content through amavisd-new, which then filters the email through SpamAssassin and ClamAV. SpamAssassin, is an Apache subproject and is probably the most famous and powerful free software spam filtering program. It uses a wide range of tests to assign an overall score to a piece of mail, if the score is above the acceptable threshold it is considered spam. Amavisd-new comes with SpamAssassin embedded, rather than the stand alone daemon. This simplifies the configuration greatly, since SpamAssassin can be configured in the same place that amavisd-new can. The SpamAssassin directives always begin with sa as in $sa_kill_level_deflt, which sets the threshold above which SpamAssassin will simply drop an email instead of tagging it as spam. Amavisd-new is under /usr/ports/security/amavisd-new. It should be built with the with the spamassassin option, which will automatically build and install SpamAssassin during the amavisd installation.

Installing amavisd-new cd /usr/ports/security/amavisd-new make install && make clean

You’ll be presented with a config dialog. The required options are MySQL, and SpamAssassin. Again, after selecting these, feel free to install other options if you feel adventurous.

Amavisd-new configuration dialog Tip: If you’ve already installed amavisd-new, and wish to re-configure it, start the process with make config, which will re-present you with the configuration dialog.

Configuring Postfix to work with amavisd-new It’s simple, yet important to configure Postfix to filter all mail through amavisd-new, which then acts as a broker and filters the mail through both ClamAV and SpamAssassin. To do so add the following lines to the main.cf file: # filter with amavisd-new which proxies to ClamAV and SpamAssassin content_filter=smtp-amavis:[127.0.0.1]:10024

Amavisd-new & SpamAssassin

Issue 15

Configuring the amavisd-new daemon Amavisd-new is now installed, and Postfix is filtering to it. The only thing left to do is configure the amavisd-new daemon itself. The ports package doesnâ&#x20AC;&#x2122;t create an amavisd-new directory under /usr/local/etc, unlike most of the other packages. Instead, the amavisd.conf is directly under /usr/local/etc. Similar to the Postfix configuration block above, Iâ&#x20AC;&#x2122;ll be mixing explanation tid-bits with the configuration, so make sure to read it thoroughly. # Important lines to check, as I said above, I'm mixing # in my own comments with the configuration file. # Make sure these two lines are commented out, as they # will disable spam and virus filtering # uncomment to DISABLE anti-virus code # @bypass_virus_checks_maps = (1); # uncomment to DISABLE anti-spam code # @bypass_spam_checks_maps = (1); # (no default; customary: vscan or amavis), -u $daemon_user = 'vscan'; # (no default; customary: vscan or amavis), -g $daemon_group = 'vscan'; # Configure this to your domain # a convenient default for other settings $mydomain = 'mydomain.com'; # Configure the FQDN of your mail host $myhostname = 'myhost.mydomain.com'; # Change the local_domains_maps to include all # the mail domains you are hosting @local_domains_maps = ( [ ".$mydomain", '.example2.com'] ); # IF you want email marked as spam to have a different # subject header, change the following line $sa_spam_subject_tag = '***SPAM*** '; # Default is 6.31, if message is above this # it will be dropped, and nothing sent to the user $sa_kill_level_deflt = '20'; # Change the default tag level from 2.0 to undef # This will insure all mail addressed to domains # in @local_domains will get a spam score in the header, spam or not. $sa_tag_level_deflt = undef; # Make sure this matches what's in main.cf # listen on this local TCP port(s) (see $protocol) $inet_socket_port = 10024; # Configure these two lines to receive email alerts # when mail is blocked $spam_admin = 'webmaster@mydomain.com'; $virus_admin = 'webmaster@mydomain.com'; ### CLAM_AV INTEGRATION ### ['ClamAV-clamd', \&ask_daemon, ["CONTSCAN {}\n", "/usr/share/clamav/clamd.sock"], qr/\bOK$/, qr/\bFOUND$/, qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],

Configuring the amavisd-new daemon

Issue 15 ClamAV comes with `freshclam`, an automated process for staying up to date with the latest virus signatures

ClamAV ClamAV is a widely used free software anti-virus package. It is released under the GPL license, and is available for most unix-like platforms. It has many features, but its main goal is an integrated attachment scanner for mail servers. ClamAV is used in a production setting across many private companies, large universities, and government agencies such as: Barracuda Networks, Michigan State University, Brandeis University, Webmail.us, Slashmail, and the US state of Vermont. ClamAV’s configuration file, like that of amavisd-new is directly under /usr/local/etc, and is called clamd.conf. The default configuration is more or less what we need, but it’s good to take a peek through the file anyway. Make sure that mail scanning is enabled: # Example LogFile /var/log/clamd.log LogFileMaxSize 5M DatabaseDirectory /usr/share/clamav # UNIX Socket which other programs will use # ie: amavisd-new, will use to connect to ClamAV LocalSocket /usr/share/clamav/clamd.sock # Obviously, the user who will own the ClamAV process. User clamav

Courier-IMAP Courier-IMAP is the IMAP client which allows Squirelmail, and other mail clients such as Mozilla Thunderbird to connect and download messages. There are actually two components installed: • courier-authlib, which manages user authentication; and • courier-imap, which is the actual IMAP server. However, this dependency is automatically resolved by FreeBSD’s ports, all you need to do is select the AUTH_MYSQL option.

Installing and configuring /usr/ports/mail/courier-imap make install && make clean

Make sure to enable the MySQL option, so the authmysql module is built with courier-authlib.

Courier-IMAP

Issue 15

Courier configuration dialog After the installation is complete, there will be two directories under /usr/local/etc: authlib, and courier-imap.

courier-authlib authlib/authdaemonrc: authmodulelist="authmysql"

authlib/authmysqlrc: # User Information MYSQL_CRYPT_PWFIELD MYSQL_GID_FIELD MYSQL_UID_FIELD MYSQL_HOME_FIELD MYSQL_LOGIN_FIELD MYSQL_MAILDIR_FIELD MYSQL_NAME_FIELD

password '125' '125' '/usr/local/virtual' username maildir name

# MySQL Server information MYSQL_SERVER localhost MYSQL_USERNAME mail_admin MYSQL_PASSWORD mail_admin_password MYSQL_DATABASE mail MYSQL_OPT 0 MYSQL_USER_TABLE mailbox

courier-imap This puts in place the mechanism to authenticate IMAP requests against the information you have stored in your MySQL database. courier-imap/imapd: # Change this from to 127.0.0.1, so that only # SquirrelMail has un-encrypted access to IMAP ADDRESS=127.0.0.1 PORT=143

courier-imap/imapd-ssl: # The port, and IP address to listen for ssl/tls encrypted connections. SSLADDRESS=0

Installing and configuring

Issue 15 SSLPORT=993

Webmail with SquirrelMail SquirrelMail is a flexible, easy-to-configure webmail package written in PHP. Configuring Apache is beyond the scope of this article, and there are many other good tutorials out there, so I’m going to focus on installing the necessary packages, and then on configuring SquirrelMail.

Configuring SquirrelMail Install Apache: /usr/ports/www/apache22 make install && make clean

Install PHP: /usr/ports/lang/php5 make install && make clean

Install SquirrelMail: /usr/ports/mail/squirrelmail make install && make clean

Installing SquirrelMail plugins: /usr/ports/mail/squirrelmail-compatibility-plugin make install && make clean /usr/ports/mail/squirrelmail-vlogin-plugin make install && make clean

Configuring SquirrelMail: cd /usr/local/www/squirrelmail ./configure

1. Select 2 to configure the server, make sure to configure the domain name, and the imap server settings. 2. Select R to return to the main menu. 3. Select 8 to enable plugins, make sure you’ve enabledthe vlogin plugin. 4. Save and quit!

Conclusion I’ve shown all the pieces required to put together a secure, full-featured, FreeBSD mail server. It was a lot to cover, and I thank you for staying with me. The first article took you through the steps of installing FreeBSD, managing updates, and installing the core server components for a fully functional and secure email server; this article covered configuring the core components for a robust, spam and virus filtering mail system. The journey is by no means over. I strongly encourage you to take the time to explore the packages used, and learn them thouroughly: they are powerful tools which will serve you well in the future.

Bibliography http://www.postfix.org/ http://www.clamav.net/

Bibliography

Issue 15 http://www.ijs.si/software/amavisd/ http://spamassassin.apache.org/index.html http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/

Biography Yousef Ourabi (/user/43" title="View user profile.): Yousef Ourabi (http://yousefourabi.com) is a developer in the San Francisco bay area. He is currently working at the startup he recently founded, Zero-Analog (http://www.zero-analog.com " title="Zero-Analog). Zero-Analog is currently developing an enterprise application, however, one of its stated goals is "to increase the rate of open source adoption in companies of all sizes, across all industries". Zero-Analog also offers consulting services, all based around open source tools, frameworks and applications.

Copyright information Verbatim copying and distribution of this entire article is permitted in any medium without royalty provided this notice is preserved. Source URL: http://www.freesoftwaremagazine.com/articles/secure_email_servers_from_scratch_with_freebsd_6_part_2

Bibliography