Setting up Nagios 3 in Ubuntu 9.04 Jaunty + forward notification using postfix

September 3, 2009 ICE, Open Office, The Fascinator Desktop, Ubuntu/Linux, eResearch 7 Comments

Since The Fascinator Desktop 0.1 Release, #tf4desktop, we need to make sure ICE service and OpenOffice are running in the server to support the conversion sent thru The Fascinator Desktop.

I managed to setup Nagios in ICE server to watch ICE and OpenOffice process and when ICE or OpenOffice or even server are down, notification email will be sent to the developer team. I wrote the installation and configuration instruction below on setting up nagios in ubuntu:

Step 1: Installing Apache and other libraries

  • Install required library before installing nagios3
    [html]
    sudo -s apt-get install libgd2-xpm-dev build-essential apache2
  • Configure Apache if you want apache to listen to other port (by default, apache is listening to port 80)
    [html]
    gedit /etc/apache2/ports.conf
  • Change the Listen port number, apache is listening to port 85 in the below example:
    [html]
    NameVirtualHost *:80
    Listen 85
    <IfModule mod_ssl.c>
              # SSL name based virtual hosts are not yet supported, therefore no
              # NameVirtualHost statement here
              Listen 443
    </IfModule>
  • Restart apache:
    [html]
    /etc/init.d/apache2 restart
  • Check if Apache is running: http://localhost or http://localhost:85 (if you change the port number)

Step 2: Nagios 3

  • Creating user for nagios

    [html]
    /usr/sbin/useradd nagios
    passwd nagios
  • Creating group for nagios user
    [html]
    /usr/sbin/usermod -G nagios nagios
    /usr/sbin/groupadd nagcmd
    /usr/sbin/usermod -G nagcmd nagios
    /usr/sbin/usermod -G nagcmd www-data
  • Download nagios
    [html]
    cd /tmp
    wget http://freefr.dl.sourceforge.net/sourceforge/nagios/nagios-3.0.6.tar.gz
  • Untar and compile
    [html]
    tar zxvf nagios-3.0.6.tar.gz
    cd nagios-3.0.6
    ./configure --with-command-group=nagcmd
    make all
    make install
    make install-init
    make install-config
    make install-commandmode
    make install-webconf
  • Get the nagios plugin
    [html]
    cd /tmp
    wget http://freefr.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.13.tar.gz
  • Untar and compile
    [html]
    tar zxvf nagios-plugins-1.4.13.tar.gz
    cd nagios-plugins-1.4.13
    ./configure --with-nagios-user=nagios --with-nagios-group=nagios
    make
    make install
  • Make sure nagios is running every time system started.
    [html]
    ln -s /etc/init.d/nagios /etc/rcS.d/S99nagios
  • Configure contact for nagios
    [html]
    gedit /usr/local/nagios/etc/objects/contacts.cfg

    Put the contact information:

    [html]
    define contact{
            contact_name                    nagiosadmin     ; Short name of user
        use             generic-contact     ; Inherit default values from generic-contact template (defined above)
            alias                           Nagios Admin        ; Full name of user

            email                           me@gmail.com            ; Email where the notification will be forwarded to
        host_notifications_enabled  1                       ; Enable host notification
        service_notifications_enabled   1                       ; Enable service notification
        service_notification_period 24x7                    ; Period of notification
        host_notification_period    24x7                    ; Period of notification
        service_notification_options    w,u,c,r                 ; Notify if service: w: warning; u: unreachable; c: critical; r: recovered
        host_notification_options   d,u,r                   ; Notify if host: d: down; u: unreachable; r: recovered
        service_notification_commands   notify-service-by-email ; Service notification command
        host_notification_commands  notify-host-by-email    ; Host notification command
        can_submit_commands     1                       ; Enable allow command submission
            }
  • If you would like to add new command or edit existing command such as “notify-service-by-email” or “notify-host-by-email”, you can do it through:
    [html]
    gedit /usr/local/nagios/etc/objects/command.cfg

    Below are example of adding new command to check if OpenOffice is still running in port 2002:

    [html]
    # 'check_tcp_local' command definition
    define command{
            command_name    check_tcp_local
            command_line    $USER1$/check_tcp -p $ARG1$ $ARG2$
            }
  • Configure nagios host
    [html]
    gedit /usr/local/nagios/etc/objects/localhost.cfg

    Configuration:

    [html]
    define host{
            use                     linux-server            ; Name of host template to use
                                                            ; This host definition will inherit all variables that are defined
                                                            ; in (or inherited by) the linux-server host template definition.
            host_name               localhost
            alias                   localhost
            address                 XX.XXX.XXX.XXX          ; IP address of the host where the nagios is going to watched
            }
  • Adding Services to be watched. (below are the example how nagios watching ICE and Openoffice)
    [html]
    gedit /usr/local/nagios/etc/objects/localhost.cfg

    Adding new services at the end of the localhost.cfg file:

    [html]
    define service{
            use                             local-service         ; Name of service template to use
            host_name                       localhost             ; Must be the same as the host_name define on the top
            service_description             ICE running           ; Name of the new service
            check_command                   check_tcp!80          ; ICE is running in port 80, so we use check_tcp command to check if ICE is still running
            check_interval                  60                    ; Checking interval 60 minutes
            check_period                    24x7                  ; Allow 24x7 checking
            retry_interval                  3                     ; Trying for three times
            max_check_attempts              3                     ; Maximum trying attempt
            notification_interval           60                    ; Notify admin every 60 minutes if ICE service is still not running
            notification_period             24x7                  ; Enable notification for 24x7
            notification_options            w,c,u                 ; Notify if: w: warning; c: critical; u: unreachable
            contact_groups                  admins                ; Notify admins group
            }

    define service{
            use                             local-service         ; Name of service template to use
            host_name                       localhost             ; Must be the same as the host_name define on the top
            service_description             Open Office running   ; Name of the new service
            check_command                   check_tcp_local!2002  ; OpenOffice is running in port 2002, so we use check_tcp_local command to check if OpenOffice is still running
            check_interval                  60                    ; Checking interval 60 minutes
            check_period                    24x7                  ; Allow 24x7 checking
            retry_interval                  3                     ; Trying for three times
            max_check_attempts              3                     ; Maximum trying attempt
            notification_interval           60                    ; Notify admin every 60 minutes if ICE service is still not running
            notification_period             24x7                  ; Enable notification for 24x7
            notification_options            w,c,u                 ; Notify if: w: warning; c: critical; u: unreachable
            contact_groups                  admins                ; Notify admins group
            }

    NOTE: if the SSH and HTTP are not enabled in the host, comment out the service configuration from localhost.cfg

    [html]
    #define service{
    #        use                             local-service         ; Name of service template to use
    #        host_name                       localhost
    #        service_description             SSH
    #        check_command                   check_ssh
    #        notifications_enabled           0
    #        }

    #define service{
    #        use                             local-service         ; Name of service template to use
    #        host_name                       localhost
    #        service_description             HTTP
    #        check_command                   check_http
    #        notifications_enabled           0
    #        }
  • Before you start up Nagios, make sure there are no errors with the command. (This need to be done everytime the configuration files has been modified)
    [html]
    /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
  • Restart apache & nagios
    [html]
    /etc/init.d/apache2 restart
    /etc/init.d/nagios start
  • Check if nagios has started
    [html]
    /etc/init.d/nagios status
  • Nagios from web (Assume that apache listening port is 85). Log in using username nagiosadmin and password for nagiosadmin
    http://localhost:85/nagios
  • To see nagios log:
    [html]
    tail -f /usr/local/nagios/var/nagios.log &

Step 3: Postfix to send outgoing mail

  • Install postfix

    [html]
          apt-get install postfix
  • Configure postfix
    [html]
          dpkg-reconfigure postfix

    Configuration:

    • General type of mail configuration: Internet Site
    • System mail name: e.g. smtp.gmail.com
    • Root and postmaster mail recipeint: e.g. me@gmail.com
    • Other destinations to accept mail for (blank for none): use the default value
    • Force synchronous updates on mail queue?: Yes
    • Local networks: Use default: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
    • Mailbox isze limit (bytes): 0 (default)
    • Local address extension character: + (default)
    • Internet protocols to use: all (default)
  • Do some testing using mailx:
    [html]
    sudo apt-get install mailx
    echo test |mail -s "test mail sent to external" me@gmail.com
    echo test |mail -s "test mail sent to normal user" <normal user name>
    echo test |mail -s "test mail sent to root" root
  • To forward email to me@gmail.com (change to your email address)
    [html]
          echo me@gmail.com > ~/.forward

The Fascinator Desktop 0.1.0 Release

August 31, 2009 The Fascinator Desktop, Ubuntu/Linux, Window, eResearch No Comments

Working together with Oliver Lucido (Main developer), Ron Ward (developer), me (developer), Duncan Dickinson (Project Manager) and Bron Chandler (Project Manager), last week we announced the release of The Fascinator Desktop 0.1.0: http://fascinator.usq.edu.au/trac/wiki/tf2/Version0.1.0

The information about what is The Fascinator Desktop, installation instruction and coding documentation can be found in:
http://fascinator.usq.edu.au/trac/wiki/Fascinator2

Multipart file upload in java (Version 2)

July 7, 2009 ICE, The Fascinator Desktop, eResearch, java 1 Comment

Time to blog my multipart file upload version 2. In this version I am no longer using the MultiPartFormOutputStream.java (see my previous blog: http://lindaocta.com/?p=185)

In this version, I use java dependencies:

Transformer_two.java code:

[html]
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;
import org.apache.commons.io.IOUtils;
 
class transformer_two{
    public static void main(String [] args) {
        String convertUrl = "http://ice-service.usq.edu.au/api/convert/";
        if(args.length == 1) {
            String fileName = args[0];
            System.out.println(fileName);
            File f = new File(fileName);
            if(f.exists()) {
                String extResult = getExtension(f);
                if (!extResult.startsWith("Error")) {
                    convertUrl = convertUrl + extResult;
                    getRendition(f, convertUrl);
                }
            } else {
                System.out.println("Error: file doesn't exist!");
            }
        } else {
            System.out.println("Usage: java transformer fileName");
        }
    }
 
    public static void getRendition (File sourceFile, String convertUrl) {
        try{
                HttpClient client = new HttpClient();  
                PostMethod filePost = new PostMethod(convertUrl);
                Part[] parts = {
                        new StringPart("zip", "1"),
                        new StringPart("toc", "1"),
                        new StringPart("pdfLink", "1"),
                        new StringPart("pathext", ""),
                        new StringPart("template", getTemplate()),
                        new FilePart("file", sourceFile)
                };
               
                filePost.setRequestEntity(new MultipartRequestEntity(parts, filePost.getParams()));
                client.executeMethod(filePost);
               
                //Get Response
                InputStream is = filePost.getResponseBodyAsStream();
                String[] filePart = sourceFile.getName().split("\\.");
                System.out.println(sourceFile.getName());
                String outputFilename = "/tmp/"+ filePart[0] + ".zip";
                FileOutputStream fos = new FileOutputStream(outputFilename);
                IOUtils.copy(is, fos);
               
                is.close();
                fos.close();
                System.out.println("Rendition zip file is located at: " + outputFilename);
        } catch (Exception e) {
            System.out.println("An error occurred: " + e.getClass());
            e.printStackTrace();
        }
    }
 
    private static String getExtension(File fileObject) {
        /** getExtension method
         * @param File fileObject
         * @return String
         */
        String[] parts = fileObject.getName().split("\\.");
        String ext = "";
        if (parts.length == 2) {
            ext = parts[1];
            return ext;
        }
        return "Error: Unable to detect file extension properly!";
    }

    private static String getTemplate (){
        String template = "<html>" +
  "<head>" +
    "<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>" +
    "<title>Default Template</title>" +
    "<style type='text/css'>" +
      ".rendition-links { text-align: right; }" +
      ".body table td { vertical-align: top; }" +
    "</style>" +
    "<style class='sub style-css' type='text/css'></style>" +
  "</head>" +
  "<body>" +
    "<div class='rendition-links'>" +
      "<span class='ins source-link'></span>" +
      "<span class='ins slide-link'></span>" +
      "<span class='ins pdf-rendition-link'></span>" +
    "</div>" +
    "<h1 class='ins title'></h1>" +
    "<div class='ins page-toc'></div>" +
    "<div class='ins body'></div>" +
  "</body>" +
"</html>";
        return template;
    }
}

To run the new transformer_two.java:

  1. download the code from: transformer_two.java and save as transformer_two.java
  2. create “lib” folder in the same level
  3. Put all the jar file from the downloaded dependencies above in “lib” folder
  4. Compile transformer_two.java:
    [html]
    javac -d . -classpath ./lib/commons-httpclient-3.1.jar:./lib/commons-io-1.4.jar transformer_two.java
  5. To run transformer_two:
    [html]
    java -classpath .:./lib/commons-httpclient-3.1.jar:./lib/commons-io-1.4.jar:./lib/commons-logging-1.1.1.jar:./lib/commons-codec-1.3.jar transformer_two ABSOLUTE_PATH_TO_YOUR_DOCUMENT

Multipart file upload in java (Version 1)

July 2, 2009 ICE, The Fascinator Desktop, eResearch, java 1 Comment

Uploading Multipart file through java is not that easy. I need to write this code for the Transformer (I don’t have a chance to write the wiki yet :P ), the bridge between The Fascinator desktop and Ice-Service.

Since I am trying my best to remember java syntax again, I am confuse which library to use. In this blog, I will put up the first version of multipart upload code, so for those who would like to use Ice-Service to get the rendered version of your documents, you can use this code independently :)
From this website http://forums.sun.com/thread.jspa?threadID=451245&forumID=31, I found a class that I can use directly, you can download the code from MultiPartFormOutputStream.java

And below are my transformer.java code:

[html]
import java.io.*;
import java.net.*;

class transformer{
    public static void main(String [] args) {
        String convertUrl = "http://ice-service.usq.edu.au/api/convert/";
        if(args.length == 1) {
            String fileName = args[0];
            System.out.println(fileName);
            File f = new File(fileName);
            if(f.exists()) {
                getRendition(convertUrl, f);
            } else {
                System.out.println("Error: file doesn't exist!");
            }
        } else {
            System.out.println("Usage: java transformer fileName");
        }
    }
   
    public static void getRendition (String convertUrl, String fileName) {
        getRendition(convertUrl, new File(fileName));
    }
   
    public static void getRendition (String convertUrl, File sourceFile) {
        String[] parts = sourceFile.getName().split("\.");
        String ext = "";
        if(parts.length == 2) { // check we have exactly two parts
            ext = parts[1];
        } else {
            System.out.println("Error: Unable to detect file extension properly!");
            return;
        }
       
        System.out.println("convertUrl: " + convertUrl);
        convertUrl = convertUrl + ext;
        System.out.println("fileName: " + sourceFile.getAbsolutePath());
       
        try{
                //Create connection
                URL url = new URL(convertUrl);
               
                //create a boundary string
                String boundary = MultiPartFormOutputStream.createBoundary();
                URLConnection urlConn = MultiPartFormOutputStream.createConnection(url);
                urlConn.setRequestProperty("Accept", "*/*");
                urlConn.setRequestProperty("Content-Type",
                    MultiPartFormOutputStream.getContentType(boundary));
                //set some other request headers...
                urlConn.setRequestProperty("Connection", "Keep-Alive");
                urlConn.setRequestProperty("Cache-Control", "no-cache");
                //no need to connect cuz getOutputStream() does it
                MultiPartFormOutputStream out =
                    new MultiPartFormOutputStream(urlConn.getOutputStream(), boundary);

                //Setting up all the options
                out.writeField("zip", "1");
                out.writeField("toc", "1");
                out.writeField("pdfLink", "1");
                out.writeField("pathext", "");
                out.writeField("template", getTemplate());
                out.writeFile("file", "application/zip", sourceFile);
                out.close();
               
                //Get Response 
                InputStream is = urlConn.getInputStream();
                BufferedReader rd = new BufferedReader(new InputStreamReader(is));
               
                //zip the file and store in the /tmp directory
                String outputFilename = "/tmp/rendition.zip";
                FileOutputStream fos = new FileOutputStream(outputFilename);
                Writer fileout = new OutputStreamWriter(fos);
                int ch;
                while ((ch = rd.read()) > -1) {
                    fileout.write((char)ch);
                }
                fileout.close();

               
        } catch (Exception e) {
            System.out.println("An error occurred: " + e.getClass());
            //e.printStackTrace();
        }
    }
   
    private static String getTemplate (){
        String template = "<html>" +
  "<head>" +
    "<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>" +
    "<title>Default Template</title>" +
    "<style type='text/css'>" +
      ".rendition-links { text-align: right; }" +
      ".body table td { vertical-align: top; }" +
    "</style>" +
    "<style class='sub style-css' type='text/css'></style>" +
  "</head>" +
  "<body>" +
    "<div class='rendition-links'>" +
      "<span class='ins source-link'></span>" +
      "<span class='ins slide-link'></span>" +
      "<span class='ins pdf-rendition-link'></span>" +
    "</div>" +
    "<h1 class='ins title'></h1>" +
    "<div class='ins page-toc'></div>" +
    "<div class='ins body'></div>" +
  "</body>" +
"</html>";
        return template;
    }
}

To use the code, simply put download both transformer.java and MultiPartFormOutputStream.java in the same directory and compile transformer.java:

[html]
lindaoctalina$ javac transformer.java
lindaoctalina$ java transformer ABSOLUTE_PATH_TO_YOUR_DOCUMENT

Your file will be saved to /tmp directory called rendition.zip

In my next few blogs, I will put up my enhanced version of this transformer.java. I am not using the MultiPartFormOutputStream.java anymore. I will also blog the transformer in php version (since I have the code somewhere when I did Open Journal System project)

Linux-FS Watcher for The Fascinator Desktop (Updated)

June 14, 2009 Firefox, The Fascinator Desktop, Ubuntu/Linux, eResearch, xhtml, xml No Comments

Recently I am involved in The Fascinator Desktop project. Peter Sefton is updated the detail workflow of this project: Desktop Repositories: Smashing up Power point. Part of my involvement is to develop filesystem watcher (Linux and Mac OS watcher).

The Linux FS watcher can be checked-out from http://fascinator.usq.edu.au/svn-auth/code/watcher/tags/0.1. This is a standalone program, thus you can use it directly to monitory your filesystem. The whole application is developed in python.

Dependencies before you run the Linux FS watcher:

  • pyinotify: Note for Ubuntu users: Try to download the pyinotify directly from http://trac.dbzteam.org/pyinotify instead of using ubuntu pyinotify package. And if you have any issue with this library, you can ask for help directly from the website, the support teams are very helpful. I had the issue with renaming a directory once, and their responses are quick! Refer to: http://trac.dbzteam.org/pyinotify/ticket/20.
  • sqlite: This library should be pre-shipped with your ubuntu, if not, you can directly get the package from the package manager. sqlite is used to store all the events happened within the watched directory. The table for the event consists of filePath, event (file deletion or modification), and timestamp when the event is invoked.
  • foresite: (Won’t be needed anymore since it return JSON file instead of atom feed) Download from http://code.google.com/p/foresite-toolkit/. This library is used to generate atom feed requested through web service to provide the list of events stored in the sqlite.

To run the program, go to the watcher directory after you checked-out the code:

  • Modify the config.json configuration file, change the watchDirs to the directory you want to watch. Make sure it’s exist, and try not to watch your home directory. (I have not developed any code to ignore the python object file, thus, when you watch your home directory, your watcher program object file will be added to the database).
    [html]
        "watchDirs" : ["/home/octalina/workspace/watcher/someDir", "/home/octalina/workspace/watcher/modeDir"],

    If you want to run the watcher as a daemon, change the daemon value to True.

    [html]
           "daemon" : "True"
  • Change to app directory and run watcher:
    [html]
           python watcher.py
  • To run the webservice, open another terminal tab and run:
    [html]
           python jsonWebservice.py

To try get the the list of file changes can be done using curl/wget:

[html]
curl localhost:9000

or if date range is specified:

[html]
curl localhost:9000 -H "Last-Modified: Wed, 17 Jun 2009 18:38:12 GMT"