Linda Octa

Soo what have I been doing lately?

The Fascinator Desktop
The Fascinator 0.3 Desktop was released 3 weeks ago.

Features implemented in The Fascinator 0.3 Desktop can be found in: http://fascinator.usq.edu.au/trac/wiki/tf2/Version0.3.0

P.S: for new features and enhancements request for The Fascinator 0.4 can be found in: http://fascinator.usq.edu.au/trac/wiki/tf2/Planning

So what are the features have been implemented in The Fascinator Desktop so far? I have compiled and created a simple features page for easy browsing: http://fascinator.usq.edu.au/trac/wiki/tf2/DeveloperNotes/Features

Watcher
Together with The Fascinator 0.3 Desktop, we released Watcher Watcher 0.3beta and 0.3pyinotify, http://fascinator.usq.edu.au/trac/wiki/Watcher

In Watcher 0.3beta, we start to support multi platform File system watcher. Ron Ward developed the watcher code in .NET, thus for linux/Mac OS X users, the same code base can be used through mono

For Watcher 0.3pyinotfy, this will be the last release for Linux that use pyinotify. Why we are not using pyinotify anymore? Basically it’s only work in Linux.

Issue found in both version of Watcher-s: file rename/move from a directory to another directory in Linux/Mac OS X is not working properly. This is a known bugs found in mono, https://bugzilla.novell.com/show_bug.cgi?id=418241 as well as pyinotify http://trac.dbzteam.org/pyinotify/ticket/30.

ICE week
Last week is ICE week. Peter Sefton wrote ICE week background on his blog: http://ptsefton.com/2009/11/25/ice-week-background.htm.

My task was generating eBook plugin for ICE package in ePub format based on the manifest items found in the ICE organiser.
Result? The generated eBook can be viewed in Peter’s Android phone as well as my iPhone. During this development, I encounter a usefull tool to check the validity of the epub structure that I created. This tool can be downloaded from http://code.google.com/p/epubcheck/

I also successfully implement epub generation in The Fascinator Desktop. This feature will be included in The Fascinator 0.4 release. Some note about epub feature in The Fascinator Desktop: http://fascinator.usq.edu.au/trac/wiki/tf2/DeveloperNotes/Features/eBook

I realise that I have been using a lot of links in this blog entry ==”. But maybe it’s for the best instead of duplicating the documentation that I have written in the wiki page.

Stay tune!!

In this release, we managed to complete a few features as well as some enhancement stated in: http://fascinator.usq.edu.au/trac/wiki/tf2/Planning

The release note can be found in:
http://fascinator.usq.edu.au/trac/wiki/tf2/Version0.2.0

Next to do list in my agenda: Watcher 0.2.0 almost there! Possible outcome in this release:

• Implement new API code to current Watcher 0.1.0 (Ubuntu)
• Able to use .net Wacher code (by Ron Ward) on Mac OSX/Ubuntu by using mono

Note for myself for next release:

• In ICE service, add functionality to Image resize to use fixed size instead of ratio
• For backup, add feedback on portal to let user see the progress of the backup
• Check how to utilise maven release plugin: e.g. site, release notes, and java docs

Note for eResearch:

• Find out how to use RDFa
• Find out how ontologize.me works

Motivate myself in WRITING!!!

When you want to insert a Landscape style in one of the page of your document, it’s really painful if you don’t know how to do it (although it’s not as difficult as Ms Office :P). Open office have a nice features that you can set your own style for you page, paragraph, frame, list and even a character.

Let’s get started on how to create a Landscape style for your page. By default all pages in your document will have default style.
1. Open up your style list

Styles and Formatting

2. Add a new Landscape style

Creating Landscape style

3. Click on the Page tab to set the page orientation to Landscape

Setting Landscape orientation

4. Press “OK” and you will have a new Style after you complete the step.
5. Set the page with your style
Place your cursor to the beginning of your page, and double click on the new style you just created. Notice that only that page is being set to landscape and pages before and after the page still in default format.

Setting Landscapce style

Installed Snow Leopard! Mmm my mac performance is better and faster Woo hoo so far so good!
Scared if my aperture would have issue after installation, but turn out everything is good!

Everytime I need to bundle image in Amazon server, I need to google on how to do the bundling. I could not remember the steps. :X so I think it’s better for me to write it down here for my own reference or maybe other can find it useful to do it.

I wrote this bundling guide with the assumption you have your AWS account created from aws.amazon.com or you can read amazon doco on getting started with AWS: http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/Welcome.html. You also need your certificate, private key as well as your keypair ready to do this bundling.

The instruction below is for bundling Ubuntu/Linux AMI:

• Preparation: ssh into your current AMI instances that you want to bundle using your keypair:

  ssh -i yourKeypair root@youriInstanceAddress

• Copy your private key and your certificate to the /mnt of instance that you are going to bundle (/mnt drive will not be bundled):

  scp -i yourKeypair pk-yourpkkey.pem cert-yourcert.pem root@yourInstanceAddress:/mnt

• Install ec2 tools your Instance does not have ec2-tool:

  sudo apt-get install ec2-api-tools

  Note: Instruction can be found at: http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/setting-up-your-tools.html

• Bundle your image (this will take awhile):

  ec2-bundle-vol -d /mnt -k /mnt/pk-yourpkkey.pem -c /mnt/cert-yourcert.pem -u AIDADH4IGTRXXKCD -r i386 -p yourImageName

• Check your bundle:

  ls -l /mnt/youImageName.*

• Uploading your image

  ec2-upload-bundle -b your-s3-bucket -m /mnt/yourImageName.manifest.xml -a your-aws-access-key-id -s your-aws-secret-access-key

• Register your AMI

  ec2-register your-s3-bucket/yourImageName.manifest.xml

• And you are done!

Since The Fascinator Desktop 0.1 Release, #tf4desktop, we need to make sure ICE service and OpenOffice are running in the server to support the conversion sent thru The Fascinator Desktop.

I managed to setup Nagios in ICE server to watch ICE and OpenOffice process and when ICE or OpenOffice or even server are down, notification email will be sent to the developer team. I wrote the installation and configuration instruction below on setting up nagios in ubuntu:

Step 1: Installing Apache and other libraries

• Install required library before installing nagios3

  sudo -s apt-get install libgd2-xpm-dev build-essential apache2

• Configure Apache if you want apache to listen to other port (by default, apache is listening to port 80)

  gedit /etc/apache2/ports.conf

• Change the Listen port number, apache is listening to port 85 in the below example:

  NameVirtualHost *:80
  Listen 85
  <IfModule mod_ssl.c>
            # SSL name based virtual hosts are not yet supported, therefore no
            # NameVirtualHost statement here
            Listen 443
  </IfModule>

• Restart apache:

  /etc/init.d/apache2 restart

• Check if Apache is running: http://localhost or http://localhost:85 (if you change the port number)

Step 2: Nagios 3

• Creating user for nagios

  /usr/sbin/useradd nagios
  passwd nagios

• Creating group for nagios user

  /usr/sbin/usermod -G nagios nagios
  /usr/sbin/groupadd nagcmd
  /usr/sbin/usermod -G nagcmd nagios
  /usr/sbin/usermod -G nagcmd www-data

• Download nagios

  cd /tmp
  wget http://freefr.dl.sourceforge.net/sourceforge/nagios/nagios-3.0.6.tar.gz

• Untar and compile

  tar zxvf nagios-3.0.6.tar.gz
  cd nagios-3.0.6
  ./configure --with-command-group=nagcmd
  make all
  make install
  make install-init
  make install-config
  make install-commandmode
  make install-webconf

• Get the nagios plugin

  cd /tmp
  wget http://freefr.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.13.tar.gz

• Untar and compile

  tar zxvf nagios-plugins-1.4.13.tar.gz
  cd nagios-plugins-1.4.13
  ./configure --with-nagios-user=nagios --with-nagios-group=nagios
  make
  make install

• Make sure nagios is running every time system started.

  ln -s /etc/init.d/nagios /etc/rcS.d/S99nagios

• Configure contact for nagios

  gedit /usr/local/nagios/etc/objects/contacts.cfg

  Put the contact information:

  define contact{
          contact_name                    nagiosadmin		; Short name of user
  	use				generic-contact		; Inherit default values from generic-contact template (defined above)
          alias                           Nagios Admin		; Full name of user
   
          email                           me@gmail.com	        ; Email where the notification will be forwarded to
  	host_notifications_enabled	1                       ; Enable host notification
  	service_notifications_enabled	1                       ; Enable service notification
  	service_notification_period	24x7                    ; Period of notification
  	host_notification_period 	24x7                    ; Period of notification
  	service_notification_options	w,u,c,r                 ; Notify if service: w: warning; u: unreachable; c: critical; r: recovered
  	host_notification_options	d,u,r                   ; Notify if host: d: down; u: unreachable; r: recovered
  	service_notification_commands 	notify-service-by-email ; Service notification command
  	host_notification_commands	notify-host-by-email    ; Host notification command
  	can_submit_commands		1                       ; Enable allow command submission
          }

• If you would like to add new command or edit existing command such as “notify-service-by-email” or “notify-host-by-email”, you can do it through:

  gedit /usr/local/nagios/etc/objects/command.cfg

  Below are example of adding new command to check if OpenOffice is still running in port 2002:

  # 'check_tcp_local' command definition
  define command{
          command_name    check_tcp_local
          command_line    $USER1$/check_tcp -p $ARG1$ $ARG2$
          }

• Configure nagios host

  gedit /usr/local/nagios/etc/objects/localhost.cfg

  Configuration:

  define host{
          use                     linux-server            ; Name of host template to use
                                                          ; This host definition will inherit all variables that are defined
                                                          ; in (or inherited by) the linux-server host template definition.
          host_name               localhost
          alias                   localhost
          address                 XX.XXX.XXX.XXX          ; IP address of the host where the nagios is going to watched
          }

• Adding Services to be watched. (below are the example how nagios watching ICE and Openoffice)

  gedit /usr/local/nagios/etc/objects/localhost.cfg

  Adding new services at the end of the localhost.cfg file:

  define service{
          use                             local-service         ; Name of service template to use
          host_name                       localhost             ; Must be the same as the host_name define on the top
          service_description             ICE running           ; Name of the new service
          check_command                   check_tcp!80          ; ICE is running in port 80, so we use check_tcp command to check if ICE is still running
          check_interval                  60                    ; Checking interval 60 minutes
          check_period                    24x7                  ; Allow 24x7 checking
          retry_interval                  3                     ; Trying for three times
          max_check_attempts              3                     ; Maximum trying attempt
          notification_interval           60                    ; Notify admin every 60 minutes if ICE service is still not running
          notification_period             24x7                  ; Enable notification for 24x7
          notification_options            w,c,u                 ; Notify if: w: warning; c: critical; u: unreachable
          contact_groups                  admins                ; Notify admins group
          }
   
  define service{
          use                             local-service         ; Name of service template to use
          host_name                       localhost             ; Must be the same as the host_name define on the top
          service_description             Open Office running   ; Name of the new service
          check_command                   check_tcp_local!2002  ; OpenOffice is running in port 2002, so we use check_tcp_local command to check if OpenOffice is still running
          check_interval                  60                    ; Checking interval 60 minutes
          check_period                    24x7                  ; Allow 24x7 checking
          retry_interval                  3                     ; Trying for three times
          max_check_attempts              3                     ; Maximum trying attempt
          notification_interval           60                    ; Notify admin every 60 minutes if ICE service is still not running
          notification_period             24x7                  ; Enable notification for 24x7
          notification_options            w,c,u                 ; Notify if: w: warning; c: critical; u: unreachable
          contact_groups                  admins                ; Notify admins group
          }

  NOTE: if the SSH and HTTP are not enabled in the host, comment out the service configuration from localhost.cfg

  #define service{
  #        use                             local-service         ; Name of service template to use
  #        host_name                       localhost
  #        service_description             SSH
  #        check_command                   check_ssh
  #        notifications_enabled           0
  #        }
   
  #define service{
  #        use                             local-service         ; Name of service template to use
  #        host_name                       localhost
  #        service_description             HTTP
  #        check_command                   check_http
  #        notifications_enabled           0
  #        }

• Before you start up Nagios, make sure there are no errors with the command. (This need to be done everytime the configuration files has been modified)

  /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

• Restart apache & nagios

  /etc/init.d/apache2 restart
  /etc/init.d/nagios start

• Check if nagios has started

  /etc/init.d/nagios status

• Nagios from web (Assume that apache listening port is 85). Log in using username nagiosadmin and password for nagiosadmin
  http://localhost:85/nagios
• To see nagios log:

  tail -f /usr/local/nagios/var/nagios.log &
  
  

Step 3: Postfix to send outgoing mail

• Install postfix

  apt-get install postfix

• Configure postfix

  dpkg-reconfigure postfix

  Configuration:

  • General type of mail configuration: Internet Site
  • System mail name: e.g. smtp.gmail.com
  • Root and postmaster mail recipeint: e.g. me@gmail.com
  • Other destinations to accept mail for (blank for none): use the default value
  • Force synchronous updates on mail queue?: Yes
  • Local networks: Use default: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
  • Mailbox isze limit (bytes): 0 (default)
  • Local address extension character: + (default)
  • Internet protocols to use: all (default)
• Do some testing using mailx:

  sudo apt-get install mailx
  echo test |mail -s "test mail sent to external" me@gmail.com
  echo test |mail -s "test mail sent to normal user" <normal user name>
  echo test |mail -s "test mail sent to root" root

• To forward email to me@gmail.com (change to your email address)

  echo me@gmail.com > ~/.forward

Working together with Oliver Lucido (Main developer), Ron Ward (developer), me (developer), Duncan Dickinson (Project Manager) and Bron Chandler (Project Manager), last week we announced the release of The Fascinator Desktop 0.1.0: http://fascinator.usq.edu.au/trac/wiki/tf2/Version0.1.0

The information about what is The Fascinator Desktop, installation instruction and coding documentation can be found in:
http://fascinator.usq.edu.au/trac/wiki/Fascinator2

Watching channel seven, Today tonight about affiliate marketing for online income. I was thinking to join one! now just finding the right time to do it ==”

Good luck for me!

Time to blog my multipart file upload version 2. In this version I am no longer using the MultiPartFormOutputStream.java (see my previous blog: http://lindaocta.com/?p=185)

In this version, I use java dependencies:

• httpclient: download HttpClient and it’s dependencies from: http://hc.apache.org/httpclient-3.x/tutorial.html
• io: download from: http://commons.apache.org/downloads/download_io.cgi

Transformer_two.java code:

import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;
 
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;
import org.apache.commons.io.IOUtils;
 
class transformer_two{
	public static void main(String [] args) {
	    String convertUrl = "http://ice-service.usq.edu.au/api/convert/";
		if(args.length == 1) {
			String fileName = args[0];
			System.out.println(fileName);
			File f = new File(fileName);
			if(f.exists()) {
				String extResult = getExtension(f);
				if (!extResult.startsWith("Error")) {
					convertUrl = convertUrl + extResult;
					getRendition(f, convertUrl);
				}
			} else {
				System.out.println("Error: file doesn't exist!");
			}
		} else {
			System.out.println("Usage: java transformer fileName");
		}
	}
 
	public static void getRendition (File sourceFile, String convertUrl) {
		try{
				HttpClient client = new HttpClient();
				PostMethod filePost = new PostMethod(convertUrl);
				Part[] parts = {
						new StringPart("zip", "1"),
						new StringPart("toc", "1"),
						new StringPart("pdfLink", "1"),
						new StringPart("pathext", ""),
						new StringPart("template", getTemplate()),
						new FilePart("file", sourceFile)
				};
 
				filePost.setRequestEntity(new MultipartRequestEntity(parts, filePost.getParams()));
				client.executeMethod(filePost);
 
			    //Get Response
				InputStream is = filePost.getResponseBodyAsStream();
				String[] filePart = sourceFile.getName().split("\\.");
			    System.out.println(sourceFile.getName());
			    String outputFilename = "/tmp/"+ filePart[0] + ".zip";
			    FileOutputStream fos = new FileOutputStream(outputFilename);
			    IOUtils.copy(is, fos);
 
			    is.close();
			    fos.close();
				System.out.println("Rendition zip file is located at: " + outputFilename);
		} catch (Exception e) {
			System.out.println("An error occurred: " + e.getClass());
			e.printStackTrace();
		}
	}
 
	private static String getExtension(File fileObject) {
		/** getExtension method
		 * @param File fileObject
		 * @return String
		 */
		String[] parts = fileObject.getName().split("\\.");
		String ext = "";
		if (parts.length == 2) {
			ext = parts[1];
			return ext;
		}
		return "Error: Unable to detect file extension properly!";
	}
 
	private static String getTemplate (){
		String template = "<html>" +
  "<head>" +
    "<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>" +
    "<title>Default Template</title>" +
    "<style type='text/css'>" +
      ".rendition-links { text-align: right; }" +
      ".body table td { vertical-align: top; }" +
    "</style>" +
    "<style class='sub style-css' type='text/css'></style>" +
  "</head>" +
  "<body>" +
    "<div class='rendition-links'>" +
      "<span class='ins source-link'></span>" +
      "<span class='ins slide-link'></span>" +
      "<span class='ins pdf-rendition-link'></span>" +
    "</div>" +
    "<h1 class='ins title'></h1>" +
    "<div class='ins page-toc'></div>" +
    "<div class='ins body'></div>" +
  "</body>" +
"</html>";
		return template;
	}
}

To run the new transformer_two.java:

1. download the code from: transformer_two.java and save as transformer_two.java
2. create “lib” folder in the same level
3. Put all the jar file from the downloaded dependencies above in “lib” folder
4. Compile transformer_two.java:

   javac -d . -classpath ./lib/commons-httpclient-3.1.jar:./lib/commons-io-1.4.jar transformer_two.java

5. To run transformer_two:

   java -classpath .:./lib/commons-httpclient-3.1.jar:./lib/commons-io-1.4.jar:./lib/commons-logging-1.1.1.jar:./lib/commons-codec-1.3.jar transformer_two ABSOLUTE_PATH_TO_YOUR_DOCUMENT

Uploading Multipart file through java is not that easy. I need to write this code for the Transformer (I don’t have a chance to write the wiki yet :P), the bridge between The Fascinator desktop and Ice-Service.

Since I am trying my best to remember java syntax again, I am confuse which library to use. In this blog, I will put up the first version of multipart upload code, so for those who would like to use Ice-Service to get the rendered version of your documents, you can use this code independently
From this website http://forums.sun.com/thread.jspa?threadID=451245&forumID=31, I found a class that I can use directly, you can download the code from MultiPartFormOutputStream.java

And below are my transformer.java code:

import java.io.*;
import java.net.*;
 
class transformer{
	public static void main(String [] args) {
		String convertUrl = "http://ice-service.usq.edu.au/api/convert/";
		if(args.length == 1) {
			String fileName = args[0];
			System.out.println(fileName);
			File f = new File(fileName);
			if(f.exists()) {
				getRendition(convertUrl, f);
			} else {
				System.out.println("Error: file doesn't exist!");
			}
		} else {
			System.out.println("Usage: java transformer fileName");
		}
	}
 
	public static void getRendition (String convertUrl, String fileName) {
		getRendition(convertUrl, new File(fileName));
	}
 
	public static void getRendition (String convertUrl, File sourceFile) {
		String[] parts = sourceFile.getName().split("\.");
		String ext = "";
		if(parts.length == 2) { // check we have exactly two parts
			ext = parts[1];
		} else {
			System.out.println("Error: Unable to detect file extension properly!");
			return;
		}
 
		System.out.println("convertUrl: " + convertUrl);
		convertUrl = convertUrl + ext;
		System.out.println("fileName: " + sourceFile.getAbsolutePath());
 
		try{
				//Create connection
				URL url = new URL(convertUrl);
 
				//create a boundary string
				String boundary = MultiPartFormOutputStream.createBoundary();
				URLConnection urlConn = MultiPartFormOutputStream.createConnection(url);
				urlConn.setRequestProperty("Accept", "*/*");
				urlConn.setRequestProperty("Content-Type",
					MultiPartFormOutputStream.getContentType(boundary));
				//set some other request headers...
				urlConn.setRequestProperty("Connection", "Keep-Alive");
				urlConn.setRequestProperty("Cache-Control", "no-cache");
				//no need to connect cuz getOutputStream() does it
				MultiPartFormOutputStream out =
					new MultiPartFormOutputStream(urlConn.getOutputStream(), boundary);
 
				//Setting up all the options
				out.writeField("zip", "1");
				out.writeField("toc", "1");
				out.writeField("pdfLink", "1");
				out.writeField("pathext", "");
				out.writeField("template", getTemplate());
				out.writeFile("file", "application/zip", sourceFile);
				out.close();
 
			    //Get Response
			    InputStream is = urlConn.getInputStream();
			    BufferedReader rd = new BufferedReader(new InputStreamReader(is));
 
			    //zip the file and store in the /tmp directory
			    String outputFilename = "/tmp/rendition.zip";
			    FileOutputStream fos = new FileOutputStream(outputFilename);
	            Writer fileout = new OutputStreamWriter(fos);
	            int ch;
	            while ((ch = rd.read()) > -1) {
	                fileout.write((char)ch);
	            }
	            fileout.close();
 
 
		} catch (Exception e) {
			System.out.println("An error occurred: " + e.getClass());
			//e.printStackTrace();
		}
	}
 
	private static String getTemplate (){
		String template = "<html>" +
  "<head>" +
    "<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>" +
    "<title>Default Template</title>" +
    "<style type='text/css'>" +
      ".rendition-links { text-align: right; }" +
      ".body table td { vertical-align: top; }" +
    "</style>" +
    "<style class='sub style-css' type='text/css'></style>" +
  "</head>" +
  "<body>" +
    "<div class='rendition-links'>" +
      "<span class='ins source-link'></span>" +
      "<span class='ins slide-link'></span>" +
      "<span class='ins pdf-rendition-link'></span>" +
    "</div>" +
    "<h1 class='ins title'></h1>" +
    "<div class='ins page-toc'></div>" +
    "<div class='ins body'></div>" +
  "</body>" +
"</html>";
		return template;
	}
}

To use the code, simply put download both transformer.java and MultiPartFormOutputStream.java in the same directory and compile transformer.java:

lindaoctalina$ javac transformer.java
lindaoctalina$ java transformer ABSOLUTE_PATH_TO_YOUR_DOCUMENT

Your file will be saved to /tmp directory called rendition.zip

In my next few blogs, I will put up my enhanced version of this transformer.java. I am not using the MultiPartFormOutputStream.java anymore. I will also blog the transformer in php version (since I have the code somewhere when I did Open Journal System project)