Replace multiple spaces/tabs with single space

Use following UNIX command to replace multiple spaces or tabs with a single space, thus enabling easier work with cut command:

tr -s ''

Installing Xen dom0 on Ubuntu 9.04

The overall structure of a 'Xen-enabled' system can be visualized by the following figure:

As shown in the figure, and contrary to my prior belief, Xen hypervisor is represented by another GRUB entry at system boot time (as opposed to simply replacing the OS that it was installed from). As a result, you can choose to boot into original OS without Xen availability or into the Xen system even after you have installed Xen on given system. This was just used to show a high-level picture of system layout...

Following are a set of commands that will install Xen hypervisor. These commands are to be executed within the original OS currently existing on the machine:
$ wget http://http.us.debian.org/debian/pool/main/l/linux-2.6/linux-image-2.6.26-2-xen-686_2.6.26-19_i386.deb
$ wget http://http.us.debian.org/debian/pool/main/l/linux-2.6/linux-modules-2.6.26-2-xen-686_2.6.26-19_i386.deb
$ sudo dpkg -i linux-modules-2.6.26-2-xen-686_2.6.26-19_i386.deb
$ sudo dpkg -i linux-image-2.6.26-2-xen-686_2.6.26-19_i386.deb
$ sudo apt-get install ubuntu-xen-server

Then, restart the machine and in the grub loader, Xen should appear as a bootable OS.

After booting up, the following command should tell you that you are in dom0:

$ sudo xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 702 1 r----- 312.9

When it comes to installing guest OS's (i.e., domU's), the following websites were helpful:

• A brief and technical process can be found here
• The same process but where each step is explained in more detail can be found here
• Another example process that shows contents of many files that need to be edited can be found he

Adding new disk to ZFS

This is a short and adjusted version of article available here.
When initially instantiating and setting up accounts on ZFS filesystem (in my case, under OpenSolaris 2009.06), it is beneficial to store user data on a pool other than root pool (i.e., rpool). This is because rpool cannot be composed from multiple hard disks and thus cannot be changed in size (unless it is mirrored) [ref].

So, start by adding a disk to the machine and creating a pool from it.
The following commands lists names of disks available in the system:
afgane@opensolaris:~$ pfexec format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c7d0
/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
1. c7d1
/pci@0,0/pci-ide@1,1/ide@0/cmdk@1,0
2. c8d1
/pci@0,0/pci-ide@1,1/ide@1/cmdk@1,0
Specify disk (enter its number): ^C
afgane@opensolaris:~$

From this, we see that the above system has three disks: c7d0, c7d1 and c8d1
NOTE: Instead of creating a filesystem on a physical disk, you may create it from a simple file that exists in current file system. To do this, create a file on the system of desired size and use it instead of the disk name. Use the following command:

afgane@opensolaris:~$ dd if=/dev/zero of=myfile bs=1024 count=10

This command creates a file called "myfile" filled with zeros with a block size of 1000 multiplied 10 tens or 10K (i.e., created file of size 10K bytes).

Before seeing which of the disks is being used by which pool, we need to see what pools are available on the machine. Do so by issuing the following command:
root@opensolaris:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
rpool 34.8G 3.79G 31.0G 10% ONLINE -
root@opensolaris:~#

From this output we can see that there is only one pool available on the machine, namely rpool.

Next, in order to see which disk is being used by rpool, issue the following command:
afgane@opensolaris:~$ zpool status
pool: rpool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c7d0s0 ONLINE 0 0 0

errors: No known data errors
afgane@opensolaris:~$

This tells us that disk named c7d0s0 is used by rpool.

We can now create a new pool and add one of the two extra disks that are available in the machine to this newly created pool. Do so by executing following command (note that you must be root to issue this command):
root@opensolaris:~$ zpool create myPool c7d1

We can now see our newly created pool and can access it as a regular directory on the machine:
root@opensolaris:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
myPool 1.95G 32.7M 1.92G 1% ONLINE -
rpool 34.8G 3.79G 31.0G 10% ONLINE -
root@opensolaris:~#
root@opensolaris:~# cd /myPool

If you ever add another disk to this machine, it is very simple to add it to the overall pool of available space. We will use the third disk that is available in this machine, as follows (remember that you need to be root to issue this command):
root@opensolaris:~# zpool add myPool c8d1

And here is the list of the new pool arrangement, showing the increased pool size:
root@opensolaris:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
myPool 2.95G 32.7M 2.91G 1% ONLINE -
rpool 34.8G 3.79G 31.0G 10% ONLINE -
root@opensolaris:~#

Add 2nd disk in VMware

1. Add a second disk to the guest OS through the VM manager
2. Boot the guest OS and go to Control Panel->Administrative Tools->Computer Management->Disk Management
3. Right click the new partition and initialize it.

Quickly toggle tab swap preference for TabMix Plus

Simply press F9 while FF in focus to toggle between tabs being swapped under Last Selected preference or Select Right preference.

Adding a tag in subversion through subclipse

In order to create a tag (or branch) select “Branch/Tag...” from the Team menu. In the “Copy Branch/Tag” dialog, in the “Copy to URL” field, make sure that the correct folder, branches, or tags were the snapshot wants to be put is displayed in the path by either browsing or typing it, for example: “https:///users/afgane/OptionView/tags/0.2”. “Browse” button can be used here to navigate to the branch or tag folder where the branch or tag would like to be created. If you would like to create a special folder which designates a version, then you can simply append a new folder to the end of the To URL (i.e. tags/0.2, in above example). This will then be created for you. Select the revision which you would like to copy (i.e. “Head revision in the repository”, “Specific revision in the repository”, or “Working copy”.
Create a message in the branch/tag comment area. If you would like to continue to work in your current working copy then leave the “Switch working copy to new branch/tag” checkbox clear. If you would like to switch so that you can make additional modifications to the branch/tag without changing the current working copy then select this checkbox.
Click OK in order to create the branch or tag.

Adding a computer to subversion repository (through subclipse)

All of this is done through Eclipse using Subclipse Project.

Assume there is a subversion repository already in place with documents in it. The repository was established with a certain version of documents from computer ONE. At the same time, computer TWO has a copy of those documents and it is not connected to the repository. Documents on computer TWO are edited before connecting the computer to the repository. Now, I would like to create a connection from computer TWO to the repository but I cannot simply issue a 'checkout' command because documents on computer TWO are the more recent version of the ones that are in the repository. While one would think this is a simple task (for a beginner w.r.t. subverion at least), it is not!
In order to achieve desired functionality, the following can be done (all of these are done on computer TWO):
1. From Windows Explored, move entire contents of the project directory to some temporary directory
2. From Eclipse (i.e., subclipse), checkout the desired project (go to SVN Repository Exploring perspective, browse to the desired project, right-click on trunk, tag (or similar) directory that wants to be checked out within the browser, and select Checkout)
3. Follow the checkout wizard and chose to 'Check out as a project in the workspace' and give it the name that matches the desired project name. After clicking Finish, it will tell you that the specified project exists and contents will be deleted. Click OK.
4. After checking out the documents has completed, go back to the Windows Explorer and copy contents of the temporary directory into the project directory
5. Refresh the project tree view in Eclipse. Right-click project name in Eclipse and select Team->'Synchronize with Repository'. This will open Team Synchronizing perspective (or ask you to do so) and show you differences from the new document versions with the ones in the repository.
6. Commit the new version into the repository by selecting documents, right-clicking and choosing Commit command.

From now on, both of the computers should be able to update/commit changes as needed.

If anyone reads this and knows a better way of doing it, please leave a comment explaining how because, in my current opinion, whoever designed document repositories to have the interface different from that of a simple drag&drop is an idiot.

Installing kinship library for R

Download the library from: http://cran.r-project.org/src/contrib/kinship_1.1.0-21.tar.gz
Save it anywhere.
From the directory where the library was saved, issue command: R CMD INSTALL kinship_1.1.0-21.tar.gz
Upon completion, you can type R to start the application and then issue command: library(kinship)

Note, for a system wide installation where one does not have access to installation paths for R, additional steps are needed:
Create a directory where extra libraries will be saved (e.g., extraLibs in $HOME)
Then, set the R_LIBS var in ~/.bashrc and you should be set.

Installing R

1. Download R from http://www.r-project.org/.
2. Untar the downloaded archive using command: $ tar -xvzf [R-version.tar.gz]
This will create a new directory named ./[R-version] in the current directory.
3. Enter newly created directory: cd [R-version]
4. Issue command: ./configure --prefix=/path/to/install (e.g., /home/afgane/R/[R-version])
In order to link BLAS and CLApack libraries, the following set of commands can be used (the export statements specify desired compiler, also make sure library paths are edited to match current resource):
export CC=icc
export CXX=icpc
./configure --prefix=/store/R/R-2.7.2 --enable-threads=posix --with-blas="-L/opt/intel/cmkl/10.0.011/lib/em64t -lmkl -lguide -lpthread" --with-lapack="-L/opt/intel/cmkl/10.0.011/lib/em64t -lmkl_lapack"
5. Type tow following commands to make and install:
$ make
$ make check
$ make install

Setting up SSH keys for authentication without passwords

1. Log into machine ONE
2. Issue command: ssh-keygen –t rsa
3. Change into ~/.ssh directory
4. Copy contents of file id_rsa.pub from machine ONE to file ~/.ssh/authorized_keys on machine TWO. Note, if this file already exists, simply add a new line to the bottom of it. Furthermore, make sure that the key being copied is all on one line.
5. Change permissions of ~/.ssh/authorized_keys on machine TWO to 700
6. Now, you are able to log in from machine TWO to machine ONE without a password

Getting BioPerf to execute

Upon executing script to start BioPerf, the following error came up:
$ ./use-bioperf.sh
You can do the following:
[R] Run BioPerf
[I] Install BioPerf on your architecture
(if your architecture is not PowerPc, x86)
[C] Clean outputs in $HOME/BioPerf/Outputs
[D] Display all versions of the installed codes
R
Please specify your architecture
[A] Alpha
[P] PowerPC
[X] x86
[H] Custom. (If you have already ran the install-codes.sh successfully
to completion, you can choose this option to run the binaries installed
on your system)
X
$HOME/BioPerf/Scripts/Run-scripts/run-bioperf.sh: line 21: syntax error near unexpected token `('
$HOME/BioPerf/Scripts/Run-scripts/run-bioperf.sh: line 21: ` echo There is no directory named $HOSTNAME-Binaries (created by install-codes.sh) in $BIOPERF/Binaries '


In order to get passed this, file $HOME/BioPerf/Scripts/Run-scripts/run-bioperf.sh should be edited on line 21 to replace parentheses with curly brackets (or similar or escaped). The same should be done on line 26 because parentheses are used there as well and are not escaped.

Add color to bash

In order to get syntax highlighting in bash when 'ls' command is executed, add the following line to (bottom of) ~/.bashrc file:
. /etc/profile.d/colorls.sh

Note: Make sure pointed file exists. Below is an example of one:

$ more colorls.sh
# color-ls initialization

alias ll='ls -l' 2>/dev/null
alias l.='ls -d .*' 2>/dev/null

COLORS=/etc/DIR_COLORS
[ -e "/etc/DIR_COLORS.$TERM" ] && COLORS="/etc/DIR_COLORS.$TERM"
[ -e "$HOME/.dircolors" ] && COLORS="$HOME/.dircolors"
[ -e "$HOME/.dircolors.$TERM" ] && COLORS="$HOME/.dircolors.$TERM"
[ -e "$HOME/.dir_colors" ] && COLORS="$HOME/.dir_colors"
[ -e "$HOME/.dir_colors.$TERM" ] && COLORS="$HOME/.dir_colors.$TERM"
[ -e "$COLORS" ] || return

eval `dircolors --sh "$COLORS"`
[ -z "$LS_COLORS" ] && return

if ! egrep -qi "^COLOR.*none" $COLORS >/dev/null 2>/dev/null ; then
alias ll='ls -l --color=tty' 2>/dev/null
alias l.='ls -d .* --color=tty' 2>/dev/null
alias ls='ls --color=tty' 2>/dev/null
fi


Setting Eclipse Keyboard Shortcut: Ctrl+Tab

Go to:
Window / Preferences / General / Keys

Scroll all the way down to the bottom. About 12th from the BOTTOM you will see category "Next Editor".
Double Click that line. The modify screen for that action will open up.
Set focus to the Key Sequence section, and hit CTRL-TAB, then click the ADD button. You should now see TWO commands for "Next Editor". You can now select the original one, and click Delete. (from here)

Get rid of horizontal line in Word 2007

"This line is actually... Wait for it...! A border! Don't ask me how it got there, but all you need to do is highlight the section around the line (just left mouse, and pull a nice big block round the area, provided there are no tables nearby where you want to preserve a border). Then just go up to the border-line selection icon in the toolbar and choose the one that clears all borders."
From http://word-tips-world.blogspot.com/2007/06/get-rid-of-that-annoying-horizontal.html

BLAST performance analysis configuration

All the files needed for BLAST performance testing (e.g., perl scripts, databases, different versions of BLAST software, ...) are located under my account on everest under ~/BLASTanalysis.
In order to run a BLAST test, a properties file (e.g., propertyFile) needs to be edited/provided. Properties file specifies the following data: database, input file, program, I/O params are specified, number of fragments and threads, and execution host.
Once configured, run run.pl script. This script will split the input file into specified number of fragments, generate LRM scripts, and submit the job:
perl run.pl {properties file}

Add multiple linux timing results (output in seconds)

If there are multiple files with timing results resulting from Linux 'time' command:
$ cat outputFile* | grep real
real 4m5.047s
real 2m51.264s
real 2m52.414s
real 2m52.293s

And these timings need to be converted into seconds and added into a single number (e.g., 761.018), use the following command:
cat outputFile* | grep real | cut -f 2 | sed 's/m/*60+/' | sed 's/s//g' | bc | awk '{ printf "%s", $0 "+" }' | sed '$s/.$/\n/' | bc

Convert output of linux 'time' command to show seconds only

After executing Linux 'time' command, the following output is obtained:
real 2m52.293s
user 1m39.670s
sys 0m4.370s

Use the following command to extract value for 'real' time only and convert the output to seconds only (i.e., convert 2m52.293s into 172.293):
cat outputFile* | grep real | cut -f 2 | sed 's/m/*60+/' | sed 's/s//g' | bc

Running JaCoP

From command prompt, while residing in the same directory as all the JaCoP java files, compile all java files using the following command:
C:\Documents and Settings\afgane\My Documents\Enis\Projects\JaCoP>java
c -classpath ".;C:\Documents and Settings\afgane\My Documents\Enis\Projects\JaCo
P\JaCoP.jar" *.java

Run any of the files using the following command, for example:
C:\Documents and Settings\afgane\My Documents\Enis\Projects\JaCoP>java
-classpath ".;C:\Documents and Settings\afgane\My Documents\Enis\Projects\JaCoP
\JaCoP.jar" Queens

After compiling it from command line, it can be run directly from Eclipse too.

Adding host info to ganglia and gmond

GridWay was not reporting OS name and OS version for mileva. After making sure gmond was running and MDS was actually being populated by available information (info on this is available in earlier post), you have to make sure all the information that is needed by GridWay is being provided by ganglia.
For example, I was noticing on the problem host that telnet localhost 8649 was giving the followgin output:
[METRIC NAME="sample-metric" VAL="Linux mileva-0-3.local 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 26 14:14:47 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[METRIC NAME="sample-metric" VAL="Linux mileva-0-2.local 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 26 14:14:47 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

While output on working host was giving the following output:
METRIC NAME="os_name" VAL="Linux" TYPE="string" UNITS="" TN="401" TMAX="1200" DMAX="0" SLOPE="zero" SOURCE="gmond"/]
[METRIC NAME="sample-metric" VAL="Linux everest-0-3.local 2.4.21-20.ELsmp #1 SMP Sat Sep 18 18:28:16 PDT 2004 x86_64 x86_64 x86_64 GNU/Linux
[METRIC NAME="os_name" VAL="Linux" TYPE="string" UNITS="" TN="943" TMAX="1200" DMAX="0" SLOPE="zero" SOURCE="gmond"/]
[METRIC NAME="sample-metric" VAL="Linux everest-0-13.local 2.4.21-20.ELsmp #1 SMP Sat Sep 18 18:28:16 PDT 2004 x86_64 x86_64 x86_64 GNU/Linux

This means ganglia is not providing needed information, so do the following to include needed info. (Following was done on Ganglia v3.0.4)
Edit /etc/gmond.conf (this is the default location) file and add the following text toward the bottom of the document (among all the other similar entries):
metric {
name = "os_name"
value_threshold = 10.0
}

It is likely that os_release will need to be added in the same fashion as was described for os_name.

Edited config file (/etc/gmond.conf) needs to be copied to all the nodes of the cluster before it will take effect.

Then restart gmond (/etc/init.d/gmond restart) on all the nodes and wait for GridWay info to be updated...

Populating gwhost in GridWay

Source of information when executing gwhost command in GridWay comes from two major sources:
From $GLOBUS_LOCATION/libexec/globus-scheduler-provider-[sge OR fork] and $GLOBUS_LOCATION/etc/globus_wsrf_mds_usefulrp/gluerp.xml. The gluerp.xml script is the one that provides info about static resource info and it obtains infrom from a provider such as ganlgia (gmond process) or hawkeye. Information available from ganglia can be checked by executing: telnet localhost 8649
gluerp.xml needs to have a line such as the following enabled in order for gwhost to be properly populated:
.../...
java org.globus.mds.usefulrp.glue.GangliaElementProducer
.../...

The globus-scheduler-provider-[sge OR fork] script provides mostly dynamic information used for scheduling.