Wednesday, January 27, 2010

Prepend every line in a file with some text

Say you have a file 'files.txt' looking like this:
p1-c-b-1.fastq.gz
p1-c-b-2.fastq.gz
p1-c-c-1.fastq.gz


Now, you would like to add a full URL path to each of those lines, prepending existing text.
Use awk as follows:
$ cat files.txt | awk '{ print "http://www.complete.url.com/", $0; }' > links.txt

You get a file names 'links.txt' looking like this:
http://www.complete.url.com/ p1-c-b-1.fastq.gz
http://www.complete.url.com/ p1-c-b-2.fastq.gz
http://www.complete.url.com/ p1-c-c-1.fastq.gz


To get rid of introduced extra white space, use sed:
$ cat links.txt | sed 's/ //g'

Tuesday, January 26, 2010

Monitor network traffic on OpenSolaris from command line

To check a summary of network activity for a NIC, use:
# netstat -i 1
    input   xnf0      output       input  (Total)    output
packets errs  packets errs  colls  packets errs  packets errs  colls
1111615 0     745655  0     0      1111684 0     745724  0     0    
399     0     306     0     0      399     0     306     0     0    
416     0     343     0     0      416     0     343     0     0 


To expolore actual packets crossing the network, use snoop command. It captures both TCP and UDP traffic. It is a tool that is shipped with Solaris.

Here is sample output:
# snoop
Using device xnf0 (promiscuous mode)
cbcb-vs.umiacs.umd.edu -> domU-12-31-39-04-EC-47.compute-1.internal TCP D=60088 S=8021 Ack=4204280156 Seq=2401233531 Len=1460 Win=92
domU-12-31-39-04-EC-47.compute-1.internal -> cbcb-vs.umiacs.umd.edu TCP D=8021 S=60088 Ack=2401246671 Seq=4204280156 Len=0 Win=49640 Options=

domU-12-31-39-04-EC-47.compute-1.internal -> dhcp243113.rollins.emory.edu TCP D=49769 S=22 Push Ack=4234202219 Seq=2467318705 Len=192 Win=49232




It shows source and destination servers, type of traffic, source and destination ports, as well as some packet info.