wikipedia

Support Wikipedia

Friday, January 30, 2009

Search and grouping results in command line unix

Ever wanted to extract some metrics from log files and group the results. Also print the count for each group. All this without using a loop statement!!!
Just using command line. Here is how it can be done.

In the results, the first number is the count and the second is the group.

For starters here is a bit of explanation...The grep is looking for the string Failed to connect in service.log file. sed is used to make a group. In this case it is the station number. Then the results are sorted, finally uniq gives the unique rows and the -c gives us the count for each group.
BTW this was done on Ubuntu.

$ grep 'Failed to connect' service.log | sed 's/.*Station :\(.*\)/\1/' | sort | uniq -c
count
station
  69    10 - Host : 123.456.103.106 -  Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
356 11 - Host : 123.456.103.105 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
24 1 - Host : 123.456.103.107 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
24 2 - Host : 123.456.103.98 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
23 3 - Host : 123.456.103.99 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
24 4 - Host : 123.456.103.100 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
25 5 - Host : 123.456.103.101 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
25 6 - Host : 123.456.103.102 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
23 7 - Host : 123.456.103.97 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
34 8 - Host : 123.456.103.104 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect
28 9 - Host : 123.456.103.103 - Failed to connect - Error :java.net.ConnectException: Connection timed out: connect

Tuesday, January 27, 2009

Regular Expression Cheat sheet for Java

Regular expression is quirky enough. On top of that it's different for each platform. Even text editors have their own flavors. One way to keep your sanity is to have a cheat sheet for each environment. Here is one for Java and a generic one.

Friday, January 2, 2009

System Clock vs Real Time clock

Applications which are time sensitive or store and process data that is periodic need to keep up with the Real Time clock (RTC) (hardware clock). Time servers can be used to synch the system clock with external time servers (for example www.nist.gov) but there is no way for applications to synchronize the system clock with the RTC.

Typically at startup the operating system (read windows) sets the system clock to the RTC. After that the operating system generates interrupts(depending on the flavor of windows it could range from 10 ms to 15 ms or more) to update the system clock. Also if Windows finds the time lag between the two clocks to be at least 1 minute (Where exactly this value is set is not known publicly!!!), then it resets the system clock to the RTC. But if you want to keep up the system time much more accurately (say 100 ms, then there is no way to do that.


ClockMon is a simple utility app that can be used to monitor the two clocks or resync the system clock when it drifts away from the RTC. It is a free program that can come quite handy when troubleshooting clock drift issues in applications running on Windows platforms.