My blog has moved!

You should be automatically redirected in 6 seconds. If not, visit
http://www.mattdturner.com/wordpress
and update your bookmarks.

AdSense

Pages

Monday, April 25, 2011

Useful .bashrc Functions

For those of you that read my other post about useful .cshrc functions, you know that I like to automate as many processes as possible.  If there is something redundant that I do frequently, I will script it or make a function to take care of it.

In an effort to assist those of you out there that like to automate processes but use bash instead of C-Shell, I have compiled a list of some useful functions to stick in your .bashrc file.  Here is a list of the functions that this post will contain:

  1. Extract:  This function will determine the type of archive you are dealing with, and issue the corresponding command to extract it.
  2. Compress:  This function will create a .tar.gz archive of whatever files you give.
  3. Swap:  This function will swap the names of 2 different files.
  4. Clock:  This is just a random function that will display a digital clock in your terminal window and update every second.
  5. Cpg:  This function will copy a file to a directory, and also cd you to a directory.
  6. Mvg:  This function works like Cpg, but moves the file instead of copying it.
  7. Findit:  This function will search for a file with the given pattern in the name.

Finding Text on Any Website on the iPad, iPhone, and iPod

One rather frustrating thing that I have noticed with the iPad (and iPhone) is the inability to search for text within Safari.  Well as it turns out, a simple javascript bookmark is all you need to add this functionality to any iOS device (just like my post on viewing source code).

To enable finding text within any website, follow the below instructions:

  1. Open Safari (pretty self explanatory).
  2. Create a new bookmark.
  3. In the Title window, write whatever you want your bookmark to be named (I chose "Find on Page")
  4. In the address bar add the following javascript code:  
    javascript:void(s=prompt('Find text:',''));s='('+s+')';x=new RegExp(s,'gi');rn=Math.floor(Math.random()*100);rid='z' + rn;b = document.body.innerHTML;b=b.replace(x,'$1');void(document.body.innerHTML=b);alert('Found ' + document.getElementsByName(rid).length + ' matches.');window.scrollTo(0,document.getElementsByName(rid)[0].offsetTop);

NOTE: As with the post on viewing source code, I did not develop this javascript bookmark. All credit goes to Matthew Panzarino.

Friday, April 22, 2011

How to View HTML Source Code on iOS | iPad, iPhone, iPod

Occasionally, when browsing the web I have a need to view the source code for a certain page.  However, there is no intrinsic way to do so on the iOS version of safari.  Well, using some simple javascript bookmark you can view the source code for any webpage.

To view the source code, follow the below instructions:

  1. Open Safari (duh!)
  2. Click the "Add Bookmark" button in Safari.
  3. Give the bookmark whatever name you wish; I chose the title "View Source".
  4. In the address bar of your new bookmark, paste the following code: 
    javascript:var%20sourceWindow%20%3D%20window.open%28%27about%3Ablank%27%29%3B%20%0Avar%20newDoc%20%3D%20sourceWindow.document%3B%20%0AnewDoc.open%28%29%3B%20%0AnewDoc.write%28%27%3Chtml%3E%3Chead%3E%3Ctitle%3ESource%20of%20%27%20%2B%20document.location.href%20%2B%20%27%3C/title%3E%3C/head%3E%3Cbody%3E%3C/body%3E%3C/html%3E%27%29%3B%20%0AnewDoc.close%28%29%3B%20%0Avar%20pre%20%3D%20newDoc.body.appendChild%28newDoc.createElement%28%22pre%22%29%29%3B%20%0Apre.appendChild%28newDoc.createTextNode%28document.documentElement.innerHTML%29%29%3B
  5. Now just navigate to whichever page you wish to view the source of, click the bookmarks button, and select your new View Source bookmark.
NOTE: I did not create this javascript bookmark.  I only discovered it while browsing the internet.  For the source of the javascript bookmark, please see Rob's Blog.

Thursday, April 21, 2011

My Top 10 iPhone Jailbreak Apps | April 2011

About once a month or so, I search around online for the best iPhone jailbreak apps.  I don't usually have the time to search through all of the new released in Cydia, so I tend to rely on other people's posts.

In an effort to provide some app ideas to new jailbreakers, below is a list of my 10 favorite jailbreak apps for the iPhone (in no particular order).


1. LockInfo

To me, this is an absolute must have app for the iPhone.  What this app does is puts many iPhone notifications together into a single user-friendly UI.  There are many different plugins for LockInfo, such as a plugin that displays RSS Feeds, a plugin that displays tasks from the AppStore 2do app, etc.

For a list of all native plugins, please see Plugins.

Mac Bluetooth Keyboard/Mouse Battery Level in Terminal

There are a few reasons why somebody might want to check the current battery percentage of a bluetooth mouse and/or keyboard within Terminal on a mac.  Some that come to mind are for use in GeekTool, if you are connected to the computer remotely through ssh, etc.

Well as it turns out, its actually very easy to do.  The tool that we will use to check the battery percentage is called "ioreg".  The Mac OS X manual describes ioreg as a tool that "displays the I/O Kit registry. It shows the hierarchical registry structure as an inverted tree."

There are many options available with the ioreg tool, however this article will just be discussing how to check the battery percentage of different devices.  See the below text for the code to view the battery percentage:

matthewturner/~:  ioreg -n "IOAppleBluetoothHIDDriver" | grep -i "batterypercent" | sed 's/[^[:digit:]]//g'
49
matthewturner/~:  ioreg -n "BNBMouseDevice" | grep -i "batterypercent" | sed 's/[^[:digit:]]//g'
45
matthewturner/~:  ioreg -n "BNBMouseDevice" | grep -i "batterypercent"
    | |   |     | | |   |   "BatteryPercent" = 45

There are a few things that should be noted about the above commands.

Wednesday, April 20, 2011

Automatically Make Shell Scripts Executable

I strongly believe that a script should be created to handle any repetitive process in the terminal.

By default, when you create a new script, it does not have executable permissions.  So for each script that you create, you then have to chmod the file to be executable.  However, its kind of hypocrital to say that a script should be created to replace any repetitive process and then not figure out a way to automate the 'chmod' command when creating a script.

Well, as it turns out, a simple addition to your vimrc file will automatically make any newly created script an executable (so long as you have the '#!' in the file).

" automatically give executable permissions if file begins with #! and contains
" '/bin/' in the path

au BufWritePost * if getline(1) =~ "^#!" | if getline(1) =~ "/bin/" | silent !chmod a+x <afile> | endif | endif
au BufWritePost * if getline(1) =~ "^#!" | if getline(1) =~ "/usr/bin/" | silent !chmod a+x <afile> | endif | endif

So just copy the above code into your .vimrc file (each uncommented line ends in 2 endif commands) and now you can create scripts that are executable by default.
 

Tuesday, April 19, 2011

How to Fix "Output Conversion Error"

As part of my research for my Ph.D. I am on a team that is currently developing an adjoint of the EPA's CMAQ air quality model.  In the process of integrating all parts of the model into the full adjoint model, I ran into an error that was rather difficult to resolve.

Running the model would result in many occurances of the following error:

forrtl: error (63): output conversion error, unit -5, file Internal Formatted Write
Image              PC                Routine            Line        Source             
ADJOINT_FWD        00000000009B34BD  Unknown               Unknown  Unknown
ADJOINT_FWD        00000000009B1FC5  Unknown               Unknown  Unknown
ADJOINT_FWD        0000000000969210  Unknown               Unknown  Unknown
ADJOINT_FWD        000000000092AADF  Unknown               Unknown  Unknown
ADJOINT_FWD        000000000092A312  Unknown               Unknown  Unknown
ADJOINT_FWD        000000000095305A  Unknown               Unknown  Unknown
ADJOINT_FWD        00000000005D9F94  ckdesc3_                  138  ckdesc3.f
ADJOINT_FWD        00000000005A9FD1  open3_                    216  open3.F
ADJOINT_FWD        000000000047B395  chk_files_impl_mp         170  CHK_FILES_IMPL.F
ADJOINT_FWD        0000000000485060  chk_files1_mp_chk         347  CHK_FILES.F
ADJOINT_FWD        00000000005666CB  vdiff_                    369  vdiffacm2.F
ADJOINT_FWD        0000000000496B7E  sciproc_                  228  sciproc.F
ADJOINT_FWD        000000000048DDB5  MAIN__                    205  driver_fwd.F
ADJOINT_FWD        0000000000404A1C  Unknown               Unknown  Unknown
libc.so.6          0000003FE9E1D994  Unknown               Unknown  Unknown
ADJOINT_FWD        0000000000404929  Unknown               Unknown  Unknown


     >>> WARNING in subroutine CRTFIL3 <<<
     Error creating netCDF variable for file ADJ_VDIFF_CHK   
     Illegal data type    0

      
     *** ERROR ABORT in subroutine CHK_FILE_CREATE_
     Could not open ADJ_VDIFF_CHK file
     Date and time  13:00:00  July 22, 2001   (2001203:130000)


I spent a lot of time searching online, however I was unable to find a solution for my problem.  After days of debugging, I finally found the source of the problem.

Useful .cshrc Commands

Throughout the years of using the command line in both Linux and Mac, I have spent quite a bit of time modifying my .cshrc file to my liking. 

This is mainly just a list of some commands that I think are useful.

# Prompt - Displayed as “@:
set prompt="`whoami`@%B`hostname -s`%B:" #Custom Prompt settings

alias lsl 'ls -larth'
alias qstatme 'qstat -u mturner'
alias qsubm 'qsub -I -V -l walltime=08:00:00,nodes=1:ppn=24'
alias quota '/usr/bin/quota -s -u mturner'


The “lsl” alias should be self explanatory.
“qstatme” displays the current PBS queue for my username only.
“qsubm” will start an interactive PBS job, using 1 full node, and requesting 8 hours.
“quota” will display my quota

# set title
alias cwdcmd 'echo -n "\033]0;$cwd\007"'


“cwdcmd” will change the title of the terminal window to match the current working directory

Monday, April 18, 2011

Stylus-R-Us BestStylus Review

Before I start, if you just want a list of pro's and con's for the stylus follow this link.

When I purchased my new iPad 2, I knew that I wanted to get the best stylus possible.  I use the iPad mostly for note taking right  now, and my finger just wasn't sufficient for taking hand-written notes on the tablet.  I spent a good amount of time researching various styli and reading reviews for the various styli.

I read a lot of good reviews for the Targus and Boxwave styli, however I still wasn't convinced.  Near the end of my search I happened upon a review of the styli sold at Stylus-R-Us (www.beststylus.com/).  The videos I watched of these styli seemed too good to be true.  And then I read reviews for the company.  Every single review that I saw for the company read as if it were a marketing ploy, and not a real review.

Well, I gave in and went ahead and purchased the New Hampshire stylus from Stylus-R-Us (the black one in the picture below).

Some Useful .cshrc Functions

If you use the command line a lot, there are numerous common commands that can be automated to save time.

Like what? Well, how about swapping the names of 2 files; or maybe unarchiving any filetype.

People who use bash are lucky, bash allows you to actually create functions in the .bashrc file. Those of use who use c-shell, however, have to deal with aliasing.

Below are a few “functions” that i have in my .cshrc file which save me a lot of time.

Swap 2 files


alias swap 'mv \!:1 tmp.1 | mv \!:2 \!:1 | mv tmp.1 \!:2'
Use: mturner@prospero: swap file1.f file2.f
Result: The filenames have been swapped for these 2 files


Move and go


alias mvg 'mv \!:1 \!:2 && cd \!:2'
Use: mturner@prospero: mvg file1.f ~/Desktop
Result: file1.f will be moved to the specified directory, and you are automatically cd’ed to the specified directory

Sunday, April 17, 2011

/etc/motd Formatting

It is good practice to announce server maintenance dates and scheduled downtime dates many days prior to the actual downtime. The easiest way to ensure that all users become aware of the scheduled downtime is to place an alert in /etc/motd.

For those of you that don’t know, /etc/motd is the file that gets printed to the screen when a user first logs into the system.

However, in the case of our server (possibly most servers), users tend to ignore the contents of motd since it very rarely changes. In order to attract the attention of any user that logs into the system, I have begun adding formatting (bold, underline, colors, etc) to the motd file on our server.

Formatting can be easily added to any motd using ANSI codes. For example, below is a copy of a sample maintenance announcement for our server:





Notice that the text “Server Maintenance” is what first got your attention. This is because of the different color, bold text, and blinking text (blinking text is not supported in IE, Chrome, and Safari).

Using the Debugging Tool Valgrind

Valgrind is an excellent, and sometimes necessary, tool for debugging troublesome programs. According to Valgrind’s website, the program can “automatically detect many memory management and threading bugs, avoiding hours of frustrating bug-hunting, making your programs more stable. You can also perform detailed profiling to help speed up your programs”.

Valgrind contains a variety of tools for use while debugging. The program can be used to debug a wide variety of programming languages including, but not limited to, C, C++, Java, Perl, Python, Fortran, Ada, etc. Using these tools, you can:

  • Detect memory errors (using the tool Memcheck)
  • Make programs run faster (using the cache and branch-prediction profilerCachegrind)
  • Detect thread errors (using the tool Helgrind and/or DRD)
  • Profile heap usage (using the tool Massif and/or DHAT)


How to Use Valgrind’s Memcheck Tool


Possibly the best thing about Valgrind is the fact that it is very easy to use. There is only 1 thing that you need to do in order to prepare your program for use with Memcheck; compile your program with the -g and -O0 compiler flags. That’s it. Now all you have to do is run your program using valgrind.

A Fix for Broken Bluetooth in Mac OS X

After updating my Mac to OS 10.6.7, I ran into a problem where the bluetooth on the computer would completely fail. I tried many times to disconnect and re-connect my mouse and keyboard, but the problem still persisted.

I tried deleting the pairing and re-pairing the devices, but I would get the following message:

”The pairing attempt was unsuccessful. This configuration of Mac OS X and Bluetooth software could be invalid or unsupported.”



Resetting the SMC seemed to fix the problem.

To summarize the steps listed in the link, do the following to reset the SMC on your MacBook:

  • Shut down the computer.
  • Ensure that the MagSafe power adapter is connected to the computer.
  • On the built-in keyboard, press the left Shift-Control-Option keys and the power button at the same time.
  • Release all keys and the power button at the same time.
  • Press the power button to turn on the computer.
Note: The LED on the MagSafe power adapter may change states or temporarily turn off when you reset the SMC.

How to Stop "Thanks for Flying Vim" Message

If you are like me, you probably do a lot of your coding on a remote computer. I have known some people who will download all files from a remote computer onto their local system, make changes in a text editor, and then upload the files back to the remote server. This is a very inefficient way of coding, and is just one of the reasons why i choose to write all of my code in VIM.

If you have ready my other posts (specifically Useful .cshrc commands), you know that I like to have the title of the terminal set to my current working directory. However, after exiting VIM, I always end up with the frustrating “Thanks for flying VIM” message in the title.

I had read on other blogs and forums that adding set notitle to your .vimrc file will prevent the “Thanks for flying VIM” message from showing up.

Well, that didn’t work for me. After a lot of scouring the internet, I was able to find the solution.
let &titleold=getcwd()
Adding this to your .vimrc file will, upon exit from vim, set the terminal title to the current working directory.

More .cshrc Commands

The following .cshrc commands are ones that I feel should be included in the default .cshrc file for the entire system (/etc/csh.cshrc).

# set file completion
set filec
# set to list all possible matches
set autolist
# set auto correct
set autocorrect

Each command is fairly self-explanatory.

Group Access to All Files

On every cluster, there is most likely a group of users who want to allow the entire group to have read, write, and execute privileges to every file they create. However, changing the permissions on each file individually is a major nuisance.

You can, however, change the default set of permissions for every file and folder that a user creates. This is done using the “umask” command.

For our air-quality modeling group here at CU Boulder, each user’s primary group is set to “henze”. Then, within each user’s .cshrc or .bashrc file, the umask command is used to modify the default permissions.

In our case, we use
        umask 0007
This makes it so that every file or folder that the user creates has permissions 770 (drwxrwx---).

This is a simple change that helps keep groups of users happy.

Cluster Status Check Script

Lets face it, nodes randomly go down without warning sometimes. If you’re like me, you aren’t going to consistently check the node status throughout the day. There have been cases where a node has gone down, and I didn’t know about it for a week.


This is very problematic, especially when you run a system that has a small number of nodes that are always in use.

In an effort to resolve both the problem of me not knowing there is a dead node without requiring me to check the node status multiple times a day, I developed a python script to check if there is a dead node. The script is set to run every day at midnight, and if there is a dead node it will send me an email.



Download the script here.


The only edits you should have to make in order to get the script to run on your system are lines 21 - 35.
  • Line 21: Change to match your username
  • Line 22: Change to match your email address.
  • Lines 25-35: Change to whatever email message you want sent.
Now all you have to do is add the script to cron.

And you’re done! 

Killing Non-queued Processes Script

When we first setup our group's HPC here at CU Boulder, we ran into issues where we would have processes hang on the compute nodes.  The jobs would no longer be in the queue, however the processes would still be running on the nodes.  This made other jobs that were supposed to be running on said nodes (queued jobs) run much slower.

As this was a problem that occurred relatively frequently, it was necessary to develop a script that could be placed in cron that would automatically kill the non-queued processes on the compute nodes.

Download the script here.


As a starting point for the script, I found a python script that had been developed by David Black-Schaffer for use on the ROCKS Linux Cluster at Stanford (David's Website).  This script was developed to use the SGE queue environment.  However, our group had decided to use the Torque PBS queue system.