Using PS to output human readable memory usage for each process using AWK

Robert Coop    linux, scripts     awk, bash, linux, programming, ps, shell script

I ran into a little issue: I wanted to list the memory usage of each process, but I wanted it to be output in human readable form (as in “df -h”).  So, after a little Googling I found a forum thread which put the idea of using awk.  A little bit later I had the solution:

Here’s what you see with normal ps:

$ ps u

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
kes1 12394 0.0 0.0 107356 2052 pts/1 Ss 08:15 0:00 -sh
kes1 12648 0.5 3.2 720808 527356 pts/0 S+ 08:16 0:16 /usr/lib64/R/bin/exec/R
kes1 13682 0.0 0.0 107360 2016 pts/3 Ss 08:18 0:00 -sh

Here’s what we want to see:

USER    PID     %CPU    %MEM            VSZ             RSS     TTY     STAT    START   TIME    COMMAND
kes1    12394   0.0     0.0            104.84Mb          2.00Mb  pts/1   Ss      08:15   0:00    -sh
kes1    12648   0.7     3.2            703.91Mb        515.00Mb  pts/0   S+      08:16   0:16    /usr/lib64/R/bin/exec/R
kes1    13682   0.0     0.0            104.84Mb          1.97Mb  pts/3   Ss      08:18   0:00    -sh

And here’s how we make that happen with awk:

ps u | awk '{
for ( x=1 ; x<=4 ; x++ ) { printf("%s\t",$x) } 
for ( x=5 ; x<=6 ; x++ ) {  if (NR>1) { printf("%13.2fMb\t",hr=$x/1024) }  
else { printf("\t%s\t",$x)  } 
}
for ( x=7 ; x<=10 ; x++ ) { printf("%s\t",$x) } 
for ( x=11 ; x<=NF ; x++ ) { printf("%s ",$x) } 
print "" 
}'

What does this awk script do?

There are a few different things going on here.  First, we want to print out all the non-memory fields as tab-separated text.  Second, we want to print out the memory as formatted kb and not bytes.  Also, we want everything to line up. Finally, we want to print out the process name normally (using space and not using tab separation).

First, the script prints the first 4 fields:

for ( x=1 ; x<=4 ; x++ ) { printf("%s\t",$x) }

Then, we check to see if this is the header line or not. If it is the header line, then NR (the number of records read) will be 1. If it's not, then NR will be greater than 1. For the header, we print out the text (adding some more tabs to line things up). For the memory fields, we use printf to format the byte count as human readable text.

for ( x=5 ; x<=6 ; x++ ) {  
if (NR>1) { printf("%13.2fMb\t",hr=$x/1024) }  
else { printf("\t%s\t",$x)  } 
}

Finally, we print the remaining PS fields as before, but then we switch to space-delimited fields for the process name and arguments, making sure to add a newline at the very end.

for ( x=7 ; x<=10 ; x++ ) { printf("%s\t",$x) } 
for ( x=11 ; x<=NF ; x++ ) { printf("%s ",$x) } 
print ""

Nothing to it!

Howto: Mount your Android SD card under linux via wifi

Robert Coop    Android, linux, Ubuntu     android, linux, samba, ubuntu

This guide will get you from having an Android phone with a Samba sharing app installed (such as the one I currently use: https://market.android.com/details?id=com.funkyfresh.samba) and an Ubuntu (or any generic Linux) machine with Samba installed to being able to mount your SD card via WiFi. We will do this by: setting the computer to use netbios names for IP address resolution, testing the mount with mount.cifs, and then adding a permanent entry to /etc/fstab.

My setup: HTC Inspire 4g, rooted running Cyanomodgen 7; Samba Filesharing on Android app (from marketplace, free, version 110130-market); Ubuntu 11.04 (with samba package installed)

Prerequisites

First, you need a Samba server app for your phone (I’m using https://market.android.com/details?id=com.funkyfresh.samba, which seems to work well enough), then you need to go into your Samba app on the Android and set a netbios name for your phone (e.g. I’m using “bcoop-android”, so replace that with whatever name you set). For security reasons, you also want to set a user (e.g. “bcoop”) and a password required to access the share. If you have the option, you need to give the shared directory a name (on my app the name is fixed as “sdcard”).  You’ll also need to make sure that your computer has a samba client installed (this can be installed in Ubuntu by installing the smbclient package).

Using Netbios for IP address resolution

Secondly, we need to tell the computer to try to perform DNS lookups using netbios names if all else fails. You can tell if you need this step by running the command “ping bcoop-android” (making sure your phone is connected to the same network as the desktop via wireless and that the Samba app on the phone is running). If you receive a “unknown host” error, then the desktop is not able to look up names via netbios, which is a simple fix. Run the command

sudo gedit /etc/nsswitch.conf

and look for the line that starts with hosts:. At the end of this line, add wins. You should end up with the line looking something like:
hosts: files [...] wins

This will tell your machine to use Netbios name lookup if all else fails. You want to make sure to add wins at the end of the string of methods so that it does not check this before other methods. Save the file and close gedit.

Update: After an hour or so, my connection started timing out and I couldn’t remount the share.  I was confused about what was going on, but noticed that when I tried to ping the netbios name I suddenly got a response from a 209.XXX.XXX.XXX IP address, and not from my phone.  Long story short, it turns out that my lovely ISP (Comcast) has a policy of hijacking domain names that don’t exist so that they can redirect browsers to a search page with ads.  A side effect of this is that all DNS resolution requests are answered, regardless of whether they exist or not.  This causes the computer to assume that the request has been answered and not to look at the wins netbios name for a possible IP address.  The solution to this was to put wins just before the dns entry in the hosts line.

You should now be able to run the ping command and have the computer try to ping an IP address (it doesn’t matter if you receive a response or not, we’re just checking to see that the computer can translate the netbios name to the ip address of the phone).

Note: If you’re not able to get this to work, you can still move on, but just use the phone’s IP address instead of netbios name for the server.  It will be necessary to either continually change the IP address to the phone’s IP, tell your router to assign the phone the same IP address always, or to use some other method to ensure that the phone’s IP address remains correct.

Testing things out: Temporarily mounting the phone

To test things out, we need to create a directory to use as a mount point.  Run the command sudo mkdir /media/android to do this (using a different directory if you’d prefer). Now, we want to manually mount the phone in this directory. There are a couple different ways to do this, depending on how you want the file permissions to work.  I’ll list the different commands you can use, and you can see the below section for further discussion about which might be best.  You will need to modify this command with some specifics for your setup, see the section immediately afterwards.

To not allow any other users to access your files (the recommended method)

sudo mount -t cifs //bcoop-android/sdcard/ /media/android  -o user=bcoop,uid=bcoop,gid=bcoop,nounix,file_mode=0770,dir_mode=0770

To avoid using ‘nounix’, but allow others to read (but not write) your files

sudo mount -t cifs //bcoop-android/sdcard/ /media/android  -o user=bcoop,uid=nobody,gid=bcoop

To disable permission checking entirely (anyone can read/write your files)

sudo mount -t cifs //bcoop-android/sdcard/ /media/android  -o user=bcoop,noperm

For all methods

You’ll need to replace some parts of this command with your setup information:

• //bcoop-android/sdcard should be your phone’s netbios name (or IP address) followed by the share name:  //NETBIOS_NAME/SHARE_NAME
• /media/android should be your mount point directory
• user=bcoop should be the user name that you set up on the phone for the Samba share:  user=PHONE_SAMBA_USER
• uid=bcoop,gid=bcoop should be your computer user’s name and group (these are likely the same on a typical setup): uid=COMPUTER_USER,gid=COMPUTER_GROUP
• uid=nobody should be the name of a fake user on your computer

After running the command, you’ll need to enter your sudo password, then your password for the phone’s samba share.  If all goes well, you should see no error messages then be able to run

ls /media/android

and see the contents of your phone. In that case, you’re ready to set the share up permanently. If you don’t mind running the mount command every time, you can just stop here.

Notes regarding file permissions

(This section can safely be skipped if you’re not interested in knowing any background about how things work behind the scene…)

When I was trying this out, the thing that took the most amount of time to figure out is the file permissions used on the phone.  When mounting a SMB (Samba) share, there are a few options when it comes to file permissions: accept the uid/gid (user and group owner id) from the phone, force the uid/gid to map to specific users/groups on the computer, or ignore the permissions reported by the phone entirely.

The most convenient option is to ignore the permissions entirely, but it is also the least secure: it would allow any program or user on your computer to have full access to the files on the phone when it is mounted.  The typical approach is to map the user and group from the phone to be equivalent to your computer user.  However, I noticed something odd about the way the permissions are reported on my setup.  I’m not positive if this is just some eccentricity of my specific setup, but the permissions reported by my phone have the user set with no read/write/execute permission, the group set with full read/write/execute permission, and everyone else set with just read/execute permission.  (For comparison, the typical setup is user with full permission, group with either full or read/execute permission, and everyone else with either read/execute permission or no permissions at all.)  So, if one maps the uid/gid to the computer user’s uid/gid then the result is that the current user will have no permissions at all.  One solution is to map the gid to the computer user’s gid, but to map the uid to some fake/unused user (I used ‘nobody’, which is a standard and safe bet).  This results in you having full access to the phone and other users being able to read but not modify the contents, and has the advantage of retaining the maximum amount of functionality (i.e. it doesn’t disable some behind-the-scenes filesystem functionality).  An alternative solution is to disable ‘CIFS Unix Extensions’ and manually set the file/directory permissions as well as the uid/gid.  This has the advantage of allowing you to explicitly remove read permission from other users if desired, but has the possible disadvantage of disabling something that is required (though I have no idea if that is likely to happen or even really possible; please leave a comment if you know something about this that I don’t).

Setting things up permanently

To permanently save these settings, we need to create a credentials file to safely hold your samba share’s username and password and we also need to add the mount information to /etc/fstab so that the system is aware of the settings.  To safely store your credentials, we want to create a file that only your user can read which holds your username and password.  To do that, run

gedit ~/.android_credentials

and add the following to this file:

username=YOUR_USERNAME
password=YOUR_PASSWORD

Save the file, close gedit, and run the command

chmod 0600 ~/.android_credentials

to make sure that only you can read that file.

Now, to save the information into fstab, run

sudo gedit /etc/fstab

and add the following line to the file:

//bcoop-android/sdcard/	/media/android	cifs	credentials=/home/bcoop/.android_credentials,uid=bcoop,gid=bcoop,nounix,file_mode=0770,dir_mode=0770,user,noauto	0	0

(Note that you may need to change the options if you’d like something other than the recommended method from above, and you’ll need to replace credentials=/home/bcoop/.android_credentials with the correct path to your credentials file. Also note that the trailing slash on //bcoop-android/sdcard/ is very important. If you forget this trailing slash then you cannot unmount the share as a regular user.)

If all goes well you should now be able to run

mount /media/android

and access your phone’s SD card contents in /media/android. Remember that you need to unmount this by running

umount /media/android

when done. Enjoy!

Increase your productivity in Linux by 200% or more with a single command!

Robert Coop    linux     linux, productivity, reddit, slashdot

I found an amazing way to increase productivity by a large amount with one simple command! Simply run this command:

sudo bash -c "echo -e \"### Sorry guys, I'll return when I have less deadlines\n127.0.0.1 reddit.com\n127.0.0.1 www.reddit.com\n127.0.0.1 slashdot.org\" >> /etc/hosts"


I’m not quite sure what it is about this, but I immediately noticed work was getting done much sooner!

A note for those not familiar with hosts and other sarcasm-deficient folks: This command makes the computer use localhost for the IP address of Reddit and slashdot, preventing you from accessing those sites.  In a pinch, this command can help people like me that suffer from a lack of self-control when it comes to online distractions…  I’ll return some day…

Use an Accurate Satellite View of the Earth as your GNOME Background

Robert Coop    linux, scripts, Ubuntu     background, bash, desktop, gnome, linux, shell script, ubuntu

This site provides an accurate satellite view of the Earth; you can see the current areas that are in sunlight and darkness, as well as being able to see the current weather patterns (updated from actual satellite weather imagery). Here’s a script that will download the latest image and set it as your GNOME background.
read more

Reboot or Hibernate into Windows from Ubuntu Easily

Robert Coop    linux, scripts, Ubuntu     bash, dual boot, hibernate, linux, shell script, ubuntu

It’s often the case that I’m working on something in Ubuntu and need to switch over to Windows. For whatever reason, I got it in my head that it was too difficult to select hibernate or reboot from the power menu, wait for the system to power down, select Windows from the boot menu list, and wait for Windows to boot up. I wanted to be able to click on an icon, go get a sandwich, and walk back to a Windows login.

So, here are two scripts that accomplish this. When run, they will locate the boot entry for your Windows system, select that as the default boot entry, and either reboot or hibernate (with a restart after hibernation).
read more

Converting WMA files to MP3 on Linux (specifically Ubuntu) and retain tag information

Robert Coop    linux, scripts, Ubuntu     bash, conversion, lame, linux, mp3, mplayer, mutagen, shell script, ubuntu, wma

I recently needed to convert a bunch of (DRM-free) WMA files to MP3 format on Ubuntu, and there’s no good tool out there to do it.  There are plenty of tutorials that tell you how to convert from WMA to WAV with Lame, then from WAV to MP3 with mplayer, but none of these tutorial talk about how that will cause you to lose all the WMA artist/album/title data.  I needed a tool that would keep the tagged data;  so, I made one.  It requires mplayer, lame, and mutagen-inspect, all of which are available from aptitude.

So, here’s my script: wma2mp3
read more