Monday, January 31, 2011

Advanced overclocking for DroidX

If you know your way around android terminal, overclocking your Droid X without a helper app is really not that difficult.  Here are the steps:
  1. Download and install Milestone Overclock (this is just so you can grab the kernel module).
  2. Find overclock.ko from the milestone directory and move/copy it over to /system/lib/modules.  You can optionally uninstall Milestone Overclock now - you no longer need it.
  3. Run the following script (be sure to substitute A, B, C, D with your own vsels, and W, X, Y, Z with your own frequencies).

    #!/system/bin/sh
    insmod /system/lib/modules/overclock.ko
    echo 0xc05a8dcc > /proc/overclock/mpu_opps_addr
    echo 0xc0048a00 > /proc/overclock/omap2_clk_init_cpufreq_table_addr
    echo 0xc059b258 > /proc/overclock/cpufreq_stats_table_addr
    echo 0xc02d1104 > /proc/overclock/cpufreq_stats_update_addr
    echo D > /proc/overclock/max_vsel
    echo Z00000 > /proc/overclock/max_rate
    echo 1 W00000000 A > /proc/overclock/mpu_opps
    echo 2 X00000000 B > /proc/overclock/mpu_opps
    echo 3 Y00000000 C > /proc/overclock/mpu_opps
    echo 4 Z00000000 D > /proc/overclock/mpu_opps
    echo 0 Z00000 > /proc/overclock/freq_table
    echo 1 Y00000 > /proc/overclock/freq_table
    echo 2 X00000 > /proc/overclock/freq_table
    echo 3 W00000 > /proc/overclock/freq_table
My values: W = 3, X = 6, Y = 9, Z = 12, A = 22, B = 33, C = 48, D = 63

If you want the new settings to stick at boot, you'd want to put the script in /etc/init.d somewhere.  You can name it 99overclock or something like that.

As for how you can determine the best frequencies and vsels, you'll have to experiment.  If you're lazy, I recommend using something like Quickclock - the developer, Paul, is an awesome guy. :)

Finally, here's a nice trick.  The numbers in the following file will tell you your usage pattern per frequency:
/sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_state
Here's an example:
bash-3.2# cat  /sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_state
1200000 1424737
900000 87456
600000 182945
300000 4688410
You will notice that your phone spends most of the time at the lowest clock frequency and rarely will it use the middle ones.  Interesting, but expected.

Saturday, January 29, 2011

Android memory manager (low memory killer)

Android memory manager works pretty hard and is constantly killing processes in the background to ensure that you have enough memory to run your apps and also to keep the phone running snappy.

If you're as much of a control freak as I am, you kinda want to see which processes it's killing in the background.  It's actually pretty easy.  You can use either of the following methods:
  1. dmesg
  2. logcat
Both commands should give you similar output.  Here's what happens when I run it on my Droid X: 

bash-3.2# logcat -t 1000 | grep "has died"
01-29 15:01:09.577  1278  1283 I ActivityManager: Process com.motorola.calendar (pid 8595) has died.
01-29 15:07:46.957  1278  1278 I ActivityManager: Process com.dropbox.android (pid 8606) has died.
 

bash-3.2# dmesg | grep sigkill
<4>[58305.115783] send sigkill to 8595 (torola.calendar), adj 13, size 3198
<4>[58702.255462] send sigkill to 8606 (dropbox.android), adj 14, size 2942


Nice thing about using logcat is that it gives you readable timestamps, so you know exactly when they were killed.  Since I'm a bit OC, I have a script that runs both:
bash-3.2# cat lowmemkilled.sh
# tells me which recent processes were killed by lowmemkiller

echo --- using dmesg ---
dmesg | grep sigkil
echo --- using logcat ---
logcat -t 1000 | grep "has died"

Friday, January 28, 2011

Bash!

Freedom!

I've been running telnetd on android for a while now, and the busybox shell sucks for obvious reasons - it's not meant to be a fully featured shell.  Luckily, there's bash.  I've been using bash on linux for years and I was stoked to find it for android. 

You can download it here.  Or use Fission ROM manager to download it.

Once you have the file downloaded, copy it to /system/bin, and make sure that permission is set properly.  And instead of using /system/bin/sh for telnetd you can use /system/bin/bash.  So your full telnetd command would look something like this:
su -c 'telnetd -l /system/bin/bash -F'
The above command runs as a sl4a script on my Droid X - super convenient this way.

To exprience the awesomeness of bash, try typing a partial command or directory and hit Tab, try hitting up and down arrow, and Ctrl-A & Ctrl-R.  Oh, and adb sucks. ;)

Thursday, January 27, 2011

OOM and Android Part Deux

(This is a follow-up to my previous post)

Success! I was able to induce kernel's own oom-killer.
# echo "0,0,0,0,0,0" > /sys/module/lowmemorykiller/parameters/minfree

# vi /sdcard/largefile
.. a few sec later (in dmesg):
<3>[ 277.294128] Out of memory (oom_kill_allocating_task): kill process 2435 (vi) score 0 or a child
<3>[ 277.294250] Killed process 2435 (vi)
So really all you gotta do is "disable" android's built-in memory killer (which i'd imagine uses watchdog to do the actual work). What does this mean? Not a whole lot more than what I already said before. As long as you stick to the lowmemorykiller settings provided by your custom ROM it's not likely that the kernel's oom-killer will be invoked, and the vm.oom* sysctl settings will be ever useful. Android lowmemorykiller > linux kernel's oom-killer.

Some juicy information here.

And again, if you'd like to check your custom ROM's minfree, check somewhere in /etc/init.d to verify that the values are set and what they're being set to.  On Rubix:
# cat /etc/init.d/02vm | grep minfree
echo "2048,3072,6144,15360,17920,20480" > /sys/module/lowmemorykiller/parameters/minfree

Wednesday, January 26, 2011

OOM and Android

Some of you were wondering what the last two lines in my sysctl.conf file are.  They control the behavior of the kernel when it detects a "OOM" event.

# reboot when OOM happens
vm.panic_on_oom = 2
# wait 5 sec before rebooting when OOM
kernel.panic = 5
The reason I wanted my phone to just go ahead reboot is mostly because I had some bad experiences with the OOM Killer in the past on other linux systems.  On my phone, I want my phone to just kernel panic and reboot instead of me having to deal with the potentially crippled phone after the OOM killer is finished with the running OS processes and applications.

So I decided to put my sysctl oom settings to the test.

First, I needed to force the phone to run out of memory completely.  Step one, create a 600MB file:
# dd if=/dev/zero of=/mnt/sdcard/largefile bs=1 count=600000000
Step two, clear dmesg buffer:
# dmesg -c
Step three, create a process that attempts to use 600MB of memory:
# vi /sdcard/largefile
Step four, observe.

Step five, analyze.

So based on my test, it turns out Android kernel's OOM Killer is *not* invoked.  There's a watchdog process that detects that the phone is about to run out of memory and kills my vi process along with a bunch of other processes.  Evidence in dmesg (it's just a snippet):
<4>[79814.464782] select 11855 (.android.tasker), adj 2, size 4694, to kill
<4>[79814.465087] select 11863 (d.process.acore), adj 4, size 7712, to kill
<4>[79814.465667] send sigkill to 11863 (d.process.acore), adj 4, size 7712
<4>[79814.559112] select 11855 (.android.tasker), adj 2, size 4694, to kill
<4>[79814.559204] select 11871 (e.process.gapps), adj 2, size 5449, to kill
<4>[79814.559356] send sigkill to 11871 (e.process.gapps), adj 2, size 5449
And what is process id 4?
root      4     2     0      0     c009a4e8 00000000 S watchdog/0
How many processes did watchdog kill?
# grep -c "to kill" /sdcard/dmesg.log
78
Yikes.  Anyway, I ran the experiment several times (oom killer on, off, etc.) all producing the same results, and still could not invoke kernel's OOM killer or cause the kernel to panic.  I'm guessing Android developers decided that the watchdog managing oom situation is superior than leaving it to the kernel.  Probably a good thing too - OOM Killer sucks.

So what does this mean?  This means all the oom related settings in sysctl are probably useless.  (It also means that for all intents and purposes the watchdog process is the new oom killer, but outside of the kernel). Because android kernel is derived from a standard linux kernel, there are going to be a number of kernel parameters that are not used, and the oom ones are possibly part of that.  Therefore, my latest sysctl.conf file reflects my latest experiement:
# cat /etc/sysctl.conf

# try to keep at least 4MB in memory
vm.min_free_kbytes = 4096

# favor block cache
vm.dirty_ratio = 90
vm.dirty_background_ratio = 55

# extremely favor file cache
vm.vfs_cache_pressure = 1
# ignore below for now - not used by android
vm.swappiness = 0

vm.panic_on_oom = 2
kernel.panic = 5
 As far as the watchdog process is concerned, I looked around a bit.  Didn't find /etc/watchdog.conf, didn't see an obvious place where it starts up, etc.  Must be part of the android base os somehow.

Tuesday, January 25, 2011

Enabling sysctl

First of all, I'd like to thank all the ROM developers for making my phone so much fun to play with.  Without them it will be like having an iphone. :)

Secondly, I strongly suggest enabling telnetd.  It makes typing (and copy/pasting) so much easier, and you can also mess with the phone while your son is playing Angry Birds on it!

Enabling sysctl on custom ROMs is mostly pretty easy.  Roughly the steps are:
  1. Verify you can run custom scripts at boot.
  2. Check to see if sysctl is already enabled.
  3. Create/modify /etc/sysctl.conf
  4. Verify that the new values stick at reboot.
BTW, if your ROM already allows you to enable/modify sysctl via settings, you can skip 1-2.
    Let's start with #1.  First open up Terminal and/or telnet into your phone.  Then verify that you got an init.d directory on the phone (I'm currently running Rubix):
    # ls -l /etc/init.d
    -r-xr-xr-x root     root          759 2008-08-01 05:00 01cpu
    -r-xr-xr-x root     root          230 2008-08-01 05:00 02vm
    -r-xr-xr-x root     root          228 2008-08-01 05:00 03pfm
    -r-xr-xr-x root     root          100 2008-08-01 05:00 04screen
    -r-xr-xr-x root     root           72 2008-08-01 05:00 05zipalign
    -r-xr-xr-x root     root          344 2008-08-01 05:00 98firstboot
    -r-xr-xr-x root     root          233 2008-08-01 05:00 99complete
    Oh look, it looks like I can run some custom scripts at boot time.  If /etc/init.d doesn't exist, stop.  This gets messy and honestly I wouldn't recommend going further because you could potentially bootloop your phone if you try to modify anything other than what's in this post.

    #2. At first glance I don't see an init script named sysctl like the Liberty Rom, but it's possible that one of the scripts takes care of it:
    # grep sysctl /etc/init.d/*
    /etc/init.d/02vm:# Load /sys/etc/sysctl.conf
    /etc/init.d/02vm:sysctl -p
     Yup, 02vm file enables sysctl.  "sysctl -p" takes what's already in /etc/sysctl.conf and applies it to the kernel.  If you don't see any init script running sysctl -p, then you'll want to create one.  Call it something like 99sysctl, and then simply add "sysctl -p" in the file.

    #3.  Modify /etc/sysctl.conf.  Here's what my latest looks like:
    # cat /etc/sysctl.conf
    # shouldn't matter - no swap is used
    vm.swappiness = 0

    # try to keep at least 4MB in memory
    vm.min_free_kbytes = 4096

    # favor block cache
    vm.dirty_ratio = 90
    vm.dirty_background_ratio = 55

    # extremely favor file cache
    vm.vfs_cache_pressure = 1

    # reboot when OOM happens
    vm.panic_on_oom = 2

    # wait 5 sec before rebooting when OOM
    kernel.panic = 5
     #4. Verify that the sysctl init script is working.  Reboot the phone and run this command:
    # sysctl -a | grep vm  
     "sysctl -a" will display all the kernel settings, and what you're interested in is the section that you modified, hence "grep vm".  The output should look something like this:
    # sysctl -a | grep vm     
    vm.overcommit_memory = 1
    vm.panic_on_oom = 2
    vm.oom_kill_allocating_task = 0
    vm.oom_dump_tasks = 0
    vm.overcommit_ratio = 50
    vm.page-cluster = 3
    vm.dirty_background_ratio = 55
    vm.dirty_background_bytes = 0
    vm.dirty_ratio = 90
    vm.dirty_bytes = 0
    vm.dirty_writeback_centisecs = 500
    vm.dirty_expire_centisecs = 200
    vm.nr_pdflush_threads = 0
    vm.swappiness = 0
    vm.lowmem_reserve_ratio = 32
    vm.drop_caches = 0
    vm.min_free_kbytes = 4096
    vm.min_free_order_shift = 4
    vm.percpu_pagelist_fraction = 0
    vm.max_map_count = 65530
    vm.laptop_mode = 0
    vm.block_dump = 0
    vm.vfs_cache_pressure = 1
    vm.mmap_min_addr = 4096
    vm.scan_unevictable_pages = 0
     There you can see that the values in your customized /etc/sysctl.conf have taken effect.  Great success!

    top: part 2


    Sometime I wonder what's eating up the memory on the phone.  Ok, I lied - I wonder all the time. Well, there's an easy, reliable and accurate way to get your answer using what else, top!

    Forget all those apps you see in the market, uninstall your task killers.  Top is where it's at!

    Let's take a quick look.  The command I used below will show you the top 10 memory hogs that are currently running on my Droid X.

    "-m" tells top to show the top 10 entries, "-n" tells it to run it 5 times and quit, and "-s rss" tells it to sort it by memory usage.

    # top -m 10 -n 5 -s rss

    User 5%, System 5%, IOW 1%, IRQ 0%
    User 16 + Nice 0 + Sys 18 + Idle 268 + IOW 6 + IRQ 0 + SIRQ 0 = 308

      PID CPU% S  #THR     VSS     RSS PCY UID      Name
    18230   2% S    70 355932K  69068K  fg system   system_server
    26822   0% S    21 203692K  35368K  bg app_32   com.alphonso.pulse
    23414   0% S    31 214036K  33912K  bg app_63   com.google.android.apps.reader
    30083   0% S     8 178724K  28324K  bg app_86   android.process.acore
    19879   0% S    13 177228K  26544K  bg app_76   com.thedeck.android.app
    24994   0% S    14 176612K  25364K  bg app_100  com.quoord.tapatalkpro.activity
    30419   0% S    15 175128K  23964K  fg app_110  com.googlecode.android_scripting
    30356   0% S    11 166524K  23876K  fg app_113  com.genusproject.apps.eztvlisting
    18311   0% S    12 186164K  22976K  fg app_33   net.thinkindifferent.inputmethod.latin


    You can ignore system_server, but you see what floats up on top? 

    /me uninstalls Pulse.  If an app is going to take up more than 30MB RSS and 200MB VSS, it better be a killer app like the Google Reader.

    Sunday, January 23, 2011

    top

    Top is probably the most popular built-in tool on linux, and has been around for ages.  And busybox gives you top!

    Have you ever noticed that sometimes your android device feels really sluggish and your cpu graph app (I use CpuNotify) is showing that your CPU is pegged?  I believe the most effective way to determine what's causing the phone to slow down is top.  Don't install apps that do similar stuff from the market - top will be more accurate and reliable.

    Let's look at an example.  The other day my DX was just running dog slow and CpuNotify was telling me that the CPU was maxed out.  So I launch Terminal, run top, and scroll up:
    # top -m 4 -n 3 -s cpu
    User 83%, System 2%, IOW 4%, IRQ 0%
    User 217 + Nice 0 + Sys 7 + Idle 75 + IOW 13 + IRQ 0 + SIRQ 0 = 312

      PID CPU% S  #THR     VSS     RSS PCY UID      Name
    27769  80% S    21 236568K  69156K  bg app_74   com.dolphin.browser
    18230   1% S    71 359916K  61852K  fg system   system_server
    27888   1% R     1    848K    420K  fg root     top
    18311   0% S    12 186360K  22416K  fg app_33   net.thinkindifferent.inputmethod.latin
    [ snip ]
    WTF!?!  What's my browser doing?  It turned out I made a classic mistake of going to a javascript intensive website and forgot to close the window.  Nine out of 10 times when your phone seems really sluggish, it's probably a runaway javascript that's still running on a browser window in the background.

    So what do you do then?  Well you can bring the browser to the foreground again and close the offending tab/window.  Or you can just kill it from shell:
    pkill -f browser

    Friday, January 21, 2011

    let's add swap!

    Android, at least on my Droid X, doesn't use swap.  And I'm sure there's a good reason for that - one of them being that it doesn't need one.  As long as it has plenty of memory and killer memory management why bother?

    Still, I thought it would be fun to add since Android runs a linux kernel and linux likes swap.

    I think you can do some interesting things. If you have swap, you can perhaps encourage the kernel to swap out some rarely used processes like Xfinity, etc. (or at least some portions of it. Isn't it annoying how Xfinity always sticks around). Then you can tell the kernel to use more of the RAM for cache, thus making the phone faster. I know android already does a good job in managing memory and killing off less used processes but I think with swap we'd have a lot more flexibility.

    Well, let's give it a shot, shall we?
    # dd if=/dev/zero of=/mnt/sdcard/swap bs=1k count=100000
    100000+0 records in
    100000+0 records out
    100000 bytes transferred in 0.713 secs (140252 bytes/sec)
    Promising...
    # mkswap /mnt/sdcard/swap
    Setting up swapspace version 1, size = 95904 bytes
    Whoa! Looking good...
    # swapon /mnt/sdcard/swap
    swapon: /mnt/sdcard/swap: Function not implemented

    Thursday, January 20, 2011

    Sysctl tweaking for faster, longer lasting Android

    If your ROM supports sysctl, you can do lots of cool things.  (Well even if your ROM doesn't you can as long as you're rooted with busybox, but it takes a bit more work).

    In this post, let's focus on three settings: vm.dirty_ratio, vm.dirty_backgroud_ratio, and vm.vfs_cache_pressure.

    vm.dirty_ratio and vm.dirty_background_ratio control how often kernel writes data to "disk" (well in our case the microSD card).  When your apps write data to disk, Linux actually doesn't write the data out to disk right away, it actually writes the stuff to system memory and the kernel handles when/how the data is actually going to be flushed to disk.

    I dunno - how much difference can it actually make?  We're not talking about traditional hard drives with spindles, we're talking about solid state stuff.  I suppose there could be a slight savings in battery life and increase in performance if you delay the data flush as much as you can.

    I ended up with:
    vm.dirty_ratio = 90
    vm.dirty_background_ratio = 70
    Now vm.vfs_cache_pressure is much more interesting. File system cache (dentry/inode) is really more important than the block cache above, so we really want the kernel to use up much of the RAM for them.  The default value is 100, and what you want to do is lower that value to tell the kernel to favor the file system cache and not drop them aggressively.

    You can also take it a step further and set it to 1 (lowest possible without being dangerous).  With the value of 1 the kernel will drop cache only when it's completely out of memory.
    vm.vfs_cache_pressure = 1
    The problem is if you lower the value too much, after a day or two your device may start getting sluggish because the amount of RAM available to the applications continue to shrink to the point where they are starved for memory.  So I came up with a solution to manually drop caches nightly!
    echo 3 > /proc/sys/vm/drop_caches
    # tells kernel to drop all file system caches
    And what I do to ensure that the caches are dropped nightly is to run the following sl4a scripts via tasker.  Cron would be better but busybox on android doesn't seem to support it yet.

    (It also reverts the dirty settings because I don't care if my phone writes to disk aggresively while i'm sleeping and it's being charged.)
    su -c 'echo 3 > /proc/sys/vm/drop_caches;\
    sysctl -w vm.dirty_background_ratio=3;\
    sysctl -w vm.dirty_ratio=15'
    And the following script at wakeup time:
    su -c 'echo 3 > /proc/sys/vm/drop_caches;\
    sysctl -w vm.dirty_background_ratio=70;\
    sysctl -w vm.dirty_ratio=90'
    Finally, here's what my sysctl.conf file looks like:
    # cat /etc/sysctl.conf
    # shouldn't matter - no swap is used
    vm.swappiness = 0

    # try to keep at least 4MB in memory
    vm.min_free_kbytes = 4096

    # favor block cache
    vm.dirty_ratio = 90
    vm.dirty_background_ratio = 70

    # extremely favor file cache
    vm.vfs_cache_pressure = 1

    # reboot when OOM happens
    vm.panic_on_oom = 2

    # wait 5 sec before rebooting when OOM
    kernel.panic = 5



    # currently experimenting
    kernel.shmmax = 268435456
    kernel.shmall = 16777216
    I've been running at the above settings for a few days now, and the phone has been faster than ever with zero ill effects.  Your mileage may vary of course.

    UPDATE: Read my follow-up post!

    Wednesday, January 19, 2011

    dmesg?!?

    Let's see what else you can do with busybox.  Oh hey how about dmesg?  That's like one of my favorite linux commands!  Dmesg on android must show all kinds of cool interesting things that the kernel is doing!

    # dmesg | tail
    <6>[22506.486755] TIWLAN: 2421.209967: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
    <6>[22507.057128] TIWLAN: 2421.780340: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
    <6>[22509.056701] TIWLAN: 2423.779944: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
    <6>[22540.139587] TIWLAN: 2454.862768: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
    <6>[22543.117156] usb_ether_get_stats
    <6>[22543.136444] usb_ether_get_stats
    <6>[22544.635040] TIWLAN: 2459.358252: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
    <6>[22547.410217] TIWLAN: 2462.133398: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
    <6>[22548.908538] TIWLAN: 2463.631720: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
    <6>[22549.147735] TIWLAN: 2463.870947: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
    That was stupid.

    Enabling remote telnet access to your phone

    I was messing with busybox on my Droid X the other day using the tiny android gingebread keyboard, and started wondering "dang, i'm making too many typos and this is really a pain".  So I decided to enable telnetd so I can telnet in from my macbook and use my gigantic keyboard.   Here's how you do it:

    1. get on a secure wifi network (well you don't have to but you'd want to - trust me)
    2. run Terminal from your phone
    3. type "telnetd -l /system/bin/sh"
    4. obtain your phone's IP address from android settings
    5. run terminal software from your laptop/desktop
    6. telnet to the ip address
    7. when you're done, kill telnetd: "pkill telnetd"
    Voila!  Now you can hack away without having to use the phone keyboard, or connect it to your PC via USB.


     [insert extremely satisfied happy face here]

    UPDATE: Be sure to read my followup post here!