Wednesday, October 5, 2011

CPU Governors

I've been tinkering around with cpu governors quite a bit in my Android kernels.  I believe governors are a single most effective way to improve or diminish the "user perceivable" performance of the device.

Here's my personal take on governors that are included in stock and other custom kernels:

performance - max speed all the time
powersave - lowest speed all the time
conservative - slow ramp-up

The next few get a bit more complex. ondemand, interactive, and smartass all try to pretty much do the same thing: perform well and power efficient. But the way they approach their goals are very different (ie. their algorithms are very different).

1) ondemand - been in linux for a long time and we got really smart linux kernel developers working on it and the code gets reviewed by really smart people. At the same time this governor is really designed to work on desktops, servers, phones, etc - a universal solution.
2) interactive - developed by CM (I think), tuned for performance. Instead of sampling at every interval like ondemand, it determines how to scale up when cpu comes out of idle.
3) smartass - developed by erasmux for his android kernel. Popular for its ability to use android's onboard suspend mechanism to keep the phone below a certain clock speed when screen is turned off. Also does slow ramp-up like the conservative governor.

Instead of the above, here are the three governors that I include in my custom kernels:

  • interactiveX - it's the interactive governor from CM, but I added suspend/wake logic so when the phone screen is off it runs at below 400Mhz. Also, I modified its code some more to minimize unnecessary cpu spikes above a certain threshold if kernel is heavily overclocked. (Most devices are unstable above a certain speed when it's heavily overclocked, and it's the quick jump to top speed that usually locks up the phone.)
    I like this governor because it's simple and fast.
  • ondemandX - ondemand governor code from latest linux (3.0 at the moment) source *plus* the suspend/wake logic described above. No further optimization is done.
  • smartass - smartass code from erasmux, but I wasn't happy with its performance so I tuned it for quicker ramp up in speed. It has the same suspend/wake logic as ondemandX, and similar stability optimizations as interactiveX. (update: smartassV2 is new and currently I include the v2 in my kernels without optimizing further).

Tuesday, September 27, 2011


Building kernels has been my thing lately.  Been developing for:
  • Thunderbolt
  • Droid Charge
  • Samsung Fascinate/Mesmerize/Showcase
  • Samsung Galaxy Tab 7"
You can find my threads on

Friday, May 20, 2011

DX/D2 tweaks new version (man it's been a while)

Sorry guys, I've been pretty busy developing kernels for thunderbolt in the recent weeks in my spare time. Which i don't have a lot of due to a full time job (in case someone from work is reading this). ;)

Here's an updated zip for DX/D2 users.  If you're already on thunderbolt, you shouldn't have to flash this zip (although it won't fail) - you should be flashing my kernels instead!  If you're on someone else's kernel and flashing this zip to get my tweaks shame on you! :-)

v7.1 (download)

To verify that all the tweaks are in place:
  1. Open Terminal Emulator
  2. su
  1. Latest busybox compiled by yours truly.
  2. Modified enable scripts to work on GB ROMs.
  3. Easier to run check script (see above).
  4. SD card speed boost.
NOTE: the latest version has been tested on Gingerbread only.  Should work fine on Froyo but hasn't been tested. Also, if you're running ZombieStomped there's no need to flash.

NOTE to D1 users: Been working with Liquid to develop kernels for D1 as well.  Still work in progress.

Sunday, April 10, 2011

Not so universal for original Droid

I was recently approached by Jason at Droidforums to see if I can make a zip for my tweaks for Droid 1.  Working with him closely (since I don't have a D1) I was able to come up with a slightly crippled version of my mods for Droid 1 running Project Elite ROM (but it should work for any ROMs that have init.d support).


To check to see if the mods are working: open Terminal Emulator and type "bash /system/etc/imoseyon/" without quotes.

I'll try my best to further enhance the mods but I'll need some help from other D1 users. :)

Tuesday, April 5, 2011

minimal kernel for tbolt

Been messing with compiling kernels for my thunderbolt.  You can read about it here:

More details on this later.  Combing through the kernel code for more tweaking opportunities. ;)

Friday, April 1, 2011

Why swap?

Why not?  ;)  If you think with 768MB of RAM on the thunderbolt you'd have way more memory than you'd ever need, then think again.  A lot of the memory is taken up by cache and buffers, which are also used pretty heavily with my tweaks.

For those of you that are new to Linux (or other UNIX variants), you'd probably heard the term "virtual memory".  Swap is pretty much that - a way to extend your physical memory.  Your OS moves the less accessed memory pages to slower, and cheaper storage medium.  This frees up RAM for things that require the speed.  And guess what? On almost all android devices you'll find tons of flash memory which are going to provide near RAM access speed for your swap space.

But setting up swap alone isn't going to do much.  In fact, it could do more harm than good.  You'd also want to adjust some OS/kernel related settings so your phone/device can use swap effectively.

You want to primarily look at two settings: vm.swappiness and minfree.

vm.swappiness is a no brainer.  The lower the number less swap the OS is going to use, the higher the more swap.  If you set it to 0 it's not going to use swap at all, 100 it's going to use a lot of swap.  If you set it to say 1 it would use swap only when absolutely necessary.  I recommend 40 on a 200MB swap file.

minfree is another tricky one.  To really get the full benefit of swap, I think you want minfree to "work less", ie. kill active and semi-active apps less frequently.  This will allow all of your frequently used apps to stay in RAM.  Something like this would work well:

echo "100,200,20000,20000,20000,25000" > /sys/module/lowmemorykiller/parameters/minfree

Finally, you'd need a kernel that supports swap.  Unfortunately if your bootloader is locked (ie. DX/D2) you're not going to get custom kernels that will support swap.  I've tested swap on some custom kernels on the Tbolt and the Gtab and it works well.  If you're running a kernel that supports swap, download and flash my zip to create and enable swap; the zip will also adjust swappiness and minfree accordingly if swap is enabled.

One final note on this is some technical stuff on why I chose to use internal flash for swap rather than the sdcard - internal flash performs better:
sh-3.2# hdparm -Tt /dev/block/mmcblk0p26

Timing buffer-cache reads: hdparm: HDIO_DRIVE_CMD: Inappropriate ioctl for device
  326 MB in 0.51 seconds = 653840 kB/s
Timing buffered disk reads:   52 MB in 3.01 seconds = 17683 kB/s
hdparm: HDIO_DRIVE_CMD: Inappropriate ioctl for device
sh-3.2# hdparm -Tt /dev/block/vold/179:33

Timing buffer-cache reads: hdparm: HDIO_DRIVE_CMD: Inappropriate ioctl for device
  218 MB in 0.51 seconds = 437047 kB/s
Timing buffered disk reads:   15 MB in 3.03 seconds = 5054 kB/s
hdparm: HDIO_DRIVE_CMD: Inappropriate ioctl for device

sh-3.2# time busybox dd if=/dev/zero of=/mnt/sdcard/test bs=1k count=100000
100000+0 records in
100000+0 records out

real    0m17.159s
user    0m0.070s
sys    0m2.710s
sh-3.2# time busybox dd if=/dev/zero of=/data/test bs=1k count=100000     
100000+0 records in
100000+0 records out

real    0m12.202s
user    0m0.100s
sys    0m2.960s

Friday, March 25, 2011


Got a thunderbolt and figured why not make the zip even more universal.  So, v6 should work for TB, DX, and D2 now. After flashing the zip be sure to reboot one more time!

What to expect - hopefully faster, more stable, and longer lasting TB/DX/D2.
How to check if the mods are in effect: 1) Launch Terminal Emulator, 2) Type "bash /data/imoseyon/", without the quotes and hit enter.
How to remove my mods: remove /etc/init.d/99imoseyon and reboot.

(Keep in mind that this is a work in progress and your mileage WILL vary.)

More details

Change Logs 
  • v6.2.1 - fixed a bug in check script.
  • v6.2 - added swap support (if your kernel supports swap, ie thunderbolt). Run "sh /data/imoseyon/ create" to enable. Blog post on this.
  • v6.1.4 - made enable script more robust for more ROMs. No need to flash if yours is already working.
  • v6.1.3 - fixed more enable bugs for some roms.  It's not easy making a zip for all ROMs for 3 devices folks! ;)
  • v6.1 - added /tmp tmpfs support for thunderbolt
  • v6.0.5 - fixed more minor bugs, and tweaked cache_pressure slightly in an attempt to reduce/eliminate occasional sluggishness.
  • v6.0.4 - fixed a bug for stock ROMs without busybox, created /tmp if it doesn't exist.
  • v6.0.3 - fixed an unsilly bug for DX/DX
  • v6.0.2 - fixed another silly bug which prevented crond from starting up on some ROMs.
  • v6.0.1 - fixed a silly bug which prevented my mods from being enabled on ROMs that don't have init.d enabled.
  •  v6.0
    • Tweaked scripts to work on thunderbolt.
    • Tweaked minfree some more (note this is still a work in progress).
    • Separated sysctl settings to a separate file (/data/imoseyon/sysctl.conf) so they are easily customizable.
    • Abandoned interactive mode - it didn't really provide much value for me.  So I'm doing ondemand only for now.  May look at conservative next.

Thursday, March 3, 2011

Universal Zip (imoseyon mods)

v5.2 is up

I finally had a chance tonight to create one zip to rule them all.  No more maintaining four different versions.  The new zip contains a script that runs when you flash it the first time to determine what needs to be enabled and makes the necessary changes regardless of which ROM you're on.

More details on the mods

What to expect - hopefully faster, more stable, and longer lasting DX/D2.
How to check if the mods are in effect: 1) Launch Terminal Emulator, 2) Type "sh /data/imoseyon/", without the quotes and hit enter.

(Keep in mind that this is a work in progress and your mileage WILL vary.)

Change Logs
  • v5.2
    • tweaked minfree to be less aggressive for foreground app and background services
    • does a nightly check to see if it's enabled at boot and fixes it if the boot startup is clobbered by another app.
    • new script - run "sh /data/imoseyon/" anytime to see if all the tweaks are in effect.
  • v5.11: fixed a bug where if you didn't already have busybox installed my tweaks probably didn't stick at boot (for 3.4.2 users).
  • v5.1: updated to work on 3.4.2. (only 3.4.2 users need to flash)
  • v5.0: universal version for all ROMs.  Created two separate zips for governor types. (upgrade from 4.3 is only necessary if you want to go back to ondemand governor).
You can find the old change logs here.

Here are the differences between the two governors (taken from the Cyanogen docs):
The CPUfreq governor "interactive" is designed for low latency,
interactive workloads. This governor sets the CPU speed depending on
usage, similar to "ondemand" and "conservative" governors. However
there is no polling, or 'sample_rate' required to scale the CPU up.

Sampling CPU load every X ms can lead to under powering the CPU
for X ms, leading to dropped framerate, stuttering UI etc..

Scaling the CPU up is done when coming out of idle, and like "ondemand"
scaling up will always go to MAX, then step down based off of cpu load.
What's my verdict?  It's hard to say.  Interactive governor does feel a bit snappier but theoretically it should consume a bit more battery.  And I had a random reboot for the first time in months, after enabling interactive governor.... I'll post my opinion after using it for a week or so.

Wednesday, February 23, 2011

Experimental Mods

UPDATE: v4.2e is now deprecated - please download the latest instead.

I've been trying out some crazy minfree settings and I have a set that seems to work fantastic for me, so here they are:
For those who are comfortable modifying init files directly, you just need to change minfree values in 99imoseyon to 15000,20000,20000,20000,20000,25000, and reboot.  For others, just flash the zip over the previous version.

Give it a try for a day or two and let me know what you guys think.  Comment here on any changes you notice in speed, battery life, FCs, etc.

Monday, February 21, 2011

Soft Reboot (quick reboot)

I talked a bit about dropping caches to free up memory and jumpstart the kernel caching system a few times in my previous posts.  I think occasionally some of us would like to clean things up even more to keep the phone running smoother (ie. maybe you feel that the phone is running really sluggish).  One way to possibly accomplish this is to do a soft reboot.

Soft reboot would essentially kill all running processes and restart the essential services.  Anything at the OS layer or below, namely bootloader, kernel and init would not be restarted.  So you're rebooting your phone without touching the OS/bootloader, and all within just a matter of seconds.

Here's what you do to cause a soft reboot:
pkill zygote
That's it.  Zygote is an android service that initializes Dalvik and rest of the system.  By killing it, you basically kill all apps registered with zygote and restarts essential services including zygote.  (At least that's how I understand it haha).

If you've been reading my blog, you probably know all about crond on android.  You could potentially use crond to schedule a soft reboot once a week.  Your crontab file may look something like this:

# drop kernel caches at 3AM every morning
0 3 * * * sync; echo 3 > /proc/sys/vm/drop_caches
# soft reboot system at 2AM monday morning
0 2 * * 1 pkill zygote

There's one slight problem with this method. This soft reboot is more or less based on my theory and I have no idea what the long term effect would be if you continue to use this method.  Use it at your own risk! :)

Friday, February 11, 2011

I Got a ZIP (imoseyon mods)

- v5.0 universal zip (this post is now deprecated - please go to the lastest post)

Due to popular requests, I decided to make a zip with most of my tweaks.  Applying the zip below will:
  1. Installs busybox 1.18.0 (in an alternate location).
  2. Enables Cron.
  3. Tweaks kernel stuff (vm, minfree, etc.).
  4. Tweaks build.prop (on some ROMs).
  5. Flushes system caches once a day (3AM PT).
  6. Disables logger
  7. Enables new governer.
  8. and more (in the change logs below).
This zip should be all you need to get your phone running in tip top shape.  More specifically, it will hopefully make your phone run faster, longer, and more stable.  I think a good majority of you will notice the difference, and some will not.  I have different versions of the zip for various custom ROMS (rubix, liberty, darkslide, etc. and maybe even stock ROM) - I hope to make a single unified zip that works for all ROMS soon.   It should also work for both Droid X and Droid 2.

Here's how you apply the zip (this is standard stuff):
  1. Back up your ROM! (Some of the stuff in the zip are experimental, and I don't test every release for every ROM, so make sure you have a nandroid backup to go back to!)
  2. Boot into Bootstrapper/clockworkmod.
  3. Install zip.
  4. Reboot.
To verify that the zip was applied successfully, you can run this command from Terminal Emulator:
pgrep -f crond
If you get a number, then you're all good. (The number just means that the process is running in the background, nothing more.)
    Warning: Flashing this zip will override some of the kernel tweaks you may have made with built-in tool provided by your ROM (ie. liberty toolbox).
    • v3.1: Fixed busybox/crond issue. Separated DX and D2 files - duh. 
    • v3.2: Fixed wget. 
    • v3.21 (D2 ONLY): Fixed crond start bug. 
    • v3.3 (DX/D2), V2.0 (Apex): busybox 1.18.0, made minfree more aggressive for secondary apps (may reduce occasional sluggishness), prettier "ls", got rid of scheduler tweaks which did nothing.
    • v4.0:
      • Timezone semi-auto detection for cron (US only).
      • Additional minfree tweaks to make memory management a bit more aggressive to prevent some reboots and freezes.
      • Added bash.
      • Added additional cron job examples (backup files including sms datase, killapps - all disabled  by default).
      • Liberty version has call delay and proximity delay slightly tweaked per Jreguit. 
      • Purge logs - view /tmp/imosey to see what kind of effects nightly purging has.
    • v4.1:
      • Changed the purge time from 1AM to 3AM.
      • Fixed a bug in timezone detection (doh!)
      • Disabled logcat at boot for extra performance, battery, etc - hopefully. (comment in 99imoseyon to disable)
      • Added zipalign for system apps at boot. (comment in 99imoseyon to disable)
      • Separated crond startup to a separate script for future proofing.
      • Added another cron job example in /data/cron/root for soft reboot, and added explanation for each cron job.
      • TODO: test the others version for APEX.
    • v4.2:
      • Disabled zipalign at boot - should fix the no service problem.  Sorry about that! 
    • v4.3:
      • Fixed a major bug in the v4.2 of "others", where if you were installing the zip for the first time the mods weren't being applied at all! (sorry)
      • Incorporated the minfree tweak from my experimental version.
      • Interactive governer for speed! (thx to drod via jrummy via milestone, etc.)

      Wednesday, February 9, 2011

      sysctl (and minfree) tweaks revisited

      (This is a follow up to my first post).

      First of all, I believe tweaking minfree is just as important, if not more, than sysctl.  Android tends to behave badly when there's very little free memory.  When you're running your device with say less than 20-30MB of free RAM, a misbehaving app or two could invoke kernel's OOM killer and cause the device to become unstable, then eventually reboot or freeze.

      For that reason, I don't like the default value of 25MB minfree.  (minfree, btw, is android's memory management system).  From my experience, I think at least 80MB minfree is required for your device to run fast and stable.  In general I recommend Rubix's minfree setting (20480 blocks x 4 will give you 80MB): 

      echo "2048,3072,6144,15360,17920,20480"  > /sys/module/lowmemorykiller/parameters/minfree

      I personally run my Droid X with the following: 

      echo "256,512,1024,5120,13000,20000" > /sys/module/lowmemorykiller/parameters/minfree

      Both should be fine.  Be sure that the last two numbers are at least 13000 and 20000.

      Now back to sysctl.  Personally, I haven't had to manually drop caches at all.  In fact, my phone will probably run fine indefinitely without having to purge (longest i've run it without rebooting was 5 days, but it's really hard not to reboot your phone when you tweak it as much as i do - the kernel will automatically drop caches when it think it needs more memory). But I'm not a heavy user (i don't play games),  your mileage will vary of course, and you may need to purge every once in while.   To purge page, dentry and inode caches, use this command:
      sync; echo 3 > /proc/sys/vm/drop_caches
      My personal recommendation for most people (no manual flushing should be required):
      vm.dirty_ratio = 90
      vm.dirty_background_ratio = 55
      vm.vfs_cache_pressure = 20
      If you are willing to play around with manual or timed flushing I think you can squeeze a little more performance and battery life with:
      vm.dirty_ratio = 90
      vm.dirty_background_ratio = 70
      vm.vfs_cache_pressure = 1
      Btw, you can schedule cache purge using Tasker/sl4a, or you can use cron.

      Tuesday, February 8, 2011

      cron on android is awesome

      Cron is a popular time based job scheduler on Unix/Linux, and is super useful as you can virtually run any command or script periodically at certain times or dates.

      Even though Android is based on linux, it does not run crond (cron daemon) by default, well at least not on my Droid X.  Fortunately, you can run crond with busybox with a few hacks.  It's not the prettiest hack but it works.  Here's how you do it:
      1. Make sure that your busybox supports crond.  Try typing "crond" in Terminal and see what it says.  I run busybox v1.17.1.
      2. Run your favorite command line tool telnet, Terminal, adb, etc. to run the commands below.
      3. Remount / and /system rw.
      4. Create a /etc/passwd. (crond calls getpwnam to search user db)
        echo "root:x:0:0::/data/cron:/system/bin/bash" > /etc/passwd
      5. Create a symlink for /system/bin in /bin.
        ln -s /system/bin/ /bin
      6. Create your crontab file, and call it "root" (I created /data/cron/root).  Check the example section in this article to see what a crontab file should look like:
      7. Set your timezone.  I live in California so mine looks like this (apparently the new Olson style timezones do not work):
        export TZ
      8. Run crond:
        crond -c /data/cron
      9. To verify that crond is running succesfully run "pgrep crond".  If you get a number (PID) then you know it's running.
      Obviously you don't want to do the above steps every time your phone boots, so you want to create an init.d script with the above commands to run at boot, or add the commands to the end of an existing init.d script.  I've added them at the end of my init.d script called /etc/init.d/99imoseyon:
      # enable crond
      # crond calls getpwnam (user database search)
      mount -o remount,rw -t yaffs2 `grep /system /proc/mounts | cut -d' ' -f1` /system
      echo "root:x:0:0::/data/cron:/system/bin/bash" > /etc/passwd
      mount -o remount,ro -t yaffs2 `grep /system /proc/mounts | cut -d' ' -f1` /system
      # crond has "/bin/sh" hardcoded
      mount -o remount,rw rootfs /
      ln -s /system/bin/ /bin
      mount -o remount,ro rootfs /

      # set timezone
      export TZ
      # use /data/cron, call the crontab file "root"
      crond -c /data/cron
      So what can we use it for?  Lots of things.  I'm going to start using it to periodically drop page/filesystem caches, back up SMS database nightly, kill memory resident apps hourly, etc.  There are literally hundreds of use cases for android phones. Here's a sample crontab file (backups my init.d script at 8pm every night) at /data/cron/root:

      0 20 * * * cp /system/etc/init.d/99imoseyon /sdcard/data/init.d
      UPDATE: Added timezone step to the instructions.  Also added a command to verify that crond is running.

      Saturday, February 5, 2011

      Differences between rubix and liberty

      If you're a ROM whore like me, you're constantly switching between ROMs and not loyal to a single ROM. However, my two favorites in the recent past has been rubix and liberty. I like the features of liberty and the smoothness of rubix. I believe you can have both if you're handy with vi and command line. Here are the differences at the OS level. (I could explain what each does but that would take too long).
      • build.prop ("-" is rubix, and "+" is liberty) :
      -wifi.supplicant_scan_interval = 180
      +wifi.supplicant_scan_interval = 90
      • cpufreq ondemand:
      liberty: 86, rubix: 95
      liberty: 46875, rubix: 37500
      • minfree (/sys/module/lowmemorykiller/parameters/minfree):
      liberty: 1536,2048,4096,5120,5632,6144
      rubix: 2048,3072,6144,15360,17920,20480
      • kernel scheduler (/sys/kernel/debug/sched_feature):

      • And finally, sysctl.conf:
      Liberty (yikes):
      vm.dirty_background_ratio = 5
      vm.dirty_ratio = 20
      vm.dirty_background_ratio = 60
      vm.dirty_ratio =95
      Overall, Rubix tends to tune out of the box whereas Liberty settings are more or less stock and you'll have to tune them via the toolbox.  Or if you're like me, you tune them manually. :)  (Oh fyi - liberty will run quicker after applying some or all of the changes from rubix above.)

      Below will make things easier if you want to try it out yourself.  Make sure you have a backup!

        Monday, January 31, 2011

        Advanced overclocking for DroidX

        If you know your way around android terminal, overclocking your Droid X without a helper app is really not that difficult.  Here are the steps:
        1. Download and install Milestone Overclock (this is just so you can grab the kernel module).
        2. Find overclock.ko from the milestone directory and move/copy it over to /system/lib/modules.  You can optionally uninstall Milestone Overclock now - you no longer need it.
        3. Run the following script (be sure to substitute A, B, C, D with your own vsels, and W, X, Y, Z with your own frequencies).

          insmod /system/lib/modules/overclock.ko
          echo 0xc05a8dcc > /proc/overclock/mpu_opps_addr
          echo 0xc0048a00 > /proc/overclock/omap2_clk_init_cpufreq_table_addr
          echo 0xc059b258 > /proc/overclock/cpufreq_stats_table_addr
          echo 0xc02d1104 > /proc/overclock/cpufreq_stats_update_addr
          echo D > /proc/overclock/max_vsel
          echo Z00000 > /proc/overclock/max_rate
          echo 1 W00000000 A > /proc/overclock/mpu_opps
          echo 2 X00000000 B > /proc/overclock/mpu_opps
          echo 3 Y00000000 C > /proc/overclock/mpu_opps
          echo 4 Z00000000 D > /proc/overclock/mpu_opps
          echo 0 Z00000 > /proc/overclock/freq_table
          echo 1 Y00000 > /proc/overclock/freq_table
          echo 2 X00000 > /proc/overclock/freq_table
          echo 3 W00000 > /proc/overclock/freq_table
        My values: W = 3, X = 6, Y = 9, Z = 12, A = 22, B = 33, C = 48, D = 63

        If you want the new settings to stick at boot, you'd want to put the script in /etc/init.d somewhere.  You can name it 99overclock or something like that.

        As for how you can determine the best frequencies and vsels, you'll have to experiment.  If you're lazy, I recommend using something like Quickclock - the developer, Paul, is an awesome guy. :)

        Finally, here's a nice trick.  The numbers in the following file will tell you your usage pattern per frequency:
        Here's an example:
        bash-3.2# cat  /sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_state
        1200000 1424737
        900000 87456
        600000 182945
        300000 4688410
        You will notice that your phone spends most of the time at the lowest clock frequency and rarely will it use the middle ones.  Interesting, but expected.

        Saturday, January 29, 2011

        Android memory manager (low memory killer)

        Android memory manager works pretty hard and is constantly killing processes in the background to ensure that you have enough memory to run your apps and also to keep the phone running snappy.

        If you're as much of a control freak as I am, you kinda want to see which processes it's killing in the background.  It's actually pretty easy.  You can use either of the following methods:
        1. dmesg
        2. logcat
        Both commands should give you similar output.  Here's what happens when I run it on my Droid X: 

        bash-3.2# logcat -t 1000 | grep "has died"
        01-29 15:01:09.577  1278  1283 I ActivityManager: Process com.motorola.calendar (pid 8595) has died.
        01-29 15:07:46.957  1278  1278 I ActivityManager: Process (pid 8606) has died.

        bash-3.2# dmesg | grep sigkill
        <4>[58305.115783] send sigkill to 8595 (torola.calendar), adj 13, size 3198
        <4>[58702.255462] send sigkill to 8606 (, adj 14, size 2942

        Nice thing about using logcat is that it gives you readable timestamps, so you know exactly when they were killed.  Since I'm a bit OC, I have a script that runs both:
        bash-3.2# cat
        # tells me which recent processes were killed by lowmemkiller

        echo --- using dmesg ---
        dmesg | grep sigkil
        echo --- using logcat ---
        logcat -t 1000 | grep "has died"

        Friday, January 28, 2011



        I've been running telnetd on android for a while now, and the busybox shell sucks for obvious reasons - it's not meant to be a fully featured shell.  Luckily, there's bash.  I've been using bash on linux for years and I was stoked to find it for android. 

        You can download it here.  Or use Fission ROM manager to download it.

        Once you have the file downloaded, copy it to /system/bin, and make sure that permission is set properly.  And instead of using /system/bin/sh for telnetd you can use /system/bin/bash.  So your full telnetd command would look something like this:
        su -c 'telnetd -l /system/bin/bash -F'
        The above command runs as a sl4a script on my Droid X - super convenient this way.

        To exprience the awesomeness of bash, try typing a partial command or directory and hit Tab, try hitting up and down arrow, and Ctrl-A & Ctrl-R.  Oh, and adb sucks. ;)

        Thursday, January 27, 2011

        OOM and Android Part Deux

        (This is a follow-up to my previous post)

        Success! I was able to induce kernel's own oom-killer.
        # echo "0,0,0,0,0,0" > /sys/module/lowmemorykiller/parameters/minfree

        # vi /sdcard/largefile
        .. a few sec later (in dmesg):
        <3>[ 277.294128] Out of memory (oom_kill_allocating_task): kill process 2435 (vi) score 0 or a child
        <3>[ 277.294250] Killed process 2435 (vi)
        So really all you gotta do is "disable" android's built-in memory killer (which i'd imagine uses watchdog to do the actual work). What does this mean? Not a whole lot more than what I already said before. As long as you stick to the lowmemorykiller settings provided by your custom ROM it's not likely that the kernel's oom-killer will be invoked, and the vm.oom* sysctl settings will be ever useful. Android lowmemorykiller > linux kernel's oom-killer.

        Some juicy information here.

        And again, if you'd like to check your custom ROM's minfree, check somewhere in /etc/init.d to verify that the values are set and what they're being set to.  On Rubix:
        # cat /etc/init.d/02vm | grep minfree
        echo "2048,3072,6144,15360,17920,20480" > /sys/module/lowmemorykiller/parameters/minfree

        Wednesday, January 26, 2011

        OOM and Android

        Some of you were wondering what the last two lines in my sysctl.conf file are.  They control the behavior of the kernel when it detects a "OOM" event.

        # reboot when OOM happens
        vm.panic_on_oom = 2
        # wait 5 sec before rebooting when OOM
        kernel.panic = 5
        The reason I wanted my phone to just go ahead reboot is mostly because I had some bad experiences with the OOM Killer in the past on other linux systems.  On my phone, I want my phone to just kernel panic and reboot instead of me having to deal with the potentially crippled phone after the OOM killer is finished with the running OS processes and applications.

        So I decided to put my sysctl oom settings to the test.

        First, I needed to force the phone to run out of memory completely.  Step one, create a 600MB file:
        # dd if=/dev/zero of=/mnt/sdcard/largefile bs=1 count=600000000
        Step two, clear dmesg buffer:
        # dmesg -c
        Step three, create a process that attempts to use 600MB of memory:
        # vi /sdcard/largefile
        Step four, observe.

        Step five, analyze.

        So based on my test, it turns out Android kernel's OOM Killer is *not* invoked.  There's a watchdog process that detects that the phone is about to run out of memory and kills my vi process along with a bunch of other processes.  Evidence in dmesg (it's just a snippet):
        <4>[79814.464782] select 11855 (.android.tasker), adj 2, size 4694, to kill
        <4>[79814.465087] select 11863 (d.process.acore), adj 4, size 7712, to kill
        <4>[79814.465667] send sigkill to 11863 (d.process.acore), adj 4, size 7712
        <4>[79814.559112] select 11855 (.android.tasker), adj 2, size 4694, to kill
        <4>[79814.559204] select 11871 (e.process.gapps), adj 2, size 5449, to kill
        <4>[79814.559356] send sigkill to 11871 (e.process.gapps), adj 2, size 5449
        And what is process id 4?
        root      4     2     0      0     c009a4e8 00000000 S watchdog/0
        How many processes did watchdog kill?
        # grep -c "to kill" /sdcard/dmesg.log
        Yikes.  Anyway, I ran the experiment several times (oom killer on, off, etc.) all producing the same results, and still could not invoke kernel's OOM killer or cause the kernel to panic.  I'm guessing Android developers decided that the watchdog managing oom situation is superior than leaving it to the kernel.  Probably a good thing too - OOM Killer sucks.

        So what does this mean?  This means all the oom related settings in sysctl are probably useless.  (It also means that for all intents and purposes the watchdog process is the new oom killer, but outside of the kernel). Because android kernel is derived from a standard linux kernel, there are going to be a number of kernel parameters that are not used, and the oom ones are possibly part of that.  Therefore, my latest sysctl.conf file reflects my latest experiement:
        # cat /etc/sysctl.conf

        # try to keep at least 4MB in memory
        vm.min_free_kbytes = 4096

        # favor block cache
        vm.dirty_ratio = 90
        vm.dirty_background_ratio = 55

        # extremely favor file cache
        vm.vfs_cache_pressure = 1
        # ignore below for now - not used by android
        vm.swappiness = 0

        vm.panic_on_oom = 2
        kernel.panic = 5
         As far as the watchdog process is concerned, I looked around a bit.  Didn't find /etc/watchdog.conf, didn't see an obvious place where it starts up, etc.  Must be part of the android base os somehow.

        Tuesday, January 25, 2011

        Enabling sysctl

        First of all, I'd like to thank all the ROM developers for making my phone so much fun to play with.  Without them it will be like having an iphone. :)

        Secondly, I strongly suggest enabling telnetd.  It makes typing (and copy/pasting) so much easier, and you can also mess with the phone while your son is playing Angry Birds on it!

        Enabling sysctl on custom ROMs is mostly pretty easy.  Roughly the steps are:
        1. Verify you can run custom scripts at boot.
        2. Check to see if sysctl is already enabled.
        3. Create/modify /etc/sysctl.conf
        4. Verify that the new values stick at reboot.
        BTW, if your ROM already allows you to enable/modify sysctl via settings, you can skip 1-2.
          Let's start with #1.  First open up Terminal and/or telnet into your phone.  Then verify that you got an init.d directory on the phone (I'm currently running Rubix):
          # ls -l /etc/init.d
          -r-xr-xr-x root     root          759 2008-08-01 05:00 01cpu
          -r-xr-xr-x root     root          230 2008-08-01 05:00 02vm
          -r-xr-xr-x root     root          228 2008-08-01 05:00 03pfm
          -r-xr-xr-x root     root          100 2008-08-01 05:00 04screen
          -r-xr-xr-x root     root           72 2008-08-01 05:00 05zipalign
          -r-xr-xr-x root     root          344 2008-08-01 05:00 98firstboot
          -r-xr-xr-x root     root          233 2008-08-01 05:00 99complete
          Oh look, it looks like I can run some custom scripts at boot time.  If /etc/init.d doesn't exist, stop.  This gets messy and honestly I wouldn't recommend going further because you could potentially bootloop your phone if you try to modify anything other than what's in this post.

          #2. At first glance I don't see an init script named sysctl like the Liberty Rom, but it's possible that one of the scripts takes care of it:
          # grep sysctl /etc/init.d/*
          /etc/init.d/02vm:# Load /sys/etc/sysctl.conf
          /etc/init.d/02vm:sysctl -p
           Yup, 02vm file enables sysctl.  "sysctl -p" takes what's already in /etc/sysctl.conf and applies it to the kernel.  If you don't see any init script running sysctl -p, then you'll want to create one.  Call it something like 99sysctl, and then simply add "sysctl -p" in the file.

          #3.  Modify /etc/sysctl.conf.  Here's what my latest looks like:
          # cat /etc/sysctl.conf
          # shouldn't matter - no swap is used
          vm.swappiness = 0

          # try to keep at least 4MB in memory
          vm.min_free_kbytes = 4096

          # favor block cache
          vm.dirty_ratio = 90
          vm.dirty_background_ratio = 55

          # extremely favor file cache
          vm.vfs_cache_pressure = 1

          # reboot when OOM happens
          vm.panic_on_oom = 2

          # wait 5 sec before rebooting when OOM
          kernel.panic = 5
           #4. Verify that the sysctl init script is working.  Reboot the phone and run this command:
          # sysctl -a | grep vm  
           "sysctl -a" will display all the kernel settings, and what you're interested in is the section that you modified, hence "grep vm".  The output should look something like this:
          # sysctl -a | grep vm     
          vm.overcommit_memory = 1
          vm.panic_on_oom = 2
          vm.oom_kill_allocating_task = 0
          vm.oom_dump_tasks = 0
          vm.overcommit_ratio = 50
 = 3
          vm.dirty_background_ratio = 55
          vm.dirty_background_bytes = 0
          vm.dirty_ratio = 90
          vm.dirty_bytes = 0
          vm.dirty_writeback_centisecs = 500
          vm.dirty_expire_centisecs = 200
          vm.nr_pdflush_threads = 0
          vm.swappiness = 0
          vm.lowmem_reserve_ratio = 32
          vm.drop_caches = 0
          vm.min_free_kbytes = 4096
          vm.min_free_order_shift = 4
          vm.percpu_pagelist_fraction = 0
          vm.max_map_count = 65530
          vm.laptop_mode = 0
          vm.block_dump = 0
          vm.vfs_cache_pressure = 1
          vm.mmap_min_addr = 4096
          vm.scan_unevictable_pages = 0
           There you can see that the values in your customized /etc/sysctl.conf have taken effect.  Great success!

          top: part 2

          Sometime I wonder what's eating up the memory on the phone.  Ok, I lied - I wonder all the time. Well, there's an easy, reliable and accurate way to get your answer using what else, top!

          Forget all those apps you see in the market, uninstall your task killers.  Top is where it's at!

          Let's take a quick look.  The command I used below will show you the top 10 memory hogs that are currently running on my Droid X.

          "-m" tells top to show the top 10 entries, "-n" tells it to run it 5 times and quit, and "-s rss" tells it to sort it by memory usage.

          # top -m 10 -n 5 -s rss

          User 5%, System 5%, IOW 1%, IRQ 0%
          User 16 + Nice 0 + Sys 18 + Idle 268 + IOW 6 + IRQ 0 + SIRQ 0 = 308

            PID CPU% S  #THR     VSS     RSS PCY UID      Name
          18230   2% S    70 355932K  69068K  fg system   system_server
          26822   0% S    21 203692K  35368K  bg app_32   com.alphonso.pulse
          23414   0% S    31 214036K  33912K  bg app_63
          30083   0% S     8 178724K  28324K  bg app_86   android.process.acore
          19879   0% S    13 177228K  26544K  bg app_76
          24994   0% S    14 176612K  25364K  bg app_100  com.quoord.tapatalkpro.activity
          30419   0% S    15 175128K  23964K  fg app_110  com.googlecode.android_scripting
          30356   0% S    11 166524K  23876K  fg app_113  com.genusproject.apps.eztvlisting
          18311   0% S    12 186164K  22976K  fg app_33   net.thinkindifferent.inputmethod.latin

          You can ignore system_server, but you see what floats up on top? 

          /me uninstalls Pulse.  If an app is going to take up more than 30MB RSS and 200MB VSS, it better be a killer app like the Google Reader.

          Sunday, January 23, 2011


          Top is probably the most popular built-in tool on linux, and has been around for ages.  And busybox gives you top!

          Have you ever noticed that sometimes your android device feels really sluggish and your cpu graph app (I use CpuNotify) is showing that your CPU is pegged?  I believe the most effective way to determine what's causing the phone to slow down is top.  Don't install apps that do similar stuff from the market - top will be more accurate and reliable.

          Let's look at an example.  The other day my DX was just running dog slow and CpuNotify was telling me that the CPU was maxed out.  So I launch Terminal, run top, and scroll up:
          # top -m 4 -n 3 -s cpu
          User 83%, System 2%, IOW 4%, IRQ 0%
          User 217 + Nice 0 + Sys 7 + Idle 75 + IOW 13 + IRQ 0 + SIRQ 0 = 312

            PID CPU% S  #THR     VSS     RSS PCY UID      Name
          27769  80% S    21 236568K  69156K  bg app_74   com.dolphin.browser
          18230   1% S    71 359916K  61852K  fg system   system_server
          27888   1% R     1    848K    420K  fg root     top
          18311   0% S    12 186360K  22416K  fg app_33   net.thinkindifferent.inputmethod.latin
          [ snip ]
          WTF!?!  What's my browser doing?  It turned out I made a classic mistake of going to a javascript intensive website and forgot to close the window.  Nine out of 10 times when your phone seems really sluggish, it's probably a runaway javascript that's still running on a browser window in the background.

          So what do you do then?  Well you can bring the browser to the foreground again and close the offending tab/window.  Or you can just kill it from shell:
          pkill -f browser

          Friday, January 21, 2011

          let's add swap!

          Android, at least on my Droid X, doesn't use swap.  And I'm sure there's a good reason for that - one of them being that it doesn't need one.  As long as it has plenty of memory and killer memory management why bother?

          Still, I thought it would be fun to add since Android runs a linux kernel and linux likes swap.

          I think you can do some interesting things. If you have swap, you can perhaps encourage the kernel to swap out some rarely used processes like Xfinity, etc. (or at least some portions of it. Isn't it annoying how Xfinity always sticks around). Then you can tell the kernel to use more of the RAM for cache, thus making the phone faster. I know android already does a good job in managing memory and killing off less used processes but I think with swap we'd have a lot more flexibility.

          Well, let's give it a shot, shall we?
          # dd if=/dev/zero of=/mnt/sdcard/swap bs=1k count=100000
          100000+0 records in
          100000+0 records out
          100000 bytes transferred in 0.713 secs (140252 bytes/sec)
          # mkswap /mnt/sdcard/swap
          Setting up swapspace version 1, size = 95904 bytes
          Whoa! Looking good...
          # swapon /mnt/sdcard/swap
          swapon: /mnt/sdcard/swap: Function not implemented

          Thursday, January 20, 2011

          Sysctl tweaking for faster, longer lasting Android

          If your ROM supports sysctl, you can do lots of cool things.  (Well even if your ROM doesn't you can as long as you're rooted with busybox, but it takes a bit more work).

          In this post, let's focus on three settings: vm.dirty_ratio, vm.dirty_backgroud_ratio, and vm.vfs_cache_pressure.

          vm.dirty_ratio and vm.dirty_background_ratio control how often kernel writes data to "disk" (well in our case the microSD card).  When your apps write data to disk, Linux actually doesn't write the data out to disk right away, it actually writes the stuff to system memory and the kernel handles when/how the data is actually going to be flushed to disk.

          I dunno - how much difference can it actually make?  We're not talking about traditional hard drives with spindles, we're talking about solid state stuff.  I suppose there could be a slight savings in battery life and increase in performance if you delay the data flush as much as you can.

          I ended up with:
          vm.dirty_ratio = 90
          vm.dirty_background_ratio = 70
          Now vm.vfs_cache_pressure is much more interesting. File system cache (dentry/inode) is really more important than the block cache above, so we really want the kernel to use up much of the RAM for them.  The default value is 100, and what you want to do is lower that value to tell the kernel to favor the file system cache and not drop them aggressively.

          You can also take it a step further and set it to 1 (lowest possible without being dangerous).  With the value of 1 the kernel will drop cache only when it's completely out of memory.
          vm.vfs_cache_pressure = 1
          The problem is if you lower the value too much, after a day or two your device may start getting sluggish because the amount of RAM available to the applications continue to shrink to the point where they are starved for memory.  So I came up with a solution to manually drop caches nightly!
          echo 3 > /proc/sys/vm/drop_caches
          # tells kernel to drop all file system caches
          And what I do to ensure that the caches are dropped nightly is to run the following sl4a scripts via tasker.  Cron would be better but busybox on android doesn't seem to support it yet.

          (It also reverts the dirty settings because I don't care if my phone writes to disk aggresively while i'm sleeping and it's being charged.)
          su -c 'echo 3 > /proc/sys/vm/drop_caches;\
          sysctl -w vm.dirty_background_ratio=3;\
          sysctl -w vm.dirty_ratio=15'
          And the following script at wakeup time:
          su -c 'echo 3 > /proc/sys/vm/drop_caches;\
          sysctl -w vm.dirty_background_ratio=70;\
          sysctl -w vm.dirty_ratio=90'
          Finally, here's what my sysctl.conf file looks like:
          # cat /etc/sysctl.conf
          # shouldn't matter - no swap is used
          vm.swappiness = 0

          # try to keep at least 4MB in memory
          vm.min_free_kbytes = 4096

          # favor block cache
          vm.dirty_ratio = 90
          vm.dirty_background_ratio = 70

          # extremely favor file cache
          vm.vfs_cache_pressure = 1

          # reboot when OOM happens
          vm.panic_on_oom = 2

          # wait 5 sec before rebooting when OOM
          kernel.panic = 5

          # currently experimenting
          kernel.shmmax = 268435456
          kernel.shmall = 16777216
          I've been running at the above settings for a few days now, and the phone has been faster than ever with zero ill effects.  Your mileage may vary of course.

          UPDATE: Read my follow-up post!

          Wednesday, January 19, 2011


          Let's see what else you can do with busybox.  Oh hey how about dmesg?  That's like one of my favorite linux commands!  Dmesg on android must show all kinds of cool interesting things that the kernel is doing!

          # dmesg | tail
          <6>[22506.486755] TIWLAN: 2421.209967: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
          <6>[22507.057128] TIWLAN: 2421.780340: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
          <6>[22509.056701] TIWLAN: 2423.779944: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
          <6>[22540.139587] TIWLAN: 2454.862768: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
          <6>[22543.117156] usb_ether_get_stats
          <6>[22543.136444] usb_ether_get_stats
          <6>[22544.635040] TIWLAN: 2459.358252: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
          <6>[22547.410217] TIWLAN: 2462.133398: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
          <6>[22548.908538] TIWLAN: 2463.631720: TrafficMonitor_ChangeDownTimerStatus: Going to power save, clear wake_lock!!
          <6>[22549.147735] TIWLAN: 2463.870947: TrafficMonitor_ChangeDownTimerStatus: Going to active, set wake_lock!!
          That was stupid.

          Enabling remote telnet access to your phone

          I was messing with busybox on my Droid X the other day using the tiny android gingebread keyboard, and started wondering "dang, i'm making too many typos and this is really a pain".  So I decided to enable telnetd so I can telnet in from my macbook and use my gigantic keyboard.   Here's how you do it:

          1. get on a secure wifi network (well you don't have to but you'd want to - trust me)
          2. run Terminal from your phone
          3. type "telnetd -l /system/bin/sh"
          4. obtain your phone's IP address from android settings
          5. run terminal software from your laptop/desktop
          6. telnet to the ip address
          7. when you're done, kill telnetd: "pkill telnetd"
          Voila!  Now you can hack away without having to use the phone keyboard, or connect it to your PC via USB.

           [insert extremely satisfied happy face here]

          UPDATE: Be sure to read my followup post here!