Success! I was able to induce kernel's own oom-killer.
# echo "0,0,0,0,0,0" > /sys/module/lowmemorykiller/parameters/minfree.. a few sec later (in dmesg):
# vi /sdcard/largefile
<3>[ 277.294128] Out of memory (oom_kill_allocating_task): kill process 2435 (vi) score 0 or a childSo really all you gotta do is "disable" android's built-in memory killer (which i'd imagine uses watchdog to do the actual work). What does this mean? Not a whole lot more than what I already said before. As long as you stick to the lowmemorykiller settings provided by your custom ROM it's not likely that the kernel's oom-killer will be invoked, and the vm.oom* sysctl settings will be ever useful. Android lowmemorykiller > linux kernel's oom-killer.
<3>[ 277.294250] Killed process 2435 (vi)
Some juicy information here.
And again, if you'd like to check your custom ROM's minfree, check somewhere in /etc/init.d to verify that the values are set and what they're being set to. On Rubix:
# cat /etc/init.d/02vm | grep minfree
echo "2048,3072,6144,15360,17920,20480" > /sys/module/lowmemorykiller/parameters/minfree