The Sickness is Revealed

The Sickness: Why my T42 randomly hangs.

This is what is wrong with the Thinkpad. The T42 is hanging because the linux kernel does not manage the video memory correctly on it's Radeon 9600.

I can now only sit here and wonder how long the patches will take to be merged into the main linux kernel, then into the ubuntu system. Of course, I suppose I could try compiling a patched kernel myself...

... But it has been pointed out that I have spent a great deal of free time, perhaps a couple of weekends and all the free evening I can muster over the past 2 weeks attempting to solve this little thinkpad crash faux-pax. I think my interests are better served by just being patient, at this point. This is one of those things I would like to -work- so that I can get other things done. I am somewhat dissenterested in crossing all the bridges it takes to effectively debug kernels when I would really like to be thinking about my photography or looking for exciting new ways to analyze geographic data. Or writing...

I keep thinking I will stop using the mac due to it's age and incredible milage, however, it keeps turning on when other machines go up in smoke. It's not hard to really like a computer that works this well.

Myth busters wave data shows we are in the middle of the rally!

Autumn is Finally Creeping in...

I am still working the bugs out for posting photos. However, something is better than nothing, yes?

Why, yes! That is prickly pear in Virginia!

Full Album

Reverse Typo Squatting 101

This morning I received an urgent message from someone highly placed in the large government organisation that I work for.

Apparently, we used to have a 1-800-x number connected to a very popular, publicly available data product. This number was prominently located on the data product's web pages. However, times have changed and phones are not really the default mode of business for the product any more. Thus, that phone number has been defunct (no ringy!) for several years, but many pages have not been updated.

This 1-800-x number has been purchased and activated by an adult services company. This is not related in anyway at all to the product our agency was providing.

Removing all traces of it from *.gov webpages is now a national priority.

Kernel Panic

I dissabled pm-tools.

I dissabled xfce4-power-manager

I dissabled acpid.

I'm doing it wrong.

The T42 induces kernel panics under ubuntu 8.10, 9.?? 10.04, and 10.10 for some reason. I think it likes to do it most when running on battery.


And, to answer the anonymous commenter's question: yes, this is not isolated, but it is poorly documented.


T42 Kernel Panic


Drat.

Next up: can I get a core dump?

An Unusually Balmy Morning

That's what the NPR announcer said. Unusually balmy.


I could have left the doors open to let the morning air in. The house is drafty, with single pane windows and questionable insulation. The damper for the huge fan that can vent to the attic is certainly not air tight. Yet, somehow, it can get stuffy in here. Leaving the doors open sounds nice.

Fresh air, NPR, and coffee. I mill around waiting for the sun to rise, so I can ride to work without worrying as much about be invisible to the vast ocean of Mercedes and Hondas that passively seek to kill me, guided by the forces of evil that release text messages into the ether just as i approach a crosswalk.

-I am not a squirrel, I will leave a dent-

The warmth is welcome. I'm trying to ride my bike to work as much as possible. I have not been successful in finding a way to work in an hour for running in my day. I suspect if I screwed around somewhere else a little less, it could be done. However, if I can get exercise and commute at the same time, all the better. I must admit. My ride to and from work is one of my favorite parts of my day. I'm trying to avoid the inevitable need to wear gloves and a hat. Last week I got chilblains a couple of times. The day is not far off.

An interesting aspect of the complexity of our digital world is that, instead of checking my email for messages for my parents, I must now search the Internet to find out what they are doing. While this situation has not yet reached the point proverbial sewing-tool-in-arbitrary-pile-of-silage, I am surprised to find a few pages of text here and there. To find not one, but two extra entries ghost written in my brother's blog explained much, and assuaged my lingering worries that they were in trouble somewhere.

I find that I do not miss graduate school at all, perhaps with the few lingering exceptions that I have very much changed fields, everyone I knew there is still there, and I now have to learn an entirely new body of material. I spend a lot of time asking people about the science that they do. I am more also more cautious than ever in attempting to analyze and explain what is going on. I feel very green.

I also feel really good. I pulled off my dissertation. I somehow found a job. We pulled off the move. We didn't go broke in the process. Reviewing finances last night, we'll be cash flow positive again with the next pay-check. Things are looking up.

And the dog is snoring in the other room.

The sun is up, it is time to go.

(and the thinkpad crashed, so you get to read this in the evening!)

Digital Breadcrumbs

Over the past month-ish I have been taking my GPS on my regular bike route. My intent has been to use the data I collect to do detailed analysis of my travels.

I had meant to dump and analyze data (the gps only holding a ~1500 datapoints) this weekend, but then I got wrapped up in understanding linux power management.

Until I figure out more, here are some initial results for your thinking bits to think on.

Too Much Thrift...

I have become a little obsessed with the random crashing on the Thinkpad (a T42). Having a computer that hangs every 20-120 minutes is really weird and annoying.

Hangs without logfile notes are evil. That leaves me to trouble shoot by trial, error, and google in the middle. This also makes the problem more intriguing than average.

Trouble Shooting Timeline.

0) Discovered random hangs. Did a complete reinstall of xubuntu 10.10 from cd. Googled a lot. Found thinkwiki article that mentioned conflicts between bios power management (pm) and linux pm.
1) Disabled bios PM: this helped a lot.
2) Installed laptop-mode-tools to try to push things a little longer. Crash reappears!
 -Maybe crash is related to some specific aspect of powersaving?
3) Fallback to basic powersaving (whatever that is... more later). I thought it was fixed until the computer hung at 10% battery. Maybe XFCE4-power-manager can do something to cause a crash too, but it's less agressive? Maybe some other aspect of pm gets wrapped around the axle in panic-powersave?
4) Google more...

How does linux power management work?

Have you ever wanted to know how linux manages power on your computer?

You want to know, trust me:
 If you're a programmer, you'd love to see the power, and want to spend every waking moment of the next week trying to harness it.
- If you're a more normal human being, you need to see the horrorshow mess that is going on under the hood.

In short, The community has taken a lot of jabs at the problem, so now we are left with a really pissed off bull running around town with lots of spears stuck in it, so it's bleeding and shitting all over the place and destroying everything in it's path. GO TEAM!


Linux power management has gone through a lot of iterations, and there is a mish-mash of solutions to the power management issue. From I learned today.... and links below (no authority guaranteed here or there, mind you!):

• speedstep-centrino became acpi-cpufreq
• acpi-cpufreq got rolled into the 2.6.31> ubuntu kernel... so no module
• something about Ubuntu reverting to a acpi-cpufreq module under some vague rumored condition
• cpufreqd and powernowd are now userspace daemons: that is something you can control w/o loading modules from the command line....
• powernowd is the preferred method of control, though cpufreqd could still be in the wild.
• hal, although depreciated, could still be messing with your pm settings.
• udisks-daemon/udisks can exhibit some control over hard drive power usage. Provides disk interface to freedesktop.org dbus interface.
• apmd (advanced power management daemon) was used to control power on older machines, but was supplanted by acpid. (also deprecated)
• acpid (advanced control and power interface daemon): Modern machines have acpi, and this package executes commands based on changes in the computers hardware state (temperature, power, buttons, ect....)
• Power status can be found in /proc/acpi 
• acpid executes commands based on events from /etc/acpi/ and /etc/acpi/events. Notably, the ac and battery commands in ~/events.
• sleepd can put the machine to sleep... seems very basic. not part of standard ubuntu install
• xfce4-governor-plugin used HAL to change the CPU speed, but is now depricated along with hal
• pm-utils appears to be the current architecture for basic power control. I am not sure if it is part of the standard ubuntu installation. maybe just on laptops? There are some scripts in /usr/sbin/ that determine this.
• pm-utils 1.4.x that ships with xubuntu 10.10 is incompatbile with laptop-mode-tools. It says it right on the freedesktop.org webpage!
• apt-get removes pm-utils when it installs laptop-mode-tools.
•  laptop-mode-tools is a set of scripts that provides fairly aggressive power. It modulates daemons, kernel laptop mode, and other hardware settings (hdparm) to squeeze power savings out of a machine
• curiously, apt-get recommends installing pm-utils, even though the packages hate each other now.
• management, making attempts to control almost every aspect of a machine.
• upowerd/upower monitors power devices  and posts changes on the freedesktop.org dbus interface. Running upower -m will show the messages when the computer is plugged/unplugged. Things that listen on dbus might respond...
• xfce4-power-manager monitors and controls power behaviour through dbus signals
• gnome-power-manager C based power manager that uses upower, dbus, and libnotify. Ironic quote:

"Power management is an essential job on portable computers, and becoming more important on todays high-powered desktops. It uses many complex (and sometimes experimental) parts of the system - each of which are slightly different, and may contain quirks to work around. The power management policy could be influenced and tweaked by an huge number of options, and each new laptop model brings more possibilities and options. This should all work in the background without even being noticed by the user." (emphasis theirs!)

•  hdparm is used to set the power management of hard drives
• sdparm is the bastard cousin of hdparm, used for things that pretend to be, or really are, scsi. (for those poor bastards who still have it).
• Network I/O can be optimized through
• iwpriv (for older wireless)

• /sys/bus/pci/drivers/iwl*/...(for newer wireless)

• ethtool (for ethernet)

• xrandr: controls active video outputs
• x.org dpms module controls the lcd on/off state
• lcd backlight control: mysterious! xbacklight is listed...
• The virtual memory subsystem tries to optimize things when there is a limited power situation, with variables set in /proc/sys/vm/
• cpufrequtils: provides some scripts that allow users to easily control the cpu speed. (possible to do this by writing values to /proc and /sys...)
• uswsusp allows suspend and resume by writing system state to disk
• noflushd: buffers hard drive writes to allow hard drives to spin down as long as possible
• rovclock, radeontool: control radeon card based clock speed and lcd brightness
• "Dynamic Clocks" option in the xorg.conf file also changes gpu speed.
• kms seems to have replaced xorg.conf
• unless you've disabled kms, then you need xorg.conf
• how kms interacts with power management and cpu speed is mysterious and a popular source of hangs that are posted in discussion boards
• This is the best info I have on using /sys/* to manage radeon: http://wiki.archlinux.org

It appears that there are three general pathways for power management. ACPI finds out that the power state has changed, then dbus, hal, and sysfs get updated. From there, the various daemons and power managers take over firing off scripts that control the sundry power settings on a modern machine.

It is probably important to dissable the xfce/gnome/kde power managers too...

So, finding the source of the crash in my computer entails fiddling with each subsystem until one generates a crash. Fun times.

Fun things to read about Power Management:

• http://packages.ubuntu.com: What comes with ubuntu, how it is connected
• http://lesswatts.org: Good general discussion, though they fall through on explaining cpu governors...
• https://help.ubuntu.com/community/PowerManagement/ReducedPower: How to use scripts to set up power management like you would like...

Capital Offence

This Saturday, the the girly and I took a small jaunt into the District (meaning, we left the land of lovers into the land of the taxed and unrepresented) (no not Puerto Rico).

A filthy, unwashed mass among the masses swarming the capitol.

We took the cattle car orange line to the Smithsonian stop, and then took a multi-hour long stroll along the length of the mall.

One of the things that I had not expected was the sheer volume of people there. DC is, by far, the largest tourist trap in the country. There were people everywhere.

There was also a protest or rally happening.

Let me paraphrase what the speakers were saying.

"GOD...BIBLE...AMERICA...REVENGE...BIBLE...AMERICA...BLESS...BIBLE...GOD...PSALMS...AMERICA...AMERICA...GOD...BIBLE...REVENGE..."

I now realized why protests don't cause change. There is probably a protest and rally in DC every weekend. I'd ignore it too. The Park Service has fenced off sections of the mall so the grass can regrow since the hoards have stampeded it to death.

And oh, hoards there were.

Hoards at the congress building, hoards at the monuments, hoards at the bathrooms, hoards and the reflection pool, hoards at the cross walks. Hoards hoards hoards hoards hoards. It was a windy day, too. I'm sure that kept the feeble inside with their soaps, but it was not enough to render the helm of our great nation desolate.

I honestly wonder how the president and the FLOTUS can find a quite place to make out with all the hoarding going on.

American public: 307,006,550 peeping Toms must be assuaged!

I told you guys I was going to blog this.

Whitehouse being loved to death.

A small hoard tramples the mall.

The Red Baron Hoards the capitol building

Hoards rap down the dome.

This must be one of those joke photos where the important thing is behind the photographer. (e.g. 'This is me at the Washington Monument'). Her photo will probably have nothing but her flavour of the week and some unremarkable cluttered background in it. She's probably just taking it to make him happy. I bet she's going to dump him soon.

All I could hear was Lisa Simpson trying to get a word in edgewise.

An very intriguing aspect was the the light poles. The street light poles do not appear to have been changed or painted since sometime in the 60's, yet the lights themselves have had some amount of maintenance. The result of an intriguing juxtaposition of new metal and peeling relic.

Broken the lock, added new hasp. Broke it again. Yei string.

Juxapose.

Peels like burnt skin.

Odd notions:

At least they own up to driving poorly.

The Canadian embassy is one of the cleanest buildings in the city.

Flags and eyeballs at the FBI.

Water, water, everywhere.

Autumn Vespas

Not that kind of fountain.

Icarus Pheonix Hybrid

You would forgive me for finding this network of interactions the least bit hard to wrap my head around.

Damn near impossible after a couple of beers, really.

Worse, it's missing parts!

Don't believe me?

Find the computer's BIOS in there.

I think it's hidden in the blue box, but I'm not sure.

Most laptops BIOS provide some mechanism to handle performance and power consumption if the OS does not take the lead. There have been some arguements (including from chipset makers) that the powermanagement would be -best- handled by the BIOS... but that's another side track we don't want to derail on).

Fortunately, some poor bloke (I think on thinkwiki.org.... but I forgot) mentioned that linux seems happier if you disable all the energy control settings in the BIOS.

After a weekend of constant on time, the computer has not crashed, yet. Though posting to blogger has been my favorite place for it to crash.

A BIOS powermanagement conflict would explain why it never crashed while plugged in....

Every Rose Has It's Thorns

The stinkpad managed to survive grad school on it's original windows installation. Somewere in 2007 it began to get a little cranky, and by 2009, the hard drive was full to the gills and it had developed the characteristic unresponsiveness that plagues a well used windows boot disk. Then, 2 weeks before my defence, it caught a rootkit. While I was able to extract part of the virus, it also became somewhat less stable... the wireless won't connect reliably and there a few blue screens to deal with.

Did I mention I wrote a dissertation on it? Yea, the keys are pretty polished too.

One of the hangovers from graduate school is that my dissertation and resulting papers are written under Word and Endnote. In fact, my entire reference library is in Endnote. I hope to write my new work in LaTex, but it is not expedient at all to convert old things.

So, simply pulling the drive and replacing it with a new one is still out of the question while I get my papers submitted (Endnote is on loan from my former research group, they'll want it back).

In the mean time, i've been trying to install Ubuntu on the expansion bay hard drive. You'd think that getting this working would be easy! Just run the Ubuntu install disc and go! Right?

Wrong.

Something about the Thinkpad causes Ubuntu to hang randomly, without leaving a trace in any log file. I thought there might be an error on the drive or the install disc. Trying a new install disk made things worse, because the newest ubuntu installer does not control where it writes grub well (POOR IMPULSE CONTROL), and ended up nuking the windows MBR on the main drive.


That wasn't inconvenient at all!

When it is not randomly hanging, I like it Xubuntu alot. It boots fast, it's clean and tiny, and it's unix. These things are goey happy joy to me. But then it randomly hangs and requires a reboot.

*sigh*

This is why I still have the mac...

Ph.D. In Anger Management

Packrat was right: Graduate school took a lot longer than I thought. This is something you should prepare for.

Higher degrees do not happen on anything that really resembles a schedule. Even if your research group dissolves and you defend and submit your dissertation, the document must still pass through the bowls of the university bureaucracy. Example: I defended in August and my dissertation was actually accepted last week.

You can't just leave the system while the dissertation makes it's rounds. This means that the worst possible time to leave is at the beginning of a new year, because you'll get stuck with a ton of fees that you'd never have a hope of recouping. (It cost $850.00 for mandatory health insurance which I would be in a position to hold and use for approximately 1 week!)

As a result of the ongoing limbo situation, every week or two something new and exciting must be dealt with using a careful combination email, phone calls, and credit cards. Someone needs more copies of the signature page, copyright clearance, a survey, or some inane form. Not only do you get to hate grad school while you're there, it hunts you down with fresh annoyance so you can regret it on a regular basis!

Big Ass Korg

We went up to Baltimore last weekend to help my brother schlep his accumulation of matter from his old flop to his new one.

Baltimore claims to be "America's Favorite City." Favorite must be some kind of euphamism.

Either that, or America loves row-houses and urban decay.

The major, remarkable difference between his old house and his new house (aside from the crime rate) was that the new one has right angles between the front and side walls, while the old one was a giant parallelogram.

Of course, the over arching joy of helping someone move is that you get to see the kind of stuff they have hidden away.

In this particular case, we were treated to the discovery of a 6 foot long Korg keyboard in a closet. The keyboard clearly was your normal cheap-dsp-from-MalWart variety.... it took two normal people to carry (or one brother...). Due, in part, to it's all metal construction, analog electronics, and 80's hairband stickers.

It is much nicer to not be the ones moving. It is a huge, expensive chore.

Crap. I would get a flu shot and then develop the potential side effects... *cough*

Perhaps the best part of moving has been that i have seen my brother more times in the past month than i used to in a year.

You are my lucky star. You... Lucky, lucky, lucky, lucky, lucky.

Before you start, i want you to know, I have taken a lot of time looking at the screens on my back porch. I want you to know this so you can trust that i know what I'm talking about.

The back porch of the loveshack is a screened-in concrete pad. The screen walls are subframes that are bolted to the pad and tacked to the supports that hold the roof up. The screens were installed in the subframes when they were laying down, so that gravity would aide in their construction. In particular, a 3 foot wide, 7 foot tall piece of aluminum screen is not fun to try to install while balancing on a ladder.

The original construction was 50 years ago, and the hardware holding the frames to the house have now all rusted. Replacing the screens is a ladder-based affair.

When we moved in, there were several panels that were severely damaged, and I took an afternoon carefully removing the flashing around the panels and applying fresh aluminum screen. (Previous tenants in our tenement had found this untenable, and stapled to the flashing.) This, combined with $15 worth of citronella candles, greatly improved the porches appeal as an evening bug-free get-away.

This is very important to have, and thus why I know about the screens.

Then, something curious happened. I received a flustered text message from The Girly. Something was putting holes in the nylon screen. Holes that are the size of a fat pencil. Given the pucker of the screen is facing the great unknown wilderness of the park behind the house, it was something that appeared to want to be out of the porch.

Whatever this could be stumps the imagination. We have been witness to quite the arthropod menagerie: There is the 18:00 mosquito rush, the unseen noseeums that leave us with the itchies, the thumb sized cricket in the bedroom, june bugs... june bugs... june bugs, stick bugs, preying mantis, flies, and even a few spiders. But what would want to get out? The dog has not produced anything remarkable, and he has not been displaying any aggression that was not linked to an obvious stimulant. There are no cocoons or nests. Nothing went bump in the night. It is a small mystery that I hope will not grow in size.

Just outward holes in the screen.

"Snakes?", I suggested over breakfast.

Not in the screen.

"Rocket propelled snakes?"

Somehow, those don't seem as scary.

Damnit: i knew it would happen one day. I tried to open my bike lock with my voicemail PIN.

Everything I have ever learned is wrapped in blankets on the street.

Stuff-Be-Gone

Whoever said "A rolling stone gathers no moss" never lived with a packrat.

I would bet $10 that I could throw away the box of ethernet cables and networking hardware I stuck in the basement. I am becoming increasingly certain there are other items that I will never need, as I have not needed them for years.

When I find something useful, and get a lot of use out of it, it leaves a signiture of in my mind, and I try to hold it in that useful state, even though it's time has gone.

There are many books I own that would fall into the same catagory.

I save things because I feel they might be useful one day. That day does not often come.

The Dog is Being Cute, He Must Want Something

It's worse than just that.

When sitting, his nose reaches about 3 inches taller than the keyboard. He wants to smell what is so interesting here.

Oooh... now he's standing and doing the big-black-eyes/headcock thing. His hair is fluffly and has a faint synthetic fruity smell.

How cute!

Of course, he is doing all this because there is a bunch of workmen next door with a woodchipper. It's all a very good act.

He wants to bark at them.

He's not allowed to do that.

I get a cute dog as a bonus.

This is pretty neat.

I am wondering what other benefits could be revealed through additional dog training?

I am sure a 20 minute commute with a pan of fresh cornbread in the car is more distracting than texting.

Badge-a-me now or badge me not...

Chemist Creates Home Networking Disaster, Innocent Dog Traumatized

I decided to go through the two large boxes of random electronics that I had accreted since 2003-ish.

I have many things that I no longer remember obtaining.

I found at least 15 Cat-5 ethernet cables. I also discovered I have several 802.11x access points and routers (one for -b and one for -g, to go with the -n that Verizon gave me). I also have a 10-BaseT hub and a 100-BaseT switch, a wireless bridge, an 802.11b pcmcia card.

I am tempted to wire it all up to maximize the blinking lights in my office.

There are least 5 wall warts that do not appear to match anything I own. One even says "Atari" on the back.

There are the guts of 3 laptops.

A water (beer) proof silicon keyboard.

Several mini DIN-8 to DB-9 serial cables...

And SCSI cables!

Curiously, no modems.

I could have left a lot of this in New Hampshire...

Consequence of Dating a Chemist:

"All I know about rare earth elements I learned sitting on the toilet." -The Girly, a vague reference to our shower curtain. In response to an NPR article about china's cornering the rare earth element market.

Since it's on Facebook, it Must Be Official!

Yes, it's true.

Somewhere, sometime, someone is going to get hitched.

Log Jam, Now with Less Sodium!

Due to dissertation related insanity, I have let my photos backlog beyond belief. I have approximately 18 months of unfiled, undeleted, and in some cases, unviewed photos.

Phocking Amazing.

Somewhat surprisingly, I appear to be just as capable of generating acceptable photos as ever. I just am not taking the time to check and create, where necessary.

Some recent examples:
I saw the light coming through the dual layered slats in the fence. I liked the glow and the break in the pattern created by the dropped lattice. A little creative cropping and desaturation helped bring this out. It violates good composition just a bit. I think it would be stronger if I cropped some of the top lattice out, but that would destroy the repetition.

Suburban Prison Cell, Herndon VA.

We went for a hike this weekend. It was amazingly relaxing to hit the trail without the guilt of a million years of school work behind me. Did I take photos? Oh yes. I didn't pay as much attention to composition as I should have, but there are still three of four hot spots. What these leaves really screamed to me was texture. The dry crinkly shape with little divots looked like used and rotted sandpaper left carelessly on the branch.

Dry Leaves. Camp Roosevelt, VA

I have a lot of trouble taking big pictures of forests. There is a certain glow of a dense forest canopy that is nearly impossible to photograph. It is this same glow that makes it so completely inviting to wander in the woods and forget that one resides in one of the most automotively, bureaucratically, and politically congested locations on the mother-loving planet.

A dim little idea dawned on me this afternoon as we wandered through the forest. I decided to try blurring the photo by moving the camera, and that I would saturate more in post processing. I am trying to show the forest, without so much detail in the trees.

Hints of Autumn. George Washington National Forest, VA

Dog Life

Bean.

Nap/Bath Time

No Front McPherson Strut

I was impressed to notice that my neighbors new-ish F250 has front and rear leaf springs. I though these things were essentially OBE, but apparently not.

F250 Front Axle

'We're not meant to have litters.' -The Girly, on fertility treatment

Found a veiled reference to 'Whats the frequency kenneth?' in code in the last part of the igor manual. Heres to the esoteric.