email sol follow sol rss feed of the blog wishlist Sol::News

Important note:

When linking to these pages, please use the URL: - it's permanent.

(1,0) (1,3) (-2,-1) (3,-1) (3,-3) (-2,-3)

System Crash, Part 2

March 22nd, 2018 #

As I had feared, my system went back to the INACCESSIBLE BOOT MEDIA loop. Like the last time, my multimedia keys stopped working, and the next time I booted (which was several days later), the boot got stuck in the bluescreen boot loop.

Given the experience I had from before, I dug up the USB boot media, messed with BIOS to boot from it, went to the recovery console and ran a few commands based on twitter conversation I had after the last time this happened.

First, I found the volume that has my system files in it. The drive letters are pretty random. I went through drives starting from C:, checking for windows directory and the correct volume label. I found my system on drive H:.

Next, I checked what the latest installed package was:

    Dism /Image:H:\ /Get-Packages

The latest package was a "rollupfix" package, so I presume it's a collection of patches which happens to include the one that blew up my PC a couple months ago. I removed the package with:

    Dism /Image:H:\ /Remove-Package /Packagename:Package_for_RollupFix~31bf3856ad364e35~amd64~~16299.192.1.9

Package names are long. As I was in the recovery console, I don't have the exact name copied down, but it was something like that.

After the package was uninstalled, I rebooted, changed boot device back from the USB stick to the hard drive in BIOS, and... the system booted.

This doesn't fix the root problem, however, and I'm pretty sure I'll find my PC in the boot loop some morning after windows has run automated updates. Sigh. At the moment I'm pretty much dreading the next cretor's update.

Case Against Attribution

February 16th, 2018 #

I've been tinkering with SoLoud again after taking a couple years off due to being burned out on the project. Long story short, I spent all my free time on it for a few years, and then got some, erm, constructive criticism from a game developer, which hurt enough that I didn't want to have anything to do with the project for a while. The while become a couple months, then a couple more, until a couple years had gone by.

SoLoud, like any non-trivial piece of software, builds upon the work of others. I have been careful to only include libraries that do not require attribution in binary form. That means that BSD license is out, and don't get me started on GPL.

Some people feel this is unreasonable. I, on the other hand, feel requiring attribution in binary form is unreasonable.

In a few cases I've reached out to the authors of the libraries and they've changed licenses to suit my needs. In some cases the authors simply did not care about the license, and had included it because someone asked them to - or just included a license because everybody else was doing it. In some cases the authors were not available, because, you know, they were dead.

If I didn't have this limitation, building an audio library like this would be much easier, but I'd rather not to force people who use my library to have to include kilometers-long legal notes in binary form.

As an example, Steam ships with ThirdPartyLegalNotices.html - currently an approximately 200kB, 4000 line file.

Now, I'll build an exaggeration to make a point. Bear with me.

Let's say that the author of a certain Linux filesystem had gotten his way and all software required a splash screen at startup for any significant piece of software used. Let's call "significant piece" 10kLOC for the heck of it. It's industry standard, everybody's doing it, so everybody's doing it. Right? The SoLoud project, including the third party libraries it contains in source form, clocks at about 130kLOC at the moment, which would mean 13 splash screens you'd have to add to every application you wish to use SoLoud. And 13 splash screens every user would have to drag themselves through every time they launch your game.

And that's, of course, just SoLoud. The GNU C library is about 1.3MLOC - I presume other C libraries are of similar sizes - and don't get me started on the game engines out there..

I've seen code of a hundred lines that requires attribution, explicitly, in binary form.

Wouldn't it be much nicer to just include the library in your code and get on with life, without having to worry about all this? In the end, most people who release source code would rather their work is used, than that it's not used.

In the end, the end users don't care.


January 31st, 2018 #

Some ten years ago I was walking the dogs with my wife, and commented that maybe I need a new goal, like having my code in the hands of a million people. My wife said that was megalomaniac. A year or so later Qualcomm shipped that many devices in a day with drivers that had my code in it. And unless Qualcomm's corporate culture has changed (which I find extremely unlikely), my code has likely touched a billion people by now, only through those drivers.

I'd like to stress that this does not make me unique by any means - those drivers alone have code from hundreds of people.

In any case that's just my corporate work. I've also worked on a bunch of open source, and I sometimes hear through third parties how my code is used somewhere. Some 3d chat used my (currently horribly outdated) virtual filesystem. A game where you build and battle robots used it too - I sent that company an email asking for comments on the library and they sent me a copy of the game. Most of the time I don't hear about the code use though. Is my code used in space? On another planet? I have no way of knowing.

SoLoud has officially shipped on all major consoles.

Maybe I need a new goal.

System Crash

January 30th, 2018 #

Yesterday I noticed that the media keys on my Natural Keyboard Pro stopped working, probably as a reaction to me switching from a broken mouse to a new one. I figured I'll just quickly boot to get things working again.

The boot resulted in a bluescreen saying INACCESSIBLE BOOT MEDIA. In itself it's a bit weird error message given that it's loaded from the boot media. After showing it a while the system rebooted and returned to the bluescreen. I let this go on for a few cycles and then figured it's not going to right itself.

I started googling about this and found a bunch of resources starting from a decade ago, and apparently the magic thing to do is to boot up with windows usb stick and hit boot repair, which (according to the discussions I found) is bound to fail. And it did.

The tool threw up a log file which says that an unfinished patch is stopping the boot, which sounds weird, and an error code of 0x490. Further googling gave more magic things to do, like running "bootrec /fixboot", which gave an "access denied" message. Several threads said that this is new from the latest win10 boot media, and using an earlier one worked. After trying a bunch of other things like setting the partition active and rebuilding boot records, I managed to get something done and the bluescreen was replaced with a dull message saying there's no operating system here.

I hunted down an older boot media (which is from several years ago, because the boot media build tool has "improved" since and only lets you build the very latest), booted in and got the fixboot command run without being denied access. After this the automated recovery tool managed to do something, the system booted.. back to the bluescreen.

There's a bunch of threads explaining how to fix the UEFI boot, but the thing is, this is a legacy system - there's no secure boot here. I also don't know what exactly broke it, as it's been a very long time since I last rebooted (just how long I have no idea).

Back in, I dunno, windows XP times when things were borked enough you could just reinstall windows on top of windows and it overwrote some system files but overall your system was left intact. I tried running upgrade on the media but that just said that upgrade is only allowed from a booted-up system. Shrug. I tried to roll back from the last system checkpoint but there was none, and rolling back to the earlier windows build also just failed.

A little progress, though; booting up now popped up boot options. Unfortunately, all of the options (like boot in safe mode) just ended up in the same bluescreen. Turning on boot logging didn't end up touching the log file, so the crash occurred before the filesystem was up. This probably also means that the drivers were fine.

As a funny side note, the most recently updated file in the windows directory (where the boot log resides) was windowsupdate.log which just contained a message stating the windows update logs are only accessible through powershell now. Which is, like, super helpful when trying to figure out what went wrong from a recovery console.

All integrity checks succeeded, including chkdsk, so it wasn't hardware failure. No popups asking me to pay bitcoin either, so probably not a virus.

Now, I bought myself an SSD as a xmas present, so I had done migration about a month ago. I figured that since there's nothing else to be done, I might as well (after taking last-minute backups of my source codes) re-do the migration. I'll lose about 40 days worth of changes, but I should end up with a working system. Until, in the worst case, Microsoft remotely borks my machine again. Let's hope not.

So here we are. We'll see how it goes. If things break down again, I may need to burn some money and build a new system, which isn't all bad, but there's a bunch of software licenses bound to this hardware I'd rather not lose.


January 25th, 2018 #

My brother had an idea, pointed me at some open food nutritious data and asked (in a roundabout way) if I could make a tool that uses said data so he could get food nutritious content estimate on home cooked foods. I said I'd take a look.

This is the result.

It's been ages since I last did any web work, and things have definitely changed a lot. To start, I dumped all the CSV data into a SQLite database. I had to massage the data a bit because the CS Vs had some extra newlines, I had to handle umlauts somehow and decimal points were wrong (Finns for some reason use , as decimal point, which had to be changed..). I ended up html-encoding all umlauts, which meant that the C in CSV became a problem.. anyway, I got it done.

Next I made a web 1.0 version with PHP. Everything in that version is server-side, and there's no CSS or anything. While that works, it's not convenient.

Then I asked on twitter about recommendations on modern web dev frameworks. People pointed me at vue, react, elm, rolling my own, or not using a framework at all. I looked at a few of these and vue looked most friendly to me. I watched a few vue tutorial video clips which let me get rolling pretty quickly. These tutorials also pointed me at Bulma CSS framework, which solved another big piece of the puzzle. Bulma doesn't do everything I needed, but luckily there are extensions.

I decided to ajax a bunch of stuff, and opted to just use fetch() even though caniuse said some current browsers don't support it. Since the target market for this was pretty well defined (i.e, my brother) I didn't really need to care. For more complete support I could have taken in some ajax library, but why bother?

Foodsofia uses two php scripts which it uses to ajax data in; the list of food items and the nutritious values of specific food items. Both of the scripts return data based on the language code (the original data had support for Finnish, Swedish and English, so those are supported). I also made sure to sanitize all the inputs to the php script to avoid the bobby tables attack.

The primary control in FoodSofia is the select box. First I used plain select, but that's annoying (not to mention slow) to use with ~3000 items, so I changed it to v-select, which has search capability. That worked fine, except that it was really slow to start. As a test I replaced it with vue-multiselect, and that was way, way faster, so I'm happy I tried it out.

After putting the more complicated select components in, I noticed that they conflict with the bulma css, so I got to learn at least one way of trying to figure it out. Using chrome's developer tools, I found the component in the DOM, investigated it's css and one by one disabled the css items until I found the one causing the glitch. Then I wrote a one-line style to disable it by hand, and the component worked. This leads me to wonder just how many of these kinds of kludges exist in modern web sites these days..

Last missing piece was being able to share the resulting links. I encoded the food table as a hex string - one character for the "used" toggle, 4 characters for food code (slight overkill) and 6 characters for the food amount (in 1/100ths of a gram - so if you enter amounts larger than 16777.216 grams, the resulting url will not work).

The url encoding was also helpful in making the language change happen. The language buttons simply reload the whole page with the url encoding and a different language code. That way I did not need to ajax every piece of data separately. Also, there's no need for cookies.

Working with vue and bulma was fun. This site's layout was last redesigned in 2010. Maybe it's time for a revamp? I've had a few starts at a redesign, but the sheer scope of this site (some ~250 pages of content) makes it a lot of work. These frameworks might make it easier.


January 19th, 2018 #

TMDC20 results are out. First place.

Got the breakdown writeup done too. And there's a pouet page.

Bookwork Adventures Aimbot

January 12th, 2018 #

Here's one I did not finish.

My most viewed video on youtube is my "Bookworm Aimbot", where a bot I wrote plays bookworm and is pretty good at it. I figured at some point that I could make a sequel to that, which plays Bookworm Adventures instead.

First order of business was to get the dictionary the game uses. Unlike with Bookworm, the dictionary wasn't conveniently as a separate (if slightly encoded) format. I figured the dictionary needs to be in memory, so I ran the game, dumped the memory using Process Explorer, and wrote a tool that finds the dictionary words and dumps them into a text file. According to some wiki about the game, Bookworm Adventures 2 knows more words, so their dictionaries are different; if I ever go that far, I would have had to extract that dictionary too.

Next up, I needed to deal with the OCR. I took screen caps of the tiles, which are both different size and have a different font than what Bookworm used. The font is more OCR-friendly, and no particles are drawn on top of the glyph either, making the OCR way easier. After some tweaking, I got it to recognize the tiles perfectly. No preprocessing filters needed or anything.

Working somewhat smarter than last time, I used the RegisterHotKey API in windows and bound my bot to run if control-f1 was pressed, and to die if control-f2 was pressed. This way if I ever lost control of the bot, I could just kill it with the hotkey. Except when I made a bug which ended up not ever checking window messages, of course, and I had to cycle through another account to kill the process. Yay. But overall, much smarter.

I wrote routine that looks for the best possible scoring words (biased towards longer words in case of a tie), and a routine to enter the word and submit it.

And then I stopped.

The game is just annoyingly slow. After every move the bot made, the game would play a slow animation before it could continue. Couldn't even rapidly play through the dialogue, there's a speed limit there too. I guess I could have beat the world record in fastest play-through, but I didn't really feel like it. I don't think the resulting video would have been all that interesting.

On top of that there's some special stages which require different tactics (like answering riddles) which would have required a lot of code.

After that I started looking for other letter tile games I could play and/or attack, and there doesn't seem to be a lot of these out there. I did (re-)discover Bonnie's Bookstore, which is also a rather old game at this point, but since popcap was bought by EA, I couldn't re-download the game anymore (yay DRM). I did contact the original authors who sent me a new key but that didn't work, so.. dunno.

The Sexy Brutale

January 12th, 2018 #

Some minor spoilers here. Game story is not discussed, some game mechanics are.

I recently finished playing The Sexy Brutale, and since it's game design brushes an idea I blogged about in 2008, or "groundhog day - the game".

The Sexy Brutale simplifies the idea a lot, but it's still there. The location is relatively small and there are only about twenty agents in the game. On top of this, there are some characters that don't really have any agency, and could be considered interactive furniture.

The second big simplification is that the player's agent doesn't interact with the other agents directly. Interaction is indirect, you manipulate items or press a button or some such, and that changes things.

Third simplification is that the agents don't really interact with each other either (except for the obvious killer-victim interaction).

Fourth, you can only change one thing (that matters) at a time.

All that said, there's still a lot there; repeating day cycle, picked up items returning to their original locations when day restarts, a lot of actions only possible at specific times of the day, learning what's going on in order to change things. For example, if you know the key code to a door, you can open it. But you first have to find it out. Once you do, there's no need to go through the trouble of finding out again, just run to the door and go.

One idea I really like is that you can change the location from where you start the day, which reduces the frustration of having to run through the same set of rooms every time the day starts. What I really dislike is that you have to wait several seconds for your character to get up every time you re-start the day before you can move. It's a small thing, but it's annoying.

The audio design in the game is also great. Different locations in the game have different audio, different soundtrack, but key events during the day play at the same time, so you learn to intuitively know what part of the day you're in. When you spy the agents talking to each other they sometimes mention these things too, like "was that a gunshot?".

When I was thinking of this kind of game I always started from the idea that the agents would be based on some kind of generic AI which knows how to do basic things like getting from A to B using whatever is the most convenient way (i.e, take the bus), and on top of that they would have a script of events they should try for ("get to work by 7am"), with some logic ("if no car keys, don't use car"). But it doesn't need to be that complex, like The Sexy Brutale shows.

The stuff I talked about in the old blog post - social billiards - does make things a bit more complex, but with limited number of agents it should still be totally manageable. Would need a lot of planning (basically designing a graph to see what affects what), but.. doable.

To make the workload smaller, the locale could be selected so that it makes sense that only a few people are about, like they did in The Sexy Brutale, such as an off-season ski resort or maybe a cruise ship. It's also not necessary to be able to affect all agents, some might come out of buildings the player can't access and leave the same way, for instance.

I'd also make it so that the player can retry things later in the day. Kinda like with save games, but if you manage to gain knowledge or a skill, those things you could keep. When restarting the day, you could pick the time where to rewind, and the game would act as if you had played exactly the same way up to that point. Plus, of course, an option to just idle for N hours if you're waiting for something specific to happen.

The AI logic would still have to be much more complex than in The Sexy Brutale, because, let's say someone comes home and finds you standing in the middle of their living room... the agent should definitely react somehow.

I believe it would totally be possible to implement. The big question is, would it be fun to play? I guess that depends a lot on the stories the game would tell.

Let's Have a Better New Year, Shall We

January 12th, 2018 #

Let's hope 2018 ends up better than 2017 was.

Not that 2017 was all bad. But it could have been better, overall. In 2017 I dabbled with doing some consulting on the side (which my full-time work contract is fine with), which then was abruptly moved aside because, after applying for years, we became foster parents. Which naturally took all of my free time and added tons of stress, but it's not all bad.

I spent most of what was left of my free time just going through steam backlog or similar low-stress things. I did manage to get the minimal things done, though, like the new year demo:

Like before, you can also read the breakdown, the pouet page, and download the win32 binaries and run it for yourself. Might be worth it if you want to watch it for hours. I did.

I also entered the 20th TMDC, but the results for that are not yet out, so... expect another blog post.

Talking of blog posts, I really should blog more, even though I doubt many people read these. Maybe that'll be the thing I'll try to do more this year. We'll see..


Older news have been archived here: 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 even older

Site design & Copyright © 2017 Jari Komppa
Possibly modified around: March 22 2018