July 2019 – Kev's Development Toolbox

July 26, 2019July 26, 2019

Installing Solaris 2.6 under QEMU

I’ve been looking at picking up a used Sun Sparcstation from eBay. It occurred to me that I’ve never installed an early version of Solaris before, so wondered if I could give it a try under QEMU since it’s emulates different hardware, including Sparc.

There’s an awesome step by step guide on Adafruit that takes you precisely each step to get Solairs installed un QEMU. You can follow the steps in their article here, so I won’t repeat all the steps here.

The key steps before you get to the install are creating a disk image:

qemu-img create -f qcow2 sparc.qcow2 9663676416

and then booting with the Solaris iso image as the cdrom and the disk image attached:

qemu-system-sparc -M SS-5 -m 128 -drive file=sparc.qcow2,bus=0,unit=0,media=disk -drive file=solaris_2.6_598_sparc.iso,bus=0,unit=2,media=cdrom,readonly=on

After this point it’s following through the steps in the install.

Here’s qemu booting up for the first time:

Here’s the Solaris installer starting up:

After the install had completed, here’s the rather impressive for it’s time CDE desktop:

July 22, 2019July 22, 2019

Updating my AWS Lambda Sudoku Solver to generate new puzzles (part 1)

Having spent some time in the past building an implementation of Donald Knuth’s Algorithm X in Java to solve Sudoku Puzzles, I recently wondered what it would take to modify it to generate new puzzles.

If you missed my previous posts on this investigation, see:

Building a React frontend for my AWS Lambda Sudoku solver

Revisiting Donald Knuth’s Algorithm X and Dancing Links to solve Sudoku puzzles

Can you develop code to effectively solve a problem without understanding the problem first? (writing code to solve Sudoku puzzles)

It turns out having a working solver is part of the way there to implementing a puzzle generator, because you need to be able to check if a puzzle has a single solution, since valid puzzles only have 1 solution.

When I last wrapped my Solver as an AWS Lambda, I had taken the naive approach to call System.exit() in my Solver code if I detected there was more than 1 solution as a quick way to exit and not get stuck in a loop iterating over finding possible thousands of solutions for a grid that’s not a valid puzzle.

I went back and took another look at this and reworked it so I could pass an upper limit for number of puzzles, and changed the return type to return a List of solutions, and a flag indicating if there was a single solution or not. Latest commits on my repo have these changes, and I’m now ready to move on to building the puzzle generator. More updates coming soon.

July 11, 2019July 11, 2019

Extracting (scraping) webpage content with Puppeteer

I’m working on a personal project to extract some content from a number of websites. This idea started out trying to use a REST api to a site but it turned out to be far more complicated involving far more steps that I was prepare to invest the time in to get working, so I realized I could just scrape their HTML content instead to get what I was looking for.

There are many HTML scraping and parsing libraries. I took a look at x-ray and a few others, but what I discovered is that many sites detect the fact that you’re not browsing the site in a real browser (presumably from checking your user-agent and cookies etc) and then force you into completing CAPTCHAs etc to prove that you’re not a robot.

I then stumbled across Apify, and more specifically Puppeteer. I think Apify does far more than I need for this little project. Instead, since Apify can also use Puppeteer under the covers, I found that just using Puppeteer directly does all that I need.

Here’s a small script for extracting some text from an example site:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://www.example-website.com');
    const extractedValue = await page.$eval('#element-id', el => el.innerHTML);
    console.log('extracted value: ' + extractedValue);

    await browser.close();
})();