The random software projects thread

Qt3 has a pretty good contingent of developers, and if the rest of you are like me, you have a whole raft of personal projects in various states of completion. As far as I’m aware, we don’t have a place to discuss those projects which don’t merit their own thread, whether they be half-baked or of limited general interest. Now we do.

I’ll kick things off with my as-yet unnamed Morse code engine, which isn’t quite ready for public release. I’ve been learning Morse over the past week or so, and although there are many excellent learning tools online as far as learning the signals for each letter goes, there are very few tools that help you develop your ear. With an eye toward eventually making one of those, I’ve been tooling around on a Java library to parse and generate Morse code. I’d eventually like to make the parser adapt to the speed of the sender, and allow the generator to send imperfect Morse in the same way humans do. (Experienced radiomen can tell different senders apart based on the peculiarities of their style with the Morse key. I’d like the generator to be that distinct.)

What about you? What random fun things are you working on?

For the last six months I’ve been working on foundational pieces at my attempt to make an open source live streaming video server. So far I have a fully working (albeit simple) RTMP server, working on getting rtmp client functionality working (so I can duplicate and relay video streams) and then converting an RTMP stream into an HLS feed.

Sounds so much more exciting in my head at least…

Well, right now, I barely have free time to remain sane. :-) But there are a lot of things I want to do, I just didn’t find the time (or the motivation) to do so.

Anyway, an ongoing project of mine is a budgeting and spending tracking web software, using Elm and UIKit client-side, and something else on the server side. Ideally, I’d use Elixir/Phoenix or Rust, since I’m looking for a good opportunity to use those, but I’ll most likely just use plain Python - Flask/SQLAlchemy to be more precise - to implement a simple JSON-based API.

Not really random, since it’s bound to a fellowship that I have, but I’m a member of the Lancet-American University of Beirut Commission on Syria team as a data scientist with the FXB Center.

…The Lancet and the American University of Beirut have together established the Commission on Syria: Health in Conflict. The aim of the Commission is to describe, analyse, interrogate, and decry the calamity before us. The lens is health and wellbeing, always a productive way to assess grave issues of high mortality and morbidity, disruptions of home, family, settlement, environment, and such extensive loss that the future itself is hard to discern. With this Commission, we have embarked on the difficult effort to identify these costs and enumerate them where possible. Hence, the first task ahead is to account for the burden of war. We will also examine the challenges of the international response to the crisis and learn the lessons for future crises. The Commission will develop concrete recommendations to address the unmet current and future health needs, including those related to rebuilding and to strengthening the global health response to political conflict.

It sort of is a random software project because I have a full time job and do (unpaid) work for this in addition to that. The kickoff meeting is in 10 days, so I’m working on a few websites that serve data tools from previous germane projects.

One of the more interesting ones was a “data mapping” project done in one of the states in India. A bunch of medical students and residents collaborating with our team visited all of the community, district, and state public health clinics there and recorded which data points are collected about patients and which registries (often paper) they are recorded in. There’s a “flow” of the data in the registries, where they are submitted to offices at higher levels, etc. The point of the project was to try to identify redundancies that potentially can be eliminated with a more centralized and mobile-accessible health information system. As part of that project, I took all of the data they collected and performed network analysis on it. I also took a graph structure of the flow of the data through the registries and used sigma.js to create a navigable map of the data hierarchy. It was the first phase in the project and was pretty interesting.

Other than that, I keep plugging away at my ten-minute game jam concept, when I have some time. I wish I had more time to play with that.

Very much this.

Work is keeping me very tied up right now, but in my occasional free cycle, I’m working on an instructional script compiler/virtual machine project for embedding into game engines.

I started a Minecraft mod, to make “ages” or “epochs”, something like Age of Empires. Most relevant for heavily modded Minecraft servers. You should have a server community goal before you unlocked a new range of tools, blocks and items. And then a new set of goals, to advance to the next epoch, with new unlocks, and so forth… The son burnt out on Minecraft though, and I don’t have much time to play myself, so it’s kind of in stasis. I still like the idea though.

Another project, that I was only just investigating, came from a problem I frequently run in to. My friends, family and me have very different music tastes. So when we meet up, depending on who’s there, the “acceptable” music choices vary greatly. If me, friend1 and friend 2 is there, we can play a lot of heavy metal. If friend 3 is present, a lot less heavy and none of the trashy stuff. And if the wife is there, no heavy metal at all. I would really like a mobile app, that kind of automatically detected who was in the vicinity and made playlist choices on Spotify based on that information. Early investigation has revealed, that this is a hugely complicated task though. Not the proximity thing - but the music taste matching. It’ll probably end in the overfilled half cooked ideas drawer, but for now I like to ponder over it from time to time.

I’m on-and-off-again working on a computer implementation of Lost Worlds. This is a combat game-book system where each player picks an action, cross-references with their opponent’s action in a book to get to a result page. Lots of different characters with different strengths and weaknesses, no luck involved.

I’ve got two books implemented, and with the mechanisms I’ve implemented so far, I could sit down in a couple hours and easily add another half dozen, possibly even another couple dozen beyond that without the need to write much more code to handle special features. I’m holding off adding more for now because if I make any changes to the underlying systems, then I’ll have to update all currently implemented books, and stuff is still a little bit in flux. The game is technically playable between any combination of two humans or AIs, but it certainly doesn’t look pretty, and doesn’t actually handle the game ending yet (the combat engine knows the game is over, but the UI does respond to that state change, so hit points just go negative).

I only have a little more work to do, and I’ll be able to start playing around with more interesting AI behaviors. For now one AI just selects random legal actions, and the other looks ahead at possible outcomes and picks a random action weighted towards ones that result in the most damage. AI work is really my next short-term goal on the project. I’d like to be able to support AI plugins in a scripting language of some kind.

I’ve also made sure to implement things in such a way that the system isn’t hard-wired to the Lost Worlds rule set. That’s pretty important, since I’m sure I’ll never be able to distribute actual Lost Worlds stuff. So, at some point I’ll design my own system. That’s some ways off in the future though.

That sounds great. What language/framework/engine are you using to do all that?

I’m using C#. The UI is using WPF, but I’m intentionally setting things up in layers. The core engine reads in the book definitions and handles interactions between the players. This is not tied to any particular ruleset definition, so you could use it for Lost Worlds or any other custom game. It’s even isolated in its own assembly to help keep me from making dependencies where I shouldn’t. The rules/books are purely defined in data files, so everything stays nice and isolated. In theory it shouldn’t be hard to replace the UI layer and keep the same game under the hood.

Really for now the UI is just so I can have something simple in place to develop and test and play around with things. If I ever distribute, I’ll likely reimplement the UI from scratch.

I really want to do a mobile and/or web-based implementation, since I think the concept is really well suited to those platforms.

I’d like to know more, since web-based implementations are kind of my (current) expertise. Sort of. ;)

:) Well, a web-based implementation is pretty far out there for this side-project. If I did start working on it, I’d likely be looking at a React solution with… not sure on the back end. ASP.NET? This kind of solution would give me an opportunity to learn more about the kind of web development that happens where I work.

I think the gameplay style fits a mobile or web solution nicely because:

  1. Multi-player can proceed as fast as both users submit an action for the next turn. This could be a kind of thing where you log in and submit turns for the half dozen matches you currently have running. If the other user is currently on, it would go fast, but it doesn’t have to be.
  2. In-app purchasing is a very nice fit. You can play against any other character, but if you want to play as a character, you need to purchase an unlock. It’s a very natural and discrete purchase.
  3. There are possible hooks for leveling up your characters and earning equipment, so there is some sense of progression in an arena-type environment.

Still, that’s all pie-in-the-sky. I don’t know that I’ll ever get there. It’s mostly an excuse to learn some new things, with the benefit that I get to play a game a love (Lost Worlds) even if I don’t have anyone to play against. :)

C#, asp.net core 1.1, ef core, sqlite.net, sqlite
I’m creating a media database program. The goal is so I wouldn’t accidentally buy games or other media multiple times on different platforms without knowing about it. I have multiple interfaces:

CLI:

  • Scrapes a bunch of digital services and inserts the data into a local sqlite database. This is mostly complete but a bit shoddy. It’s mainly used as a scraper but has some search capabilities against the database. Pretty much nothing out there has an api or standard way to query what you bought at each of the different services besides steam. Currently, I’m using selenium to script logging into account history pages and scraping the pages. It’s brittle but works unless they change the web pages.

services I support currently:
steam
xbox (360 and xbox one), supports 2 factor logins too
xbox live gold free games. not actually in account history so I scrape a wiki page.
psn (have to solve a captcha now. sony recently added this to their sites)
gog
humble bundle


uplay until recently. A change broke things. I didn’t have enough bought in uplay to care to fix it.

Windows forms:

  • Allows manual data entry.
  • Bulk inserts based on walking directories

Web:

  • Responsive website using asp.net core mvc. I may rewrite in angular 2.0 someday.
  • Asp.net core identity for security. I might integrate identityserver4 or openiddict.
  • This is the main way I search the database. Also, allows for manual data entry too.
  • Hosted in IIS in a vm at home on my own domain.

It’s working pretty well for me and serving my needs. I started porting to sql server for better performance but I’ve been really enjoying working with sqlite. It’s just not very performant when hitting it across a network share and WAL mode doesn’t seem to do what I thought it was suppose to do. Still, most queries are < 3 sec.

Nice project! And yeah, sqlite is awesome, though it’s designed to be used on a local filesystem.

Android uses sqlite as its default database provider. It’s a nice piece of software. All of of these sound really cool, although I also relate to the ‘need more free time’ thing.

I did spend most of my Saturday morning working on podcasting setup, which has a software angle—I have two good USB microphones, and through the magic of JACK, managed to come up with a setup which allows me to record both on the same computer, as two channels of the same input device, with real-time monitoring. Four hours ago, I thought that was impossible.

JACK is amazing. I remember its early days, when good, low-latency multimedia work on Linux was but a wet dream. I wish I had free time to do more multimedia things on Linux. :)

I must be the only one who hasn’t had a great experience with sqlite. Several years ago I coded a computer metric logging system that would constantly log any metrics it could find (cpu temp, core speeds, fan speeds, gpu temps, etc…) and also what processes were running. The point of the app was so I could use a WPF tool to generate graphs that allowed me to see things like “how has the average CPU temperature changed while playing Borderlands” (so I knew if my computer’s temperature was starting to rise over time).

Every once and a while the database would just empty and restart. I never did track it down though so I guess I don’t know for sure if it was sqllite’s fault or my own.

For the record, I once used sqlite embedded in a network analyzer software to track VoIP statistics in real-time, with no significant issues (and no data loss ever).

Yeah who knows. To be fair when I said I never tracked it down I really mean I never tried to track it down. By the time I noticed it happening I realized I never really cared about that data anyways so I never bothered to look into it more.

Update: I was arrogant, and I have been brought low.

That xkcd is awesome.

Even though I do 100% .net development at work I got annoyed with windows and put Linux on my work laptop. For the most part it went fine until a week or so later I got some error saying that some signature didn’t match and I couldn’t boot. I kept booting off of live cds trying to futz with it to fix it and kept failing until I realized the error message was coming from Dell’s secure boot. Turns out it just lost the boot signature and couldn’t verify it so I had to turn secure boot off. By that point I already messed with the system too much and it was too late to salvage, and a reinstall was in order. That whole ordeal wasted 3/4th of my day.