C#, asp.net core 1.1, ef core, sqlite.net, sqlite
I'm creating a media database program. The goal is so I wouldn't accidentally buy games or other media multiple times on different platforms without knowing about it. I have multiple interfaces:
- Scrapes a bunch of digital services and inserts the data into a local sqlite database. This is mostly complete but a bit shoddy. It's mainly used as a scraper but has some search capabilities against the database. Pretty much nothing out there has an api or standard way to query what you bought at each of the different services besides steam. Currently, I'm using selenium to script logging into account history pages and scraping the pages. It's brittle but works unless they change the web pages.
services I support currently:
xbox (360 and xbox one), supports 2 factor logins too
xbox live gold free games. not actually in account history so I scrape a wiki page.
psn (have to solve a captcha now. sony recently added this to their sites)
uplay until recently. A change broke things. I didn't have enough bought in uplay to care to fix it.
- Allows manual data entry.
- Bulk inserts based on walking directories
- Responsive website using asp.net core mvc. I may rewrite in angular 2.0 someday.
- Asp.net core identity for security. I might integrate identityserver4 or openiddict.
- This is the main way I search the database. Also, allows for manual data entry too.
- Hosted in IIS in a vm at home on my own domain.
It's working pretty well for me and serving my needs. I started porting to sql server for better performance but I've been really enjoying working with sqlite. It's just not very performant when hitting it across a network share and WAL mode doesn't seem to do what I thought it was suppose to do. Still, most queries are < 3 sec.